Skip to main content
Version: v0.14

Customize

Summary

This plugin provides users the ability to create/delete columns and extract data from certain raw layer tables. The columns created with this plugin must be start with the prefix x_

NOTE: All columns created by this plugin are of the datatype VARCHAR(255)

Sample Request

To extract data, switch to Advanced Mode on the the first step of creating a Blueprint and paste a JSON config as the following:

The example below demonstrates how to extract status name from the table _raw_jira_api_issues and assign it to the x_test column of the table issues. We leverage the package https://github.com/tidwall/gjson to extract value from the JSON. For the extraction syntax, please refer to this docs

  • table: domain layer table name
  • rawDataTable: raw layer table, from which we extract values by json path
  • rawDataParams: the filter to select records from the raw layer table (The value should be a string not an object)
  • mapping the extraction rule; the key is the extension field name; the value is json path
[
[
{
"plugin":"customize",
"options":{
"transformationRules":[
{
"table":"issues",
"rawDataTable":"_raw_jira_api_issues",
"rawDataParams":"{\"ConnectionId\":1,\"BoardId\":8}",
"mapping":{
"x_test":"fields.status.name"
}
}
]
}
}
]
]

You can also trigger data extraction by making a POST request to /pipelines.

curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "extract fields",
"plan": [
[
{
"plugin": "customize",
"options": {
"transformationRules": [
{
"table": "issues",
"rawDataTable": "_raw_jira_api_issues",
"rawDataParams": "{\"ConnectionId\":1,\"BoardId\":8}",
"mapping": {
"x_test": "fields.status.name"
}
}
]
}
}
]
]
}
'

Get all extension columns(start with x_) of the table issues

GET /plugins/customize/issues/fields

response

[
{
"columnName": "x_test",
"columnType": "VARCHAR(255)"
}
]

Create extension column x_test for the table issues

POST /plugins/customize/issues/fields

{
"name": "x_test"
}

Drop the column x_text for the table issues DELETE /plugins/customize/issues/fields/x_test