Single Line to Multiline
Description :
Single Line to Multiline operation is used to convert Single Line JSON to Multiline JSON.
The resulting JSON string can be used to store or transmit data in a structured format.
Number of Parameters : 1
Parameter : Chopkey
Chopkey Adds a new key on the fly with its value as a static value or dynamic value.
Below is an example where we are using chopkey operation to add new key.
['bizdata_dataset_response']['data']
Delimiter to JSON
Description :
Delimiter to JSON operation is used to convert Delimiter data to JSON data.
The resulting JSON string can be used to store or transmit data in a structured format.
Number of Parameters : 6
Parameter : Key Data
Key Data holds the value of delimiter data which we want to convert into JSON data.
Below is an example where we are using DataTable as key.
['DataTable']
Parameter : Delimiter
Delimiter is used to separate values in a list, record or file.
Below is an example where we are using , as delimiter.
,
Parameter : Fields
Fields is used in to specify headers name, it is not mandatory if the user wants then they can specify the fields else it will be passed as empty and pre defined fields will be generated.
Below is an example where we are using headers Customer ID, Organization Name, Month and Item as header name.
"Customer ID","Organization Name","Month","Item"
Parameter : Autodetect Column Names
Autodetect Column Names if the user defines fields then it will be false else it will be true.
Below is an example we are taking true which is for pre defined field.
true
Parameter : Skip Header
If user defines the fields it will be true else it will be false.
Below is an example we are taking false which is for predefined fields.
false
Parameter : Response Key
Response Key is the key in which your delimiter data will be stored.
Below is an example where the JSON data will be store datatable.
datatable
JSON to Delimiter
Description :
JSON to Delimiter is used to convert JSON data to Delimiter data.
The resulting Delimiter data can be used to store or transmit data in a structured format.
Number of Parameters : 6
Parameter : Key Data
Key Data holds the value of JSON data which needs to be converted into delimiter data.
Below is an example where you provide the key name [‘items’] which holds your JSON data.
['items']
Parameter : Delimiter
Delimiter is used to separate values in a list, record, or file.
Below is an example where we using \t as delimiter.
\t
Parameter : Fields
Files is used in to specify headers name, it is not mandatory if the user wants then they can specify the fields else it will be passed as empty and pre defined fields will be generated.
Below is an example where we are using values like Item1,Item2 and Item3 as header name.
"Item1","Item2","Item3"
Parameter : Autodetect Column Names
Autodetect Column Names if the user defines fields then it will be false else it will be true.
Below is an example we are taking true which is for pre defined field.
true
Parameter : Skip Header
If user defines the fields it will be true else it will be false.
Below is an example we are taking false which is for predefined fields.
false
Parameter : Response Key
Response Key is the key in which your delimiter data will be stored.
Below is an example where the JSON data will be store delimiter_data.
delimiter_data
Data Aggregation
Description:
Data aggregation operation is used for the processing of raw data, this will also help in the grouping, summarizing, and processing of the data to make it easier to understand and analyze.
Number of Parameters : 3
Parameter : Agg Data Key
Agg Data Key is passed as empty for multiline data and will have data key in case of single line data.
Use this parameter when you have multi-line json data else leave blank
Below is an example where we using [‘bizdata_dataset_response’][‘items’] as Agg Data Key for singleline data
['bizdata_dataset_response']['items']
Parameter : Groupby Key
Groupby Key gives key name that the user wants to group by, basically this is a unique identifier in the dataset.
Below is an example where we using Orders as Groupby Key as a unique identifier of a dataset which is having Order and Order Line Items, for the one Order we can have multiple items.
"Orders"
Parameter : Array Key
Array Key gives the key where the user wants to hold the common keys, the user can provide any key name
Below is an example where Order Lines holds a common key.
"Order Lines"
Parameter : Array Key Nested Columns
Give the key name as comma separated keys that should be available inside the Array Key
Below is an example where we using id, name and year as column names
"id","name","year"
Unpivot
Description:
An unpivot operation is used to convert single object into list of object based on transposed values tracking transpose key name parameters.
Number of Parameters : 2
Parameter : Transposed Key Name
Transposed Key Name used in specify name of key which needs to be transposed
Below is an example where we using bucket_type and bucket_value as values that needs to be transpose.
"bucket_type","bucket_value"
Parameter : Transposed Value
Transposed Value used in specify the values which needs to be transposed
Below is an example where the transpose values are on_hand, purchase_orders and goods_in_transit
"on_hand", "purchase_orders","goods_in_transit"
Pivot
Description:
In pivot operation we are combining multiple dictionaries (object) into a single dictionary (object) based on the transposed value and the transposed key name provided by the user.
Number of Parameters : 3
Parameter : Get Key
Get key is will be passed as empty for multiline data and will have get key in case of single line data.
Below is an example where we are using get key as items because of singleline data.
items
Parameter : Transposed Key Name
Transposed Key Name specifies name of the key which needs to be transposed.
Below is an example where we are using Item Id and Item Name as key name.
"Item Id","Item Name"
Parameter : Transposed Value
Transposed value specifies which values needs to be transposed.
Below is an example where we are using some particular item ids to be transposed OrderID-1, OrderID-2, OrderID-3
"OrderID-1", "OrderID-2"
Example Scenario :
Consider the following input data representing a dataset with various attributes.
{"bizdata_dataset":{
"id":123,
"name":"sample",
"lastname":"dataset",
"attributes":[
{
"attributename":"item",
"attributevalue":"27",
"attribute_code":12234
},
{
"attributename":"item2",
"attributevalue":"47",
"attribute_code":12334
},
{
"attributename":"item1",
"attributevalue":"37",
"attribute_code":13234
}]}}
Pivot Operation Parameters:
Parameter : Get Key
attributes
Parameter : Transposed Key Name
"attributename"
Parameter : Transposed Value
"attributevalue"
Sample Output :
After applying the Pivot Operation, the input data will be transformed as follows:
{"bizdata_dataset":{
"id": 123,
"name": "sample",
"lastname": "dataset",
"attributes":[
{
"attributename": "item",
"attributevalue": "27",
"attribute_code": 12234
},
{
"attributename": "item2",
"attributevalue": "47",
"attribute_code": 12334
},
{
"attributename": "item1",
"attributevalue": "37",
"attribute_code": 13234
}],
"item": "27",
"item2": "47",
"item1": "37"}}
Explanation of Output :
- The original array of attributes remains unchanged.
- The attributename values (“item”, “item2”, “item1”) have been transposed to the root level of the bizdata_dataset object as keys.
- The corresponding attributevalue values (“27”, “47”, “37”) are now associated with the newly created keys (“item”, “item2”, “item1”).
Single Line to Tuple
Description:
Single Line to Tuple operation is used to convert a single line of data to a tuple.
Number of Parameters : 3
Parameter : Singleline Key
Singleline Key helps in reading the dataset from a single line.
Below is an example where we are using a key DataTable holds the single line data.
['DataTable']
Parameter : Table Headers
Table Headers specifies the sequence of the converted tuple data.
Below is an example where we are using a key names Item, Customer and Month the values of these key names will appear in same sequence in the tuple data.
"Item","Customer","Month"
Parameter : Tuple Key
Tuple Key is the key which is holds tuple data.
Below is an example where we are using data as tuple key.
data
Tuple to Single line
Description:
Tuple to Single line operation is used in convert a tuple into a single line, which involves taking a tuple and converting it to a single line string.
Number of Parameters : 3
Parameter : Tuple Key
Tuple Key is used to read the user’s tuple data.
Below is an example where we are using Data which holds the tuple data .
['Data']
Parameter : Headers
Table Headers are the headers or key name of the new JSON.
Below is an example where we are using key name’s Item, Customer and Month as the values of these key names will appear in same sequence in the singleline data.
"Item","Customer","Month"
Parameter : Singleline Key
Singleline Key helps in to storing the converted singleline data.
Below is an example where we are using datatable to store the singleline data.
datatable
Delimiter to Array
No of Parameters:- 2
Parameter:- Dl Key
Converts a given key’s delimited value into array.
Below is the example how we can use Dl key.
"email"
Parameter:- Delimiter
Give the delimiter used to separate the delimited values. Delimiter can be any of “,” , “/t”, “|” etc.
Below is the example how we can use Delimiter.
“,”
Example:-
Input = {“data”:”a,b,c,d”}
Output = [a,b,c,d] .
In above example, Delimeter will be “,” and Dl key will be “data”.
How it works : From the above example we can understand that this operation is used to convert delimeter separated values to a array.
Zipfile in Base64
No of Parameters:- 5
Parameter: Source Key
This key contains all the records.
In the example below, “items” serves as the Source Key.
"items"
Parameter: File Name Key
This key holds the file name.
In the given example, “FILE_NAME” will serve as the key for the File Name Key.
"FILE_NAME"
Parameter: File Extension Key
This key contains the file extension.
In the example below, “EXTENSION” will act as the key for the file extension.
"EXTENSION"
Parameter: File Data Key
This key contains the file’s data.
In the example below, “FILE_DATA” is designated as the key for the file’s data.
"FILE_DATA"
Parameter: Base64 Response Key
This key holds the ultimate base64 encoded string of a zip file.
In the example below, “File_string” is designated as the key for the Base64 Response key.
"File_string"
Example:
Input = {“data “: {“items”: [{“FILE_NAME”: “file_01”, “EXTENSION”: “.csv”, “FILE_DATA”: “bnIsdGVzdGluZyxvcHMNCjEsZmlsZTEsemlwb3BzDQoyLGZpbGUxLHppcG9wcw0KMyxmaWxlMSx6aXBvcHM=”},
{“FILE_NAME”: “file_02”, “EXTENSION”: “.tsv”, “FILE_DATA”: “bnIJdGVzdGluZwlvcHMNCjEJZmlsZTIJemlwb3BzDQoyCWZpbGUyCXppcG9wcw0KMwlmaWxlMgl6aXBvcHM=”},
{“FILE_NAME”: “file_03”, “EXTENSION”: “.psv”, “FILE_DATA”: “bnJ8dGVzdGluZ3xvcHMNCjF8ZmlsZTJ8emlwb3BzDQoyfGZpbGUyfHppcG9wcw0KM3xmaWxlMnx6aXBvcHM=”}]}}
Output = [“File_string”: “Encoded zip file string”] .
