Sleep Time
Overview
Sleep Time operation introduces a controlled pause in pipeline execution for a user-defined number of seconds.
Description
This operation helps delay the execution flow, enabling synchronization, rate control, and dependency management within data pipelines.
Number of Parameters
1
Parameter: Sleep Time
Specifies the pause duration in seconds.
60
Python Operation
Overview
Python Operation allows users to implement custom logic for data transformation when predefined operations are insufficient.
Description
Users can write Python scripts to modify, enrich, and process pipeline data dynamically using predefined system variables.
Important Parameters
- Responsedata: Initialized as an empty list to store results.
- pycode_data: Contains the incoming pipeline data.
Number of Parameters
1
Parameter: Pycode
Defines the Python script used for data transformation.
Input Data Example
{
"attributeid": 212077,
"attributename": "item_no",
"attributevalue": "999898"
}
Python Code Example
Responsedata = []
keyname = ["attributename"]
keyvalue = ["attributevalue"]
for i in keyname:
attributename = pycode_data.pop(i, None)
for j in keyvalue:
attributevalue = pycode_data.pop(j, None)
pycode_data[attributename] = attributevalue
Output Example
{
"attributeid": 212077,
"item_no": "999898"
}
Filter Operation
Overview
Filter Operation applies conditional logic to control data flow using if-else expressions.
Description
This operation evaluates logical expressions and filters records based on defined conditions.
Parameter: Conditional Statement
Defines logical expressions using operators and data references.
Supported Operators
- ==, !=, <, >, <=, >=
- and, or, not
- in, not in
- is, is not
Examples
data['Order Status']=='pending'
'id'==5 or data['id']==6
'ipAddress' not in data
AI Operation
Overview
AI Operation transforms data using natural language instructions.
Description
Users provide plain English instructions that define how input data should be modified. The system generates and executes the logic automatically.
Number of Parameters
1
Parameter: Instructions
Defines the natural language transformation rule.
Example: Grouping Data
Group the given order data into customer, product, shipping, and financial sections.
Text to Operations
Overview
Text to Operations enables data transformation using either plain English instructions or direct Python code.
Description
This operation converts user instructions into executable Python logic or runs user-provided scripts in real time.
How It Works
Natural language prompts are translated into Python code using backend intelligence. Users may also supply their own code.
System Parameters
- Responsedata: Stores operation results.
- pycode_data: Holds incoming dataset.
Number of Parameters
1
Parameter: Instruction
Accepts either plain English instructions or Python scripts.
Example: Convert Last Name to Lowercase
From the source data, convert the value of the last_name key to lowercase.
Generated Code
Responsedata = []
for user in pycode_data['bizdata_dataset_response']['data']:
user['last_name'] = user['last_name'].lower()
Responsedata.append(user)
Forward Code Option
Forward Code is a UI toggle that allows generated Python code to be reused in subsequent operations.
Frequently Asked Questions
What is the purpose of Sleep Time?
It pauses pipeline execution for a specified duration to control processing flow.
Why is Responsedata required in Python Operation?
It stores processed results and enables structured output handling.
Can Filter Operation use multiple conditions?
Yes, logical operators such as and, or, and not can be combined.
Does AI Operation require coding?
No, users can provide instructions in natural language.
Can Text to Operations run custom scripts?
Yes, it supports both auto-generated and user-written Python code.
Notes
- Validate all custom scripts before production use.
- Ensure sensitive data is handled securely.
- Test pipeline logic thoroughly using preview features.
