Upper Case
Overview
Upper Case converts specified key values to uppercase while data is in-flight.
Number of Parameters
1
Parameter: Uppercase
Provide comma-separated keys in double quotes.
"first_name","last_name"
Lower Case
Overview
Lower Case converts specified key values to lowercase during data processing.
Number of Parameters
1
Parameter: Lowercase
"first_name","last_name"
Data Type
Overview
Data Type converts string values into Boolean, Float, Integer, or DateTime data types.
Number of Parameters
4
Boolean
"test_passed"
Float
"Amount"
Integer
"Quantity"
Date Time
"startweekdate1","%Y-%m-%d %H:%M:%S.%f%z","%Y-%m-%d %H:%M:%S", "startweekdate2","%Y-%m-%d %H:%M:%S","%Y-%m-%d"
Goldfinch Datalake Date Format:
%Y-%m-%dT%H:%M:%S.%f%z
Append
Overview
Append adds new keys with static or dynamic values during data flow.
Number of Parameters
1
Examples
"export_flag_y":"Y","export_flag_p":"P"
"concatenate_key_name":"{%ORDERNUMBER%}|{%ORDER_TYPE%}"
Title Case
Overview
Title Case converts specified key values into title case format.
"amount","first_name"
Data Extractor
Overview
Extracts specified keys and their values from JSON responses.
"['access_token']","['feedDocumentId']"
Trim
"first_name"
JSON to String / String to JSON
"key1","key2"
JSON to XML / XML to JSON
product_data_response data_response
Base64 Encoding / Decoding
"email"
Generate Array Sequence Number
key_name DATA
Today Timestamp
%Y-%m-%dT%H:%M:%S.%f%z dl_insert_date
Calculator
"Amount1-Amount2","Amount1+Amount2" "key1","key2"
Grok Pattern
Overview
Grok extracts structured data from unstructured text using predefined patterns.
This is endpoint url %{URI:endpoint_url} for mac add %{MAC:mac_address} and v4 %{IPV4:ip_address_v4} and V6 %{IPV6:ip_address_v6}
PDF Extractor
Items @xyz.grapgh.downloadUrl
ARRAY COUNT
['bizdata_dataset_response']['data']
RAW SENTENCE GENERATOR
"Name","Commands" Response
TIME UNITS
timestamp "year","month","day","hour","minute","second","microsecond"
Data Chunking
['text'] chunks 1000 Data_Chunks
Extract to Array Operation
"data" ["id","content"] ["ids","documents"]
Frequently Asked Questions
What does the Upper Case operation do?
The Upper Case operation converts the values of specified keys into uppercase format while the data is in-flight within the pipeline.
What does the Lower Case operation do?
The Lower Case operation converts the values of selected keys into lowercase format during data processing.
How does the Data Type operation work?
The Data Type operation converts string values into their respective data types such as Boolean, Float, Integer, or DateTime based on the provided configuration.
When should I use the Append operation?
Use the Append operation when you need to add new keys with static values or dynamic values derived from existing pipeline data.
What is the purpose of Title Case?
Title Case converts the values of specified keys so that each word starts with an uppercase letter.
How does Data Extractor help?
Data Extractor retrieves specific keys and their corresponding values from a JSON response for further processing.
What is the difference between JSON to String and String to JSON?
JSON to String converts structured JSON data into a string format, while String to JSON parses a string and converts it into structured JSON.
When should JSON to XML or XML to JSON be used?
These operations are used when converting data between JSON and XML formats to meet integration or system requirements.
Notes
- Validate key names before execution.
- Ensure correct datatype formats during conversion.
- Test transformations using sample datasets.
