Data lake ingestion as Data target allows you to efficiently transfer and store data from various sources directly into the Bizintel360 Datalake. After configuring Data Source and Data operations for the integration bridge configuration, we navigate to the Data Target configuration.
Steps to Configure
Step 1: Select Target Type
Select “Bizintel360 Data Lake Ingestion” in Select Target type option from Target Type drop down list.
Step 2: Select Data Lake Version
Select the required Datalake version from the drop-down menu as shown
Step 3: Add Index Name/Table Name
Write the Index or table name in which the data should be ingested.
Step 4: Select Action Type
Select the action type from the drop-down menu as per the requirement. Below mentioned is
the description for all the action types available for selection:
• Upsert- The “Upsert” action type combines “update” and “insert” functionalities, allowing data to be updated if it exists or inserted if it doesn’t, streamlining data management.
• Update- The “update” action type modifies existing data in the database, providing the ability to change specific values within a record.
• Delete- The “update” action type delete or removes specific set of data or entire record count from the datalake.
• Create- The “create” action type initiates the addition of new records or entities into datalake.
• Insert- The “insert” action type specifically adds new data into a database, appending records or entities into existing datasets.
Step 5: Insert Primary Key
Define the Primary key of the Index which will help in reducing data duplication. Primary key should be inserted in the case of UPSERT, UPDATE and DELETE.
Step 6: Select Ingestion Type
Select the ingestion type from the drop-down menu as per the requirement. Below mentioned is the description for all the ingestion types available for selection:
• Parallel Computing- Parallel computing as an ingestion type involves simultaneously processing and inputting large volumes of data across multiple computational resources for faster data intake and processing.
• Streaming Computing- Streaming computing as an ingestion type involves continuous and real-time processing of data as it flows into a system, enabling immediate analysis and action on incoming data streams.
• Bump Computing- Bump computing as an ingestion type involves ingesting data one batch at a time, based on batch defined in source bumps the data into Data Lake.
