Data Source Management

The Data Source Management page gives you full control over all indexes stored in the Bizdata Datalake that belong to you or your organization. Here you can create, view, edit, share, delegate, truncate, or delete data sources (indexes) used across Integration Bridges

Only indexes with the following prefixes are managed here:

• bizintel360-*
• goldfinch-*

Accessing Data Source Management

Go to the left sidebar menu and select Data Source Management

Main List View

The page displays a table of all your data sources with the following capabilities:

• Search: Type to instantly filter data sources by name.
• Refresh: Reload the list (useful when a new index is being created or data is being loaded and you want to see the latest record count).
• Add: Create a new data source (index).

Actions available for each data source:

• Edit: Modify settings, columns, or query data.
• Truncate: Remove all documents while keeping the index structure (confirmation prompt appears).
• Delete (Drop): Permanently delete the entire index and its data (confirmation prompt appears).
• Share: Share the data source with User Groups in your organization.
• Delegate: Transfer ownership/permissions to another user in your organization.

Creating a New Data Source

1. Click Add.

2. On the Create page (Columns tab is active by default):

   • Prefix (dropdown): goldfinch or bizintel360
   • Name: Enter the custom part of the index name (lowercase only). The full name will be {prefix}-{your-name}.
   • Containers: Dropdown (1–5, default: 1)
   • Replicas: Dropdown (1–5, default: 1)
   • Columns: Add fields with:
     • Column Name
     • Data Type (Long, Text, Date, Vector, Binary)
     • Note: Only one Vector field is allowed per index.

3. Click Submit to create the index.

    The Data tab remain disabled until the index is successfully created.

Tip: Containers decide how your data is divided into parts. Adding more containers can make the system work faster when you have a lot of data. For most situations, 1–3 containers are enough.

Tip: Replicas are extra copies of your data. They help keep your data safe and make searching faster. In live environments, use 1–2 copies. For testing, just 1 copy is enough.

Viewing and Querying Data (Data Tab)

1. Click Edit or View on an existing data source.
2. Switch to the Data tab.
3. A default query is provided:
   json
   1

   { "query": { "match_all": {} } }
4. Run the query to see all documents, or modify it to filter results as needed. read more

Sharing a Data Source

1. Click the Share icon.
2. A popup displays all the User Groups you belong to.
3. Select the desired groups.
4. Click Submit

Delegating a Data Source

1. Click the Delegate icon.
2. A popup lists all Users in your organization.
3. Select the user.
4. Click Submit

Truncating or Deleting

• Truncate: Clears all data but preserves the index and mappings (ideal before reloading fresh data).
• Delete: Permanently removes the entire index (irreversible).

Both actions require user confirmation.

With Data Source Management, you have full lifecycle and access control over your organization’s indexes in the Bizdata Datalake.