Data Source Management

Overview

Data Source Management provides full control over indexes stored in the Bizdata Datalake that belong to you or your organization.

From this page, users can create, view, edit, share, delegate, truncate, or delete data sources (indexes) used across Integration Bridges.

Only indexes with the following prefixes are managed here:

• bizintel360-*
• goldfinch-*

When to Use

Use Data Source Management when you need to control the lifecycle, structure, or access permissions of indexes in the Bizdata Datalake.

• Create new indexes for integration workflows
• Modify index structure or column definitions
• Query stored documents
• Share data sources with user groups
• Delegate ownership within the organization
• Truncate or permanently delete indexes

Accessing Data Source Management

To access Data Source Management, go to the left sidebar menu and select Data Source Management.

Main List View

The main page displays a table of all available data sources with search, refresh, and management actions.

Available Controls

• Search: Filter data sources instantly by name.
• Refresh: Reload the list to view updated record counts or newly created indexes.
• Add: Create a new data source (index).

Actions for Each Data Source

• Edit: Modify settings, columns, or query data.
• Truncate: Remove all documents while preserving the index structure (confirmation required).
• Delete (Drop): Permanently delete the entire index and its data (confirmation required).
• Share: Share the data source with user groups in your organization.
• Delegate: Transfer ownership or permissions to another user.

Creating a New Data Source

Click Add to create a new index.

On the Create page, the Columns tab is active by default.

Configuration Fields

Click Submit to create the index.

The Data tab remains disabled until the index is successfully created.

Configuration Tips

• Containers: Determine how data is divided. For most use cases, 1–3 containers are sufficient.
• Replicas: Additional copies of data for reliability and performance. Use 1–2 in production and 1 for testing.

Viewing and Querying Data (Data Tab)

To view stored documents:

1. Click Edit or View on an existing data source.
2. Switch to the Data tab.
3. Run or modify the default query.

Default query:

{ "query": { "match_all": {} } }

This query retrieves all documents. Modify the query to filter results as required.

Sharing a Data Source

To share a data source:

1. Click the Share icon.
2. Select one or more user groups from the popup.
3. Click Submit.

Delegating a Data Source

To delegate ownership or permissions:

1. Click the Delegate icon.
2. Select a user from your organization.
3. Click Submit.

Truncating or Deleting

Two data removal options are available:

• Truncate: Removes all documents but preserves the index structure and mappings.
• Delete: Permanently removes the entire index and all associated data.

Both actions require confirmation before execution.

Troubleshooting

• Ensure the index name follows lowercase naming requirements.
• Verify prefix selection before submission.
• Confirm only one Vector field is defined.
• Use Refresh if newly created indexes are not visible.
• Check permissions if Share or Delegate options are unavailable.

Frequently Asked Questions

What is Data Source Management?

Data Source Management is a centralized interface for creating, managing, sharing, and controlling indexes in the Bizdata Datalake.

Which indexes can be managed here?

Only indexes with prefixes bizintel360-* and goldfinch-* are available for management.

What is the difference between Truncate and Delete?

Truncate clears all documents while keeping the index structure. Delete permanently removes the entire index and its data.

How many Vector fields can be created per index?

Only one Vector field is allowed per index.

When does the Data tab become active?

The Data tab becomes active after the index is successfully created.

Notes

• All destructive actions require confirmation.
• Containers and replicas impact performance and reliability.
• Sharing and delegation are limited to organizational users and groups.
• Use truncate before reloading fresh datasets.
• Deleted indexes cannot be restored.

With Data Source Management, users maintain full lifecycle and access control over their organization’s indexes in the Bizdata Datalake.