Managing Data Sources
Last updated:2025-07-14 21:36:22
Overview
Data source management supports registering directories of storage instances (CFS/GooseFSx) as data sources for fast mounting. It also supports permission control for storage directories through Cloud Access Management (CAM) and Tag.
Data Source
A data source refers to "a specific directory of a storage instance". Among them:
Storage instance: refers to the storage instance of Cloud services such as "CFS/GooseFSx" on Tencent Cloud.
Directory: means the specific path for data storage. You can register the root directory or any subdirectory as a data source.
Data Source Management
Provides the ability to create and manage data sources, as follows:
Create data source: supports registering purchased storage instances on Tencent Cloud to the TI-ONE platform and creating the specified directory as a data source.
Fast mounting data source: In task-based modeling/development machine/online service and other features, it supports one-click selection of the specified directory when mounting without the need to repeat input.
Directory Permission Control: Supports the use of Access Management (CAM) and Tags to implement permission control for storage directories. When mounting CFS/GooseFSx in the TI-ONE platform, users can only access data sources with specified tags. ```
Prerequisites
Before using the data source management feature, please go to the CAM console to authorize the following CAM policy to designated users or user groups. The preset policy is as follows:
Preset policy for read-write permissions: QcloudTIONEResouceGroupFullAccessContainMultiservice
Preset policy for read-only permission: QcloudTIONEReadOnlyAccessContainMultiservice
Creating a Data Source
1. Select left sidebar Platform Management > Data Source Management, enter the Data Source Management page, and click Adding a Data Source.
2. In the Adding a Data Source popup, fill in relevant information, then click OK.
Configuration parameters and their descriptions are as follows:
Field Name | Description |
Data Source Name | 1. Required. Length not exceeding 60 characters; supports Chinese, English, digits, underscores (_), and hyphens (-). Must start with a Chinese character, letter, or digit. 2. Duplicate name check: "Data source name" cannot be repeated (ALL storage instances). Meanwhile, "storage location + read and write permissions" as a unique key cannot be duplicated at the same time. |
Storage Type | Required, options include "GooseFSx" and "CFS", with CFS selected by default. |
Region | Display the current region of the TI-ONE console, which cannot be modified. |
storage instance | 1. Required, the option source corresponds to the purchased storage instance information under the corresponding storage type. (Note: Outside the brackets refers to the instance name, inside the brackets refers to the instance ID.) 2. Note: The instance here refers to the purchased storage instance on Tencent Cloud. TI-ONE merely registers and binds it, not creates a separate instance. |
storage directory | Required, supports manual entry of file path. The path must start with a slash "/" and can only contain Chinese and English, numbers, underscore "_", hyphen "-", and dot ".". Cannot contain ".." (Remark: A data source can only be configured with one directory). |
Read/Write Permission | Required. Options include read-only and read-write. It means the read/write permissions when mounting a data source for each module, including read-only mount and read-write mount. |
Tag | 1. Support tagging for data sources. The tag option source is the Tag console. A data source can add multiple tags. 2. After adding tags, you can combine CAM policy control with visibility permissions for data sources, achieving the effect of data isolation. |
Managing Data Sources
Adding a Data Source
The configuration parameters match the newly added data source. It supports adding a data source to a designated storage instance. The type and name/ID of the selected storage instance cannot be modified.
Mounting Control
Enable mount control means users can only access the specified instance and storage path via mounting "data source". When turned off, users can access any directory through the existing "storage instance + source path".
The effective scope of mount control is the instance dimension, and all configured data sources under the instance will be restricted.
Note: This feature retains the existing mounting method while providing some permission management capabilities. It is recommended to enable it when strong control requirements exist.
Tags & Permissions
Permission configuration for data sources is supported based on CAM and tags. By tagging data sources and configuring appropriate CAM policies, data isolation between different Tencent Cloud sub-accounts is achievable, furthermore enabling permission control for storage locations.
Editing
The configuration parameters match the newly added data source. Existing information will be auto-filled during editing. Modification of "name/storage directory/read and write permissions" is supported. The type and name/ID of the storage instance cannot be modified.
Deletion
Delete a storage instance: means unbinding the binding relationship between TI-ONE data source management and the storage instance, not the actual deletion of the storage instance and its data. Once deleted, all configured data sources under the storage instance will be cleared.
Delete a data source: means deleting a data source directory under the storage instance.
Using Data Source
After completing the data source configuration, when mounting the storage path in dataset management, task-based modeling, development machine, online services, model evaluation, the usage method is as follows:
1. When mounting a "CFS/GooseFSx" instance using the existing method in the above module, you need to limit whether users can select the corresponding instance based on whether the "mount control" feature is enabled for the storage instance.
If the instance has restrictions enabled, the instance option will be disabled in the drop-down list and prompt the following information. If restrictions are not enabled, normal selection is supported.
2. When selecting the mount type as "data source," it supports one-click selection of the specified directory without the need to repeat input.
Dropdown options display the data storage type, storage instance name/ID, storage location, and read-write permissions when hovered.
Select any option to display the storage directory and read-write permissions of the selected data below the text box.
3. Likewise, when mounting a data source in business modules such as task-based modeling or development machines, the drop-down list will only display the data source range accessible to the corresponding team.
Others
Since CAM supports complex custom policies, directory hierarchy conflicts may occur in extreme cases, for example:
Extreme scenarios: Parent directory allowed but subdirectory denied—assuming the storage instance's directory hierarchy is "/", "/a", "/b", data source A is configured with storage location "/", data source B with "/a", and both data sources A and B are authorized to Zhang San. Suppose simultaneously outside the platform, a policy is configured using CAM to limit Zhang San from accessing data source C's storage directory "/b".
Platform processing logic: Both parent and subdirectory are allowed—at this point, user Zhang San can mount directory "/b" via "data source" or "storage instance + source path" within the platform and access it normally without triggering the subdirectory deny effect (because Zhang San has access permission to the parent directory "/").
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No
Feedback