Managing GooseFSx Data Pre-Warming
Last updated:2025-07-17 17:16:40
Creating a Data Pre-Warming One-Time Task
Create a data pre-warming one-time task to execute data pre-warming between the GooseFSx directory and the COS storage bucket.
Prerequisites
Operation Steps
1. Log in to the GooseFS Data Accelerator Console and enter the instance list.
2. Select a region and select the target GooseFSx instance. You can quickly find the target instance by AZ, status, type, or perform a keyword fuzzy matching search by name.
3. Click the instance ID or Manage, then select Data Flow > Data Pre-warming.
4. In single task, click Create Single Task. In the pop-up dialog box, configure as follows.
Task Name: Input an easy-to-identify name, modifiable after creation.
GooseFSx directory: Select from directories that have established data flow rules with COS. Go to the data flow rules page to create.
Dataset: Specifies the flow of certain data in a GooseFSx directory. The GooseFSx directory can be specified by directory (a subdirectory or the entire directory) or by manifest (certain files). The data flow maintains a one-to-one correspondence between the data in the GooseFSx directory and the data in the COS storage bucket. The dataset also specifies the corresponding data flow of the COS storage bucket.
If you select directory, enter a directory path that does not start with "/" but ends with "/", or provide no input.
no input: assign all data in the GooseFSx directory; as shown in the figure below, assign all data in the goosefsx-bj-auto-test-1252246555 directory.
bigfile/: assign all data under the bigfile/ subdirectory in the GooseFSx directory.
bigfile/pic/: assign all data under the bigfile/pic/ subdirectory in the GooseFSx directory.
If you select manifest file, enter the COS default access address of the manifest file as the URL. Custom domains are not supported. Example: https://fsx-bj-test-125xxx.cos.ap-beijing.myqcloud.com/list/test_list1.txt.
The manifest file must be stored in a COS bucket within the same region as the GooseFSx instance and set to public read permission.
Each row in the manifest file is a file path under the GooseFSx directory, with different file paths separated by carriage returns.
For example, to transmit three files in this directory: test_dir1/test_file_1.data, test_dir2/test_file_2.data, test_dir3/test_file_3.data, the list will have three lines. The first line is test_dir1/test_file_1.data, the second line is test_dir2/test_file_2.data, and the third line is test_dir3/test_file_3.data.
The manifest file format is a txt file.
Task type:
Metadata preheating: Only preheat metadata from a COS storage bucket to the GooseFSx directory. Suitable for initial read performance insensitive scenarios.
Data pre-warming: Pre-warm data including metadata from a COS storage bucket to the GooseFSx directory. Suitable for high performance access scenarios.
5. Click Confirm, and the backend will automatically execute data preheating.
Note:
If the data flow mode of the GooseFSx directory is Lazy-load (auto mode L), the dataset does not currently support specifying through a manifest, and the task type does not support data pre-warming.
Note: If the data flow mode of the GooseFSx directory is manual mode, the task type does not support metadata pre-warming.
Deleting a Data Pre-Warming One-Time Task
Note:
You can delete completed or pending execution data pre-warming one-time tasks. Note: pay attention to the following points:
Delete completed one-time data pre-warming tasks. You cannot query the task's relevant information.
Delete pending execution one-time data pre-warming tasks, which cancels the task.
Deletion of executing data pre-warming one-time tasks is not supported.
Prerequisites
Operation Steps
1. Log in to the GooseFS Data Accelerator Console and enter the instance list.
2. Select a region and select the target GooseFSx instance. You can quickly find the target instance by AZ, status, type, or perform a keyword fuzzy matching search by name.
3. Click the instance ID or Manage, then select Data Flow > Data Pre-warming.
4. In the one-time task list, select target task, or filter and sort target tasks by condition; or check multiple tasks and delete in batches.
Querying a Data Pre-Warming One-Time Task
Prerequisites
Operation Steps
1. Log in to the GooseFS Data Accelerator Console and enter the instance list.
2. Select a region and select the target GooseFSx instance. You can quickly find the target instance by AZ, status, type, or perform a keyword fuzzy matching search by name.
3. Click the instance ID or Manage, then select Data Flow > Data Pre-warming.
4. In the one-time task list, select target task, or filter and sort target tasks by condition, click Details to view detailed information.
Replicate Data Preheat Single Task
Prerequisites
Operation Steps
1. Log in to the GooseFS Data Accelerator Console and enter the instance list.
2. Select a region and select the target GooseFSx instance. You can quickly find the target instance by AZ, status, type, or perform a keyword fuzzy matching search by name.
3. Click the instance ID or Manage, then select Data Flow > Data Pre-warming.
4. In the one-time task list, select target task, or filter target tasks by condition, then click Copy Task to perform the operation.
5. In the pop-up dialog box, you can modify the copied task.
Creating a Data Pre-Warming Periodic Task
Periodic task, automatically triggered and executed periodically. For example, pre-warm data from COS to GooseFSx every 1-hour interval, suitable for prompt synchronization of COS bucket data changes and other scenarios.
In the periodic task list, you can see created periodic tasks and perform management operations such as adding, deleting, and querying.
The preheating task triggered by a periodic task will appear in the single task list, all named with the periodic task name and distinguished by the task ID.
Prerequisites
Operation Steps
1. Log in to the GooseFS Data Accelerator Console and enter the instance list.
2. Select a region and select the target GooseFSx instance. You can quickly find the target instance by AZ, status, type, or perform a keyword fuzzy matching search by name.
3. Click the instance ID or Manage, then select Data Flow > Data Pre-warming.
4. In Periodic Task page, click Create Periodic Task. In the pop-up dialog box, configure as follows.
Task Name: Input an easy-to-identify name, modifiable once created.
Trigger cycle:
Once per hour: Execute once every 1-hour interval.
Daily: Execute once every other day.
Weekly: Execute once per interval.
Execute once per month at a one-month interval.
GooseFSx directory: Select from the directory that has established data flow rules with COS. Go to the instance details page to create.
Target path: Specify the directory path to be preheated, such as the GooseFSx directory or a certain subdirectory under it.
Task type:
Metadata preheating: Only preheat metadata from a COS storage bucket to the GooseFSx directory. Suitable for initial read performance insensitive scenarios.
Data pre-warming: Pre-warm data including metadata from a COS storage bucket to the GooseFSx directory. Suitable for high performance access scenarios.
5. Click OK just.
Note:
If the data flow mode of the GooseFSx directory is on-demand loading (auto mode L), periodic tasks are not currently supported.
Note: If the data flow mode of the GooseFSx directory is manual mode, the task type does not support metadata pre-warming.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No
Feedback