Overview
Data lifecycle management is an advanced data management feature provided by Tencent Cloud File System (CFS) to balance high performance and low cost in large-scale file storage. With this feature, you can set custom data lifecycle management policies, and the file system can automatically move cold data to lower-cost IA storage based on these policies. When the data is accessed, it is automatically restored to file storage, with the entire process being transparent to the business. This lowers storage costs.
Use cases
Support for storage of massive amounts of data
In scenarios involving massive data storage, traditional cloud file storage is often unable to meet users' cost control requirements due to its high unit price. In the past, the common solution was to manually store data on object storage using tools. However, with data lifecycle management, data transitioning can be completed transparently through simple configuration, greatly simplifying the operation and reducing the total cost of data storage.
Hot/Cold data
In scenarios such as autonomous driving, AI training, and offline analysis, the access frequency of data is different. For example, newly written data is often accessed very frequently, while the access frequency of older data gradually decreases over time. The data lifecycle management feature is well-suited for such access patterns. It can significantly reduce the cost of storing cold data, while meeting high-performance read and write requirements.
Strengths
Support for flexible lifecycle policies
Policies based on directories: Different businesses often have different demands for lifecycle policies. Setting policies based on directories can better meet the demands for diverse policies.
Policies based on file size: Large files often take a long time to restore after transitioning. If your business is sensitive to latency in reading large files, you can set lifecycle policies based on file size to meet your requirements for timeliness.
Policies based on access period: You can flexibly adjust the scope of data for transitioning according to the characteristics of your business, reducing the impact of frequent transitioning and restoration on your business and making better use of lifecycle policies.
Transparent to the business side
After a lifecycle policy is configured, the system automatically transitions and restores data, without any change required to the data access mode of the business side.
Reduced costs
The data lifecycle management feature achieves tiered storage of hot/cold data, which can reduce the unit cost by up to 70% in some scenarios.
Feature details
Lifecycle management policies
When creating a lifecycle management policy, you can configure rules to transition files that have not been accessed within 14, 30, 60, or 90 days to IA storage. Lifecycle management will determine whether to transition a file based on its access time (atime).
The following operations will update the access time of a file:
Reading a file
Writing a file
The following operations will not update the access time of a file:
Querying file metadata (such as by performing the ls or state operation)
Renaming a file
Modifying file metadata such as user, group, or mode
Data transitioning/restoration
Data transitioning is a process in which data is moved from a Turbo file system to IA storage. When the trigger conditions of a policy are met, the system automatically adjusts the concurrency based on the current system load, and copies data to IA storage. Then, it releases the data in the Turbo file storage after 1 hour, with metadata information retained.
Data restoration is a process in which data is moved from IA storage to the Turbo file system. When data in IA storage is accessed for the first time, the system restores the data from IA storage to the Turbo file system. This process will take some time, depending on the file size and system load. When accessed later, this data will be obtained from the Turbo file system.