Overview
Bucket replication is a configuration for buckets that allows for the automatic and asynchronous replication of incremental objects across different buckets by setting up replication rules. Once bucket replication is enabled, Cloud Object Storage (COS) will accurately replicate the object content (such as object metadata and version ID) from the source bucket to the target bucket, with the replicated object copies having completely consistent attribute information. Additionally, operations on objects in the source bucket, such as adding or deleting objects, will also be replicated to the target bucket.
Note:
To enable cross-bucket replication, make sure that versioning is enabled for both the source and destination buckets.
After cross-bucket replication is enabled, the object copies will be of the same storage class as the source objects, unless you specify a different storage class during replication.
During replication, COS will copy the access control list (ACL) of the source bucket. Currently, the source bucket and destination bucket must be owned by the same account.
Use Cases
Remote disaster recovery: COS offers 12 9’s of durability for object data, but there is still a slight chance of data loss due to force majeure such as wars and natural disasters. To avoid data loss by explicitly having a separate copy in a different bucket, you can use cross-bucket replication that helps remote disaster recovery. In this way, when the IDC for one bucket is damaged due to force majeure, the IDC for the other bucket can still provide data copies for your use.
Compliance: COS ensures data availability by providing multiple copies and erasure codes for data in physical disks by default. However, some industries may have compliance requirements stipulating that you keep copies in another bucket. Cross-bucket replication allows data to be replicated across buckets to meet such requirements.
Minimizing access latency: when you have end users accessing objects from different regions, with cross-bucket replication, you can maintain object copies in the buckets closest to them. This minimizes access latency to deliver a better user experience.
Special technical requirements: if you have compute clusters in two different buckets and the clusters need to process the same set of data, with cross-bucket replication, you can maintain object copies in both buckets.
Data migration and backup: you can copy your business data from one bucket to another as needed for data migration and backup.
Notes
Fee Instructions
When copying an object, read and write requests occur. COS counts the request count and charges request fees. For details, see Request Fees. If you copy low-frequency data, there may be data retrieval fees incurred from accessing low-frequency data. For details, please refer to Data Retrieval Fees. Intra-region replication is free of traffic charges. Cross-region replication incurs cross-region replication traffic cost. COS calculates the size of cross-region replication traffic and bills according to the unit price of the source bucket's region. For details, see Traffic Fees. After copying an object, storage capacity fees occur. COS calculates the object size. Storage capacity fees are charged based on the storage type of the target object and its region. For details, see Storage Usage Fees. Since bucket replication requires users to enable the versioning feature, multiple earlier versions of objects will exist in the bucket, causing considerable storage consumption. If you wish to reduce costs caused by bucket replication and version control, or customize data retention methods, you can combine your industry background with lifecycle management to achieve cost control or define custom retention modes. Replication time
The time required for COS to replicate an object depends on factors such as the object's size, the distance between bucket regions, and the method of object upload. The synchronization time varies based on these factors, ranging from a few minutes to several hours or even days.
Object size: it takes more time to replicate larger objects, for which multipart upload is recommended for faster upload and synchronization.
Distance between the source and destination buckets: A longer distance between the regions of the buckets requires more time to transfer data.
How the objects were uploaded: unlike simple upload where data can only be uploaded or downloaded serially, multipart upload supports concurrent uploads which can speed up upload and cross-bucket replication of large files. For more information, please see Simple Upload and Multipart Upload. Lifecycle
Bucket replication requires enabling the versioning feature. Versioning maintains multiple earlier versions of objects in the bucket, increasing storage consumption. If you wish to reduce costs from bucket replication and versioning, you can use lifecycle for periodic deletion and file settlement. The lifecycle settlement and deletion operations of the source bucket will not be synchronized automatically to the target bucket. If the copied objects in the destination bucket need to follow the same lifecycle rules as the source bucket, create the same rules for the destination bucket.
If lifecycle rules exist for the destination bucket, note that the creation time of the object copies generated by cross-bucket replication is the creation time of the source objects but not the time when the copies are replicated to the destination bucket.
The object creation time. After setting up the bucket rule, the creation time of object copies in the target bucket remains consistent with the source bucket, not the time when they appear in the target bucket. If you set up a lifecycle rule for the target bucket, the lifecycle rule executes based on this creation time.
Versioning
To use cross-bucket replication, you need to enable versioning for both the source and destination buckets. For more information, please see Versioning Overview. Once versioning is enabled, you should note that disabling it will affect cross-bucket replication: If you try to disable versioning for a bucket where cross-bucket replication is enabled, COS will return an error prompting you to delete the cross-bucket replication rule before disabling versioning.
If you try to disable versioning for a destination bucket, COS will prompt you that doing so will affect cross-bucket replication. If you proceed to disable versioning, the cross-bucket replication rule that uses this bucket as the destination bucket will become invalid.