Syncing Upload/Download/Copy

Syncing Upload/Download/Copy - sync

Last updated: 2025-11-19 15:52:40

Syncing Upload/Download/Copy - sync

Last updated: 2025-11-19 15:52:40

The sync command is used to sync object upload, download, and copy. The difference between sync and cp is that sync first compares the CRC64 value of an object with the same name that already exists, and if the value is the same, the object will not be transferred.
Note: 
If you need to use the upload file command, when you perform authorization policy, set action to cos:HeadBucket,cos:GetBucket,cos:HeadObject,cos:InitiateMultipartUpload,cos:UploadPart,cos:CompleteMultipartUpload,cos:ListMultipartUploads,cos:ListParts.
If you need to use the download file command, when you perform authorization policy, set action to cos:HeadBucket,cos:GetBucket,cos:HeadObject,cos:GetObject.
 If you need to use the copy file command, when you perform authorization policy, set the target object action to cos:GetBucket,cos:HeadObject,cos:InitiateMultipartUpload,cos:PutObject,cos:CompleteMultipartUpload. Set the source object action to cos:HeadBucket,cos:GetBucket,cos:HeadObject,cos:GetObject.
For more authorizations, please refer to CAM-Enabled API.
Command Syntax
./coscli sync <source_path> <destination_path> [flag]
The sync command includes the following parameters:
Parameter Format
Description
Sample
source_path
Source file path, which can be a local path or a COS file path. The COS path is accessible by using the bucket alias or bucket name configured in the configuration file as detailed in Download and Installation Configuration. If you use the bucket name for access, you also need to include the endpoint flag.
Local path: ~/example.txt
COS file path specified with the bucket alias: cos://bucketalias/example.txt
COS file path specified with the bucket name: cos://examplebucket-1250000000/example.txt
destination_path
Destination file path, which can be a local path or a COS file path. The COS path is accessible by using the bucket alias or bucket name configured in the configuration file as detailed in Download and Installation Configuration. If you use the bucket name for access, you also need to include the endpoint flag.
Local path: ~/example.txt 
COS file path specified with the bucket alias: cos://bucketalias/example.txt
COS file path specified with the bucket name: cos://examplebucket-1250000000/example.txt
sync includes the following optional flags:
Flag Abbreviation
Flag Name
Description
None
--include
Includes specific objects (Versions prior to v1.0.4 only filter the local file name during upload, while versions v1.0.4 and later will filter the full path.)
For example: upload all files under ./test to COS. The ./test folder contains the aaa folder, which includes the 1.txt file.
Versions prior to 1.0.4 will match aaa/1.txt
1.0.4 and later versions will match ./test/aaa/1.txt
None
--exclude
Excludes specific objects. (Versions prior to v1.0.4 only filter the local file name during upload, while versions v1.0.4 and later will filter the full path.)
For example: upload all files under ./test to COS. The ./test folder contains the aaa folder, which includes the 1.txt file.
Versions prior to 1.0.4 will match aaa/1.txt
1.0.4 and later versions will match ./test/aaa/1.txt
-r
--recursive
Specifies whether to traverse all objects in the directory recursively.
None
--storage-class
Specifies the storage class of the uploaded file. Default value: STANDARD. For more information, see Storage Class Overview.
None
--part-size
The part size of the file (default 32 MB, supports up to 5 GB). If you need to adaptively adjust the part size based on file size, set it to 0.
None
--thread-num
Number of concurrent threads. Default value: 5
None
--rate-limiting
Speed limit for a single URL. Value range: 0.1–100 MB/s
None
--snapshot-path
Specifies the directory where the snapshot information is stored when the uploaded or downloaded file is saved. During the next file upload or download, COSCLI will read the snapshot information in the specified directory for incremental upload or download. This option is used to speed up directory file sync.
None
--meta
Metadata of the uploaded file, including certain HTTP standard attributes (HTTP Header) and custom metadata prefixed with x-cos-meta- (User Meta). The file metadata is in the format of header:value#header:value, such as Expires:2022-10-12T00:00:00.000Z#Cache-Control:no-cache#Content-Encoding:gzip#x-cos-meta-x:x.
None
--routines
Specifies the number of files for concurrent upload or download threads between files. The default value is 3.
None
--fail-output
This option determines whether to enable the error output of files when upload or download fails (The default value is true, enabled). If enabled, failed file transfers will be recorded in the specified directory (If no directory is specified, the default directory ./coscli_output is used). If disabled, only the number of failed files will be output to the console.
None
--fail-output-path
This option is used to specify the error output folder for recording failed file uploads or downloads. By providing a custom folder path, you can control the location and name of the error output folder. If this option is not set, the default error log folder ./coscli_output is used.
None
--retry-num
Number of frequency limit retry times (The default value is 0, indicating no retry will be performed). It can be specified as 1-10 times. When multiple machines execute download operations on the same COS directory simultaneously, you can specify this parameter to retry to avoid frequency limit errors.
None
--err-retry-num
Number of error retry times (The default value is 5). Set it as 1-10 times, or set it to 0, indicating no retry will be performed.
None
--err-retry-interval
Retry interval (only available when --err-retry-num is set as 1-10). Set the retry interval as 1-10 seconds. If it is not set or is set to 0, each retry interval will be a random value between 1-10 seconds.
None
--only-current-dir
Whether to upload only files in the current directory, ignoring subdirectories and their contents (The default value is false, indicating they will not be ignored).
None
--disable-all-symlink
Whether to ignore the subfiles and subdirectories of all the soft links during upload (The default value is true, indicating they will not be ignored). Currently only supported on Linux and MacOS systems
None
--enable-symlink-dir
Whether to upload the subdirectories of the soft links (The default value is false, indicating they will not be uploaded). Currently only supported on Linux and MacOS systems
None
--disable-crc64
Whether to disable CRC64 data validation (The default value is false, indicating validation is enabled).
None
--delete
Deletes any other files in the specified target path, retaining only the files synchronized this time (The default value is false, indicating the files will not be deleted). It is recommended to enable version control before using the --delete option to prevent accidental data deletion.
None
--backup-dir
Synchronizes the backup of deleted files, which preserves files that are deleted on the target end but do not exist on the source end (effective only during the download, and must be specified when --delete=true). For upload and bucket copies, use version control to restore accidentally deleted data.
None
--force
Forced operation, without confirmation prompt (The default value is false).
None
--disable-checksum
The default is true, verifying only the fragment crc64. When set to false, it verifies the entire file crc64. (coscli V1.0.6 and prior versions default to false)
None
--process-log
Whether process log is enabled, the default is true, enabled
None
--process-log-path
This option is used to specify a dedicated output folder to store process logs. The logs will record information related to file upload or download, including error logs, normal execution logs, retry details. By providing a custom folder path, you can control the location and name of the log output folder. If not set, the default log folder (coscli_output) will be used.
None
--update
Transmit only files with a newer last modified tim
None
--ignore-existing
Transmit only non-existent fil
None
--skip-dir
Defaults to false. When set to true, it skips folders during transmission.
None
--acl
Set the file's ACL, such as private, public-read.
None
--grant-read
Grant the authorized entity permission to read the object. The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None
--grant-read-acp
Grant the authorized entity permission to read the object's access control list (ACL). The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None
--grant-write-acp
Grant the authorized entity permission to write to the object's access control list (ACL). The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None
--grant-full-control
Grant the authorized entity all privileges on the operation object. The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None
--tags
The object tag collection supports up to 10 tags (for example, --tags="Key1=Value1&Key2=Value2")
None
--forbid-overwrite
For buckets with Versioning Not Enabled, when uploading files, specify whether to forbid overwriting objects with the same name.
When set to false, default overwrite for objects with the same name.
When set to true, it means overwriting an Object with the same name is forbidden.
When the bucket is in enabled or paused versioning status, the x-cos-forbid-overwrite header setting is invalid, allowing overwriting objects with the same name.
None
--encryption-type
Server-side encryption method (SSE-COS/SSE-C).
None
--server-side-encryption
Server-side encryption algorithm, supports AES256, cos/kms.
This field is required when using SSE-COS or SSE-KMS.
None
--sse-customer-algo
Server-side encryption algorithm, supports AES256.
This field is required when using SSE-C.
None
--sse-customer-key
Base64-encoded server-side encryption key.
For example, MDEyMzQ1Njc4OUFCQ0RFRjAxMjM0NTY3ODlBQkNERUY=.
This field is required when using SSE-C.
None
--sse-customer-key-md5
Base64-encoded MD5 hash value of the server-side encryption key.
For example, U5L61r7jcwdNvT7frmUG8g==.
This field is required when using SSE-C.
None
--check-point
Whether to enable checkpoint restart, default is true
Note:
sync automatically uses concurrent upload/download for large objects.
If an object is larger than --part-size, COSCLI will split the object into multiple parts according to --part-size and use --thread-num threads to concurrently upload/download the object.
Each thread maintains a URL. For each URL, you can use the --rate-limiting parameter to limit the speed of a single URL. When concurrent upload/download is enabled, the total rate is --thread-num * --rate-limiting.
If an object is uploaded/downloaded in parts, checkpoint restart will be enabled by default.
--include and --exclude support standard regular expression syntax, so you can use them to filter out objects that meet specific criteria.
When using zsh, you may need to add double quotes at both ends of the pattern string.
Do not set snapshot-path to the directory to be migrated or its subdirectories. 
    ./coscli sync ~/test/ cos://bucket1/example/ -r --include ".*\\.txt$" --snapshot-path=/path/snapshot-path  --meta=x-cos-meta-a:a#ContentType:text#Expires:2022-10-12T00:00:00.000Z
When you use the sync command, in addition to PUT request fees, there are two other scenarios where you will have a HEAD request for cloud files, which will incur additional fees:
If the parameter to specify the snapshot directory (`--snapshot-path`) is not added, HEAD request fees will be incurred.
If the parameter to specify the snapshot directory (`--snapshot-path`) is added and it is the first time the snapshot directory is generated, HEAD request fees will be incurred. If it is not the first time the snapshot directory is generated, no additional request fees will be incurred.
For other common options of this command (such as switching bucket and user account), see Common Options.
﻿
Sample
Syncing object upload
./coscli sync ~/example.txt cos://bucket1/example.txt
Syncing object download
./coscli sync cos://bucket1/example.txt ~/example.txt
Syncing intra-bucket replication
./coscli sync cos://bucket1/example.txt cos://bucket1/example_copy.txt
Syncing cross-bucket replication
./coscli sync cos://bucket1/example.txt cos://bucket2/example_copy.txt
﻿

Was this page helpful?

You can also Contact Sales or Submit a Ticket for help.

Yes

Feedback

Parameter Format	Description	Sample
source_path	Source file path, which can be a local path or a COS file path. The COS path is accessible by using the bucket alias or bucket name configured in the configuration file as detailed in Download and Installation Configuration. If you use the bucket name for access, you also need to include the `endpoint` flag.	Local path: ~/example.txt COS file path specified with the bucket alias: cos://bucketalias/example.txt COS file path specified with the bucket name: cos://examplebucket-1250000000/example.txt
destination_path	Destination file path, which can be a local path or a COS file path. The COS path is accessible by using the bucket alias or bucket name configured in the configuration file as detailed in Download and Installation Configuration. If you use the bucket name for access, you also need to include the `endpoint` flag.	Local path: ~/example.txt COS file path specified with the bucket alias: cos://bucketalias/example.txt COS file path specified with the bucket name: cos://examplebucket-1250000000/example.txt

Flag Abbreviation	Flag Name	Description
None	--include	Includes specific objects (Versions prior to v1.0.4 only filter the local file name during upload, while versions v1.0.4 and later will filter the full path.) For example: upload all files under ./test to COS. The ./test folder contains the aaa folder, which includes the 1.txt file. Versions prior to 1.0.4 will match aaa/1.txt 1.0.4 and later versions will match ./test/aaa/1.txt
None	--exclude	Excludes specific objects. (Versions prior to v1.0.4 only filter the local file name during upload, while versions v1.0.4 and later will filter the full path.) For example: upload all files under ./test to COS. The ./test folder contains the aaa folder, which includes the 1.txt file. Versions prior to 1.0.4 will match aaa/1.txt 1.0.4 and later versions will match ./test/aaa/1.txt
-r	--recursive	Specifies whether to traverse all objects in the directory recursively.
None	--storage-class	Specifies the storage class of the uploaded file. Default value: `STANDARD`. For more information, see Storage Class Overview.
None	--part-size	The part size of the file (default 32 MB, supports up to 5 GB). If you need to adaptively adjust the part size based on file size, set it to 0.
None	--thread-num	Number of concurrent threads. Default value: 5
None	--rate-limiting	Speed limit for a single URL. Value range: 0.1–100 MB/s
None	--snapshot-path	Specifies the directory where the snapshot information is stored when the uploaded or downloaded file is saved. During the next file upload or download, COSCLI will read the snapshot information in the specified directory for incremental upload or download. This option is used to speed up directory file sync.
None	--meta	Metadata of the uploaded file, including certain HTTP standard attributes (HTTP Header) and custom metadata prefixed with `x-cos-meta-` (User Meta). The file metadata is in the format of `header:value#header:value`, such as `Expires:2022-10-12T00:00:00.000Z#Cache-Control:no-cache#Content-Encoding:gzip#x-cos-meta-x:x`.
None	--routines	Specifies the number of files for concurrent upload or download threads between files. The default value is `3`.
None	--fail-output	This option determines whether to enable the error output of files when upload or download fails (The default value is `true`, enabled). If enabled, failed file transfers will be recorded in the specified directory (If no directory is specified, the default directory `./coscli_output` is used). If disabled, only the number of failed files will be output to the console.
None	--fail-output-path	This option is used to specify the error output folder for recording failed file uploads or downloads. By providing a custom folder path, you can control the location and name of the error output folder. If this option is not set, the default error log folder `./coscli_output` is used.
None	--retry-num	Number of frequency limit retry times (The default value is `0`, indicating no retry will be performed). It can be specified as `1-10` times. When multiple machines execute download operations on the same COS directory simultaneously, you can specify this parameter to retry to avoid frequency limit errors.
None	--err-retry-num	Number of error retry times (The default value is `5`). Set it as `1-10` times, or set it to `0`, indicating no retry will be performed.
None	--err-retry-interval	Retry interval (only available when `--err-retry-num` is set as `1-10`). Set the retry interval as `1-10` seconds. If it is not set or is set to `0`, each retry interval will be a random value between `1-10` seconds.
None	--only-current-dir	Whether to upload only files in the current directory, ignoring subdirectories and their contents (The default value is `false`, indicating they will not be ignored).
None	--disable-all-symlink	Whether to ignore the subfiles and subdirectories of all the soft links during upload (The default value is `true`, indicating they will not be ignored). Currently only supported on Linux and MacOS systems
None	--enable-symlink-dir	Whether to upload the subdirectories of the soft links (The default value is `false`, indicating they will not be uploaded). Currently only supported on Linux and MacOS systems
None	--disable-crc64	Whether to disable CRC64 data validation (The default value is `false`, indicating validation is enabled).
None	--delete	Deletes any other files in the specified target path, retaining only the files synchronized this time (The default value is `false`, indicating the files will not be deleted). It is recommended to enable version control before using the --delete option to prevent accidental data deletion.
None	--backup-dir	Synchronizes the backup of deleted files, which preserves files that are deleted on the target end but do not exist on the source end (effective only during the download, and must be specified when `--delete=true`). For upload and bucket copies, use version control to restore accidentally deleted data.
None	--force	Forced operation, without confirmation prompt (The default value is `false`).
None	--disable-checksum	The default is true, verifying only the fragment crc64. When set to false, it verifies the entire file crc64. (coscli V1.0.6 and prior versions default to false)
None	--process-log	Whether process log is enabled, the default is true, enabled
None	--process-log-path	This option is used to specify a dedicated output folder to store process logs. The logs will record information related to file upload or download, including error logs, normal execution logs, retry details. By providing a custom folder path, you can control the location and name of the log output folder. If not set, the default log folder (coscli_output) will be used.
None	--update	Transmit only files with a newer last modified tim
None	--ignore-existing	Transmit only non-existent fil
None	--skip-dir	Defaults to false. When set to true, it skips folders during transmission.
None	--acl	Set the file's ACL, such as private, public-read.
None	--grant-read	Grant the authorized entity permission to read the object. The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None	--grant-read-acp	Grant the authorized entity permission to read the object's access control list (ACL). The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None	--grant-write-acp	Grant the authorized entity permission to write to the object's access control list (ACL). The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None	--grant-full-control	Grant the authorized entity all privileges on the operation object. The format is id="[OwnerUin]", for example id="100000000001". Use comma (half-width) to separate multiple authorized entities, such as id="100000000001",id="100000000002".
None	--tags	The object tag collection supports up to 10 tags (for example, --tags="Key1=Value1&Key2=Value2")
None	--forbid-overwrite	For buckets with Versioning Not Enabled, when uploading files, specify whether to forbid overwriting objects with the same name. When set to false, default overwrite for objects with the same name. When set to true, it means overwriting an Object with the same name is forbidden. When the bucket is in enabled or paused versioning status, the x-cos-forbid-overwrite header setting is invalid, allowing overwriting objects with the same name.
None	--encryption-type	Server-side encryption method (SSE-COS/SSE-C).
None	--server-side-encryption	Server-side encryption algorithm, supports AES256, cos/kms. This field is required when using SSE-COS or SSE-KMS.
None	--sse-customer-algo	Server-side encryption algorithm, supports AES256. This field is required when using SSE-C.
None	--sse-customer-key	Base64-encoded server-side encryption key. For example, MDEyMzQ1Njc4OUFCQ0RFRjAxMjM0NTY3ODlBQkNERUY=. This field is required when using SSE-C.
None	--sse-customer-key-md5	Base64-encoded MD5 hash value of the server-side encryption key. For example, U5L61r7jcwdNvT7frmUG8g==. This field is required when using SSE-C.
None	--check-point	Whether to enable checkpoint restart, default is true

tencent cloud

Command Syntax

Sample

Syncing object upload

Syncing object download

Syncing intra-bucket replication

Syncing cross-bucket replication