tencent cloud


Shipping Overview

Last updated: 2022-10-14 15:51:35

    Shipping to COS

    CLS can ship data in a log topic to COS to meet the needs in the following scenarios:

    • Logs are shipped to and stored in COS in STANDARD storage class. If you need other storage classes, perform relevant operations in COS. For more information, see Overview.
    • Log data is processed through offline computing or other computing programs. Such data is shipped to COS first and then loaded by Data Lake Compute (data lake) or EMR (big data platform) for further analysis. For more information, see Using Data Lake Compute (Hive) to Analyze CLS Log. We recommend you choose CSV or Parquet as the shipping format.

    Billing Description

    Log shipping generates private network read traffic fees (cross-region shipping is not supported for now), and CLS will charge fees based on the compression format (Snappy/GZIP/lzop). If your raw log is 100 GB and you choose Snappy for compression, around 50 GB will be billable. As the read traffic price is 0.032 USD/GB, the fees will be 50 GB * 0.032 USD/GB = 1.6 USD.

    Feature Limits

    • Historical data cannot be shipped.
    • Cross-region shipping is not supported. The log topic and COS bucket must be in the same region.
    • Cross-account shipping is not supported.

    Shipping Format

    Data Shipping FormatDescriptionRecommended Scenario
    CSV shipping Log data is shipped to COS based on the specified separator, such as space, tab, comma, semicolon, and vertical bar.
    • It can be used for computing in Data Lake Compute.
    • It can be used to ship raw logs (logs collected in a single line, in multiple lines, and with separators).
    JSON shipping Log data is shipped to COS in JSON format. It is a common data format and can be selected as needed.
    Parquet shipping Log data is shipped to COS in Parquet format. Log data needs to be structured data. The data type can be converted (data not collected in a single line or multiple lines). This format is mainly used for Hive batch processing.

    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support