tencent cloud

masukan

Raw Log Shipping

Terakhir diperbarui:2024-01-20 17:44:35

    Overview

    This document describes how to ship the collected raw log to COS. Currently, only logs collected in a single line, in multiple lines, or with certain separators are supported in this scenario.

    Notes

    You can configure CSV shipping to COS to ship certain raw logs. The following table describes the options of CSV shipping to COS but doesn't apply to JSON or Parquet shipping.
    Log Collection Format
    Support for Raw Log Shipping
    User Raw Log
    CLS Storage Format
    Format for Shipping to COS
    Full text in a single line
    Yes. For more information, see Logs collected in a single line.
    yourlog
    __CONTENT__:yourlog
    yourlog
    Full text in multiple lines
    Yes. For more information, see Logs collected in multiple lines.
    yourlog
    __CONTENT__:yourlog
    yourlog
    Separator (CSV) format
    It depends. For more information, see CSV format. Only space, tab, comma, semicolon, and vertical bar are supported as raw log separators.
    V1 separator V2 separator V3 separator...Vn
    For example, V1,V2,V3...Vn
    K1:V1 K2:V2 K3:V3...Kn:Vn
    V1 separator V2 separator V3 separator...Vn
    For example, V1,V2,V3...Vn
    JSON format
    No
    K1:V1 K2:V2 K3:V3...Kn:Vn
    K1:V1 K2:V2 K3:V3...Kn:Vn
    V1,V2,V3...Vn differs from the original JSON.
    Full regex
    No
    V11V22V33...Vnn
    K1:V1 K2:V2 K3:V3...Kn:Vn
    V1,V2,V3...Vn differs from the raw log.

    Directions

    Log collected in a single line or multiple lines

    For logs collected in a single line or multiple lines, you can configure parameters based on CSV shipping to implement raw log shipping.
    1. Complete the first step of Basic Configuration as instructed in CSV Shipping.
    2. Set Shipping Format to CSV, retain the __CONTENT__ field, delete other fields, set Separator to Space, set Escape Character to Space, set Invalid Field Filling to None, and disable Key in First Line. The parameters are described as follows:
    Configuration Item
    Description
    Remarks
    Key
    __CONTENT__
    For full text in a single line or multiple lines, __CONTENT__ is used as the default key, and the raw log is used as the value. When the raw log is shipped, only the __CONTENT__ field is retained.
    Separator
    Space
    Set **Separator** to **Space** for the full text in a single line or multiple lines.
    Escape Character
    None
    To prevent the raw log from being modified due to escape characters, set Escape Character to None.
    Invalid Field Filling
    None
    Set Invalid Field Filling to None.
    Key in First Line
    Disabled
    You don't need to add a description of the field name in the first line of the CSV file for raw log shipping.
    3. Click OK to enable shipping.

    Logs collected in CSV format

    Note:
    CSV shipping supports limited types of separators, including space, tab, comma, semicolon, and vertical bar. Therefore, you can ship log data in the raw format only when the separator in the raw log content is supported by CSV shipping.
    1. Complete the first step of Basic Configuration as instructed in CSV Shipping.
    2. Set Shipping Format to CSV. During field configuration, you need to delete CLS metadata fields.
    The parameters are described as follows:
    Configuration Item
    Value
    Description
    Key
    Keys
    Only the user field is retained.
    Separator
    A value selected from the drop-down list
    Select the separator of the raw log content. If separators are different, raw log shipping is not supported. Currently, only space, tab, comma, semicolon, and vertical bar are supported.
    Escape Character
    None
    To prevent the raw log from being modified due to escape characters, set Escape Character to None.
    Invalid Field Filling
    None
    Set Invalid Field Filling to None.
    Key in First Line
    Disabled
    You don't need to add a description of the field name in the first line of the CSV file for raw log shipping.
    3. Click OK to enable shipping.
    Hubungi Kami

    Hubungi tim penjualan atau penasihat bisnis kami untuk membantu bisnis Anda.

    Dukungan Teknis

    Buka tiket jika Anda mencari bantuan lebih lanjut. Tiket kami tersedia 7x24.

    Dukungan Telepon 7x24