tencent cloud

masukan

LogListener Service Logs

Terakhir diperbarui:2024-01-20 17:14:28

    Overview

    LogListener service logs are used to record the operation, collection, and monitoring activities of LogListener, and you can configure visualized graphs to display such log data.

    Default configuration

    Default Configuration Item
    Description
    Log Topic
    When you enable LogListener service logs, the logset cls_service_logging will be created for you automatically, and all log data generated by associated machine groups will be categorized and stored in corresponding log topics. The following three log topics are created by default:
    loglistener_status: the heartbeat status logs for the corresponding LogListener.
    loglistener_alarm: logs corresponding to LogListener's monitoring by collection metric/error type .
    loglistener_business: logs corresponding to LogListener's collection operations, with each log corresponds to one request.
    Region
    After the LogListener log services are enabled, logsets and log topics will be created under the machine groups in the same region of LogListener.
    Log Storage Duration
    The default storage duration is 7 days, and the value cannot be modified.
    Index
    Full-text index and key-value index are enabled for all collected log data by default. You can modify the index configuration. For details, please see Configuring Index.
    Dashboard
    Dashboard service_log_dashboard will be created in the same region of LogListener by default.
    Note:
    LogListener service logs are only about collection and monitoring activities of LogListener, and do not support writing into other kinds of data.
    Log data generated by this feature does not incur costs.
    cls_service_logging is a unified logset for LogListener service logs.

    Operations

    Viewing LogListener status When the LogListener service logs are enabled, you can view LogListener running status and collection statistics. You can view the number of active LogListeners, LogListener status distribution and other statistical metrics on the service_log_dashboard dashboard.
    Log collection monitoring configuration You can configure the collection and monitoring service logs by metric/error type. For example:
    MEM, CPU, collection speed, collection latency, or other metrics.
    The number of parsing errors of LogListener.
    File-level monitoring With the LogListener service logs are enabled, you can view the monitoring logs of files and directories. For example:
    All collection statistics files of one IP
    The amount of logs collected under a certain path on a certain IP, such as app1 application logs located in /var/log/app1/. You can get the statistics of logs collected under this path.
    The collection statistics of a topic.

    Prerequisites

    Only LogListener v2.5.4 and above support collection and monitoring service logs by machine/machine group. You’re advised to upgrade LogListener to the latest version.

    Directions

    Enabling the service logs

    1. Log in to the CLS console.
    2. In the left sidebar, click Machine Group to go to the machine group management page.
    3. On the machine group list page, select the target machine group and click
    
    to enable the LogListener service logs.

    Disabling LogListener service logs

    1. Log in to the CLS console.
    2. In the left sidebar, click Machine Group to view the machine group list.
    3. Select the target machine group, and click
    
    to disable the LogListener service logs.
    Note:
    After this feature is disabled, the log data saved in the logset cls_service_logging will not be deleted automatically. You can manually delete the logset where the service logs are saved.

    Dashboard of Service Logs

    When the LogListener service logs are enabled, CLS will create a dashboard service_log_dashboard by the type of recorded logs to display LogListener’s collection and monitoring statistics.

    Collection statistics dashboard

    You can go to the Dashboard page of the CLS console, click the ID of the target dashboard to view LogListener collection statistics, including its status, parsing failure rate, sending success rate, and other metrics.

    Log Types

    LogListener status logs

    The parameters of the log topic loglistener_status are detailed as follows:
    Parameter
    Description
    InstanceId
    LogListener unique identifier
    IP
    Machine group IP
    Label
    An array of machine IDs
    Version
    Version number
    MemoryUsed
    Memory utilization of LogListener
    MemMax
    Memory utilization threshold on this machine set by the Agent
    CpuUsage
    LogListener CPU utilization
    Status
    LogListener running status
    TotalSendLogSize
    Size of logs sent
    SendSuccessLogSize
    Size of successfully sent logs
    SendFailureLogSize
    Size of sending-failed logs
    SendTimeoutLogSize
    Size of logs with sending timed out
    TotalParseLogCount
    Total number of logs parsed
    ParseFailureLogCount
    Number of parsing-failed logs
    TotalSendLogCount
    Number of logs sent
    SendSuccessLogCount
    Number of successful sent logs
    SendFailureLogCount
    Number of sending-failed logs
    SendTimeoutLogCount
    Number of logs with sending timed out
    TotalSendReqs
    Total number of requests sent
    SendSuccessReqs
    Number of successful sent requests
    SendFailureReqs
    The number of sending-failed requests
    SendTimeoutReqs
    Number of requests with sending timed out
    TotalFinishRsps
    Total number of RSP files received
    TotalSuccessFromStart
    Total number of successfully sent requests since LogListener was enabled
    AvgReqSize
    Average request packet size
    SendAvgCost
    Average sending time
    AvailConnNum
    Number of available connections
    QueueSize
    The size of queued requests

    LogListener alarm logs

    The parameters of the log topic loglistener_alarm are detailed as follows:
    Monitoring Metric
    Description
    InstanceId
    LogListener unique identifier
    Label
    An array of machine IDs
    IP
    Machine group IP
    Version
    LogListener version
    AlarmType.count
    Statistics of alarm types
    AlarmType.example
    Sample alarm type
    AlarmType:
    alarm type
    type ID
    Description
    UnknownError
    0
    Initializing the alarm type.
    UnknownError
    1
    Failed to parse.
    CredInvalid
    2
    Failed to verify.
    SendFailure
    3
    Failed to send.
    RunException
    4
    Abnormal LogListener running.
    MemLimited
    5
    Reached the memory utilization threshold.
    FileProcException
    6
    Exceptions occurred in file processing.
    FilePosGetError
    7
    Failed to get the file publishing info.
    HostIpException
    8
    Exceptions occurred in the server IP thread.
    StatException
    9
    Failed to get the process info.
    UpdateException
    10
    Exceptions occurred in the CLS modification feature.
    DoSendError
    11
    Failed to confirm sending.
    FileAddError
    12
    Failed to create the file.
    FileMetaError
    13
    Failed to create the metadata file.
    FileOpenError
    14
    Failed to open the file.
    FileOpenError
    15
    Failed to read the file.
    FileStatError
    16
    Failed to get the file status.
    getTimeError
    17
    Failed to get the time from the log content.
    HandleEventError
    18
    Exceptions occurred in processing the file.
    handleFileCreateError
    19
    Exceptions occurred in handleFileCreateEvent().
    LineParseError
    20
    Failed to parse the log directory.
    Lz4CompressError
    21
    Failed to compress.
    readEventException
    22
    Failed to read.
    ReadFileBugOn
    23
    A bug exists.
    ReadFileException
    24
    Exceptions occurred in the read file.
    ReadFileInodeChange
    25
    File node changed.
    ReadFileTruncate
    26
    The read file is truncated.
    WildCardPathException
    27
    Exceptions occurred in addWildcardPathInotify().

    LogListener collection logs

    The parameters of the log topic loglistener_business are detailed as follows:
    Parameter
    Description
    InstanceId
    LogListener unique identifier
    Label
    An array of machine IDs
    IP
    Machine group IP
    Version
    LogListener version
    TopicId
    The target topic of the collected file
    FileName
    File path name
    FileName
    Actual file path
    FileInode
    File node
    FileSize
    File size
    LastReadTime
    The most recent read time of the file
    ParseFailLines
    Number of parsing-failed logs within a time window
    ParseFailSize
    Size of parsing-failed logs within a time window
    ParseSuccessLines
    Number of logs successful parsed within a time window
    ParseSuccessSize
    Size of logs successful parsed within a time window
    ReadOffset
    Offset of file reading in bytes
    TruncateSize
    Size of truncated log files within a time window
    ReadAvgDelay
    Average time delay for reads within a time window
    TimeFormatFailuresLines
    Number of timestamp matching errors within a time window
    SendSuccessSize
    Size of logs successful sent within a time window
    SendSuccessCount
    Number of logs successful sent within a time window
    SendFailureSize
    Size of sending-failed logs within a time window
    SendFailureCount
    Number of sending-failed logs within a time window
    SendTimeoutSize
    Size of logs with sending timed out within a time window
    SendTimeoutCount
    Number of logs with sending timed out within a time window
    DroppedLogSize
    Size of dropped logs within a time window
    DroppedLogCount
    Number of dropped logs in a time window
    ProcessBlock
    Whether the current file has triggered collection blocking in a statistical period (collection blocking will be triggered if the sliding window of a file has not moved for 10 minutes)
    
    Hubungi Kami

    Hubungi tim penjualan atau penasihat bisnis kami untuk membantu bisnis Anda.

    Dukungan Teknis

    Buka tiket jika Anda mencari bantuan lebih lanjut. Tiket kami tersedia 7x24.

    Dukungan Telepon 7x24