tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Elastic MapReduce
    Documentation
    Download PDF

    Log

    Last updated: 2024-01-10 09:56:47
    Download PDF

      Overview

      The log search feature collects component operation logs. Then, you can search for core service logs of the current cluster and node system logs by keyword to quickly view key service logs without logging in to nodes.

      Directions

      1. Log in to the EMR console and click the ID/Name of the target cluster in the cluster list.
      2. On the cluster details page, choose Cluster Monitoring > Log Search to filter and view log content by current cluster, log file, node IP, and time range. Alternatively, choose Cluster Service > target component block > Role Management, click a Node IP in Role List to enter the page where the node monitoring metrics are displayed, and then click Role Log to enter the Log Search page.
      
      Click a Node IP to enter the Node Status page, or click Log Source to enter the page where the node monitoring metrics are displayed.
      
      Keyword description:
      Supports full-text search based on keywords.
      Supports search by using special symbols, including hyphen (-), period (.), asterisk (*), greater-than sign (>), less-than sign (<), equal sign (=), exclamation mark (!), brackets (() and {}), and forward slash (/).
      Supports phrase-based search. Example: address=/ip:port.
      3. During troubleshooting, you usually need to pay attention to contextual logs of keywords. On the Log Search page, click View Context to enter the Log Context page.
      Note:
      Currently, only logs in the past 15 days can be searched for.
      If log collection has not been enabled on your cluster, you can contact your Tencent Cloud rep for assistance.
      Component
      Role
      Log
      Description
      HDFS
      NameNode
      /data/emr/hdfs/logs/hadoop-hadoop-namenode.log
      NameNode runtime log
      ZKFC
      /data/emr/hdfs/logs/hadoop-hadoop-zkfc.log
      ZKFC runtime log
      DataNode
      /data/emr/hdfs/logs/hadoop-hadoop-datanode.log
      DataNode runtime log
      JournalNode
      /data/emr/hdfs/logs/hadoop-hadoop-journalnode.log
      JournalNode runtime log
      DFSRouter
      /data/emr/hdfs/logs/hadoop-hadoop-dfsrouter.log
      DFRouter runtime log
      YARN
      ResourceManager
      /data/emr/yarn/logs/yarn-hadoop-resourcemanager.log
      ResourceManager runtime log
      NodeManager
      /data/emr/yarn/logs/yarn-hadoop-nodemanager.log
      NodeManager runtime log
      JobHistoryServer
      /data/emr/yarn/logs/mapred-hadoop-historyserver.log
      JobHistoryServer runtime log
      HBase
      HMaster
      /data/emr/hbase/logs/hbase-hadoop-master.log
      HMaster runtime log
      ThriftServer
      /data/emr/hbase/logs/hbase-hadoop-thrift.log
      ThriftServer runtime log
      RegionServer
      /data/emr/hbase/logs/hbase-hadoop-regionserver.log
      RegionServer runtime log
      ClickHouse
      ClickHouse-server
      /data/clickhouse/clickhouse-server/logs/clickhouse-server.log
      ClickHouse-server runtime log
      Druid
      Broker
      /data/emr/druid/var/log/druid/broker.log
      Broker runtime log
      Coordinator
      /data/emr/druid/var/log/druid/coordinator.log
      Coordinator runtime log
      Router
      /data/emr/druid/var/log/druid/router.log
      Router runtime log
      Overload
      /data/emr/druid/var/log/druid/overload.log
      Overload runtime log
      Historical
      /data/emr/druid/var/log/druid/historical.log
      Historical runtime log
      MiddleManager
      /data/emr/druid/var/log/druid/middleManager.log
      MiddleManager runtime log
      Zookeeper
      Zookeeper
      /data/emr/zookeeper/logs/zookeeper-root-server.log
      ZooKeeper runtime log
      Hive
      HiveServer2
      /data/emr/hive/logs/hadoop-hive
      HiveServer2 runtime log
      Kudu
      KuduMaster
      /data/emr/kudu/logs/kudu-master.WARNING
      KuduMaster runtime log
      KuduServer
      /data/emr/kudu/logs/kudu-tserver.WARNING
      KuduServer runtime log
      Alluxio
      AlluxioMaster
      /data/emr/alluxio/logs/master.log
      AlluxioMaster runtime log
      AlluxioWorker
      /data/emr/alluxio/logs/worker.log
      AlluxioWorker runtime log
      Ranger
      EmbeddedServer
      /data/emr/ranger/logs/ranger-admin.log
      EmbeddedServer runtime log
      CosRanger
      CosRangerServer
      /usr/local/service/cosranger/log/info.log
      COSRanger runtime log
      Impala
      Catalogd
      /data/emr/impala/logs/catalogd.INFO
      Cataloged runtime log
      Statestored
      /data/emr/impala/logs/statestored.INFO
      Statestored runtime log
      Impalad
      /data/emr/impala/logs/impalad.INFO
      Impalad runtime log
      Spark
      HistoryServer
      /data/emr/spark/logs/spark-hadoop.log
      HistoryServer runtime log
      Kylin
      Kylin
      /data/emr/kylin/logs/kylin.log
      Kylin runtime log
      Zeppelin
      ZeppelinServer
      /data/emr/zepplin/logs/zeppelin-hadoop.log
      ZeppelinServer runtime log
      Knox
      Gateway
      /data/emr/knox/logs/gateway.log
      Gateway runtime log
      Doris
      BrokerBootstrap
      /data/emr/doris/broker/log/apache_hdfs_broker.log
      BrokerBootstrap runtime log
      PaloFe
      /data/emr/doris/fe/log/fe.log
      PaloFe runtime log
      PaloBe
      /data/emr/doris/be/log/be.INFO
      PaloBe runtime log
      Kafka
      Kafka
      /user/local/service/kafka/logs/server.log
      Kafka runtime log

      Minimum Log Level Supported by Services

      Service
      Default Minimum Log Level
      Impala and Kudu
      INFO
      Other services
      WARN

      Query Rules for the Minimum Log Level

      Minimum Log Level
      Queryable Log Levels
      INFO
      INFO, WARN, ERROR, FATAL
      WARN
      WARN, ERROR, FATAL
      ERROR
      ERROR, FATAL
      FATAL
      FATAL
      
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy