Release Notes
Announcements
Security Announcements
Dimension | Feature | Metric | Trigger Policy | Default Status | Severity Level | Whether Disablement Is Supported |
Basic Diagnosis | Data spike | HBase RS request processing latency | Feature analysis | Enabled | Low | Yes |
| | HBase read-write total request volume | | Enabled | | Yes |
| | HBase RS slow operation count_slowAppendCount | | Enabled | | Yes |
| | HBase RS slow operation count_slowDeleteCount | | Enabled | | Yes |
| | HBase RS slow operation count_slowPutCount | | Enabled | | Yes |
| | Percentage of used node memory | | Enabled | | Yes |
| | TCP LISTEN exception_ListenDrops | | Enabled | | Yes |
| | TCP retransmission rate_InErrRate | | Enabled | | Yes |
| | SR EDITLOG write latency | | Enabled | | Yes |
| | SR FE query latency | | Enabled | | Yes |
| Full load | HDFS storage space utilization | threshold=90 | Enabled | | Yes |
| | Percentage of used HiveServer2 heap memory | threshold=90 | Enabled | | Yes |
| | Node storage space utilization | threshold=90 | Enabled | | Yes |
| | TCP socket memory | threshold=3221225472 | Enabled | | Yes |
| | UDP socket memory | threshold=3221225472 | Enabled | | Yes |
| | TCP4 connection status_CloseWait | threshold=50000 | Enabled | | Yes |
| | TCP4 connection status_ESTABLISHED | threshold=50000 | Enabled | | Yes |
| | TCP4 connection status_TimeWait | threshold=50000 | Enabled | | Yes |
| | YARN Node Manager heap memory utilization | threshold=90 | Enabled | | Yes |
| | YARN Resource Manager heap memory utilization | threshold=90 | Enabled | | Yes |
| Mean offset | TCP4 connection count_CLOSE-WAIT | Feature analysis | Enabled | | Yes |
| | TCP4 connection count_ESTABLISHED | | Enabled | | Yes |
| | TCP4 connection count_TIME-WAIT | | Enabled | | Yes |
| | Number of system processes | | Enabled | | Yes |
Dimension | Engine | Insight Item | Severity Level |
Computing Insights | Hive | JOIN data inflation | Low |
| | Empty input | Low |
| | Full table scanning of partitioned tables | Low |
| | Small input files | Medium |
| | Excessive data scanning | High |
| | MapJoin optimization | Low |
| | Excessive metadata scanning | High |
| | Large table scanning | High |
| | Prolonged compilation duration | High |
| | Improper parameter | Medium |
| Spark | BroadcastJoin optimization | Low |
| | Query condition not pushed down | Low |
| | CPU resource wastage | Low |
| | JOIN data inflation | Low |
| | Data skew | High |
| | Empty Task input | High |
| | ExecutorGC | Medium |
| | Full table scanning of partitioned tables | Low |
| | Global sorting | High |
| | Too many small input files | High |
| | Insufficient resources | High |
| | Excessive data scanning | High |
| | Peak memory exceeds limit | Low |
| | Memory resource wastage | Low |
| | Too many small output files | High |
| | Task reading lag | High |
| | Abnormal resource overhead | High |
| | Scheduling delay | Low |
| | ScheduleOverhead | High |
| | Scheduling skew | High |
| | ShuffleFailure | Medium |
| | Slow Tasks | Medium |
| | Too small Task input data | High |
| | Abnormal Stage duration | Medium |
| | StageScheduleDelay | Medium |
| | Multiple Spark apps concurrently insert into the same table | Low |
| | ShuffleServer application writes TopN | Low |
| Trino | Full table scanning of partitioned tables | Low |
| | Excessive data scanning | High |
| StarRocks | JOIN data inflation | Low |
| | Data Skew | High |
| | Full table scanning of partitioned tables | High |
| | Excessive data scanning | High |
Dimension | Type | Monitoring Data | Trigger Policy | Default Status | Whether Configuration Is Supported |
HDFS Storage Insights | Large and small files | Large file storage greater than 3072 MB | Percentage of storage capacity | Greater than 30% | Yes |
| | Small file storage greater than 0 MB but less than 2 MB | Percentage of file count | Greater than 30% | Yes |
| | Empty file storage equal to 0 MB | Percentage of file count | Greater than 15% | Yes |
| | Junk directory last modified 7 days ago | The junk directory matches the regular expression | .*/warehouse/.*/_temporary/.*/task_.*|.*/.hive-staging.*/ | No |
| Cold and hot data | Last access time of hot files | Date | Less than 1 month | Yes |
| | Last access time of warm files | Date | Greater than or equal to 1 month, less than or equal to 1 year | Yes |
| | Last access time of cold files | Storage capacity | An alert is triggered when the percentage of storage capacity exceeds 50%. | Yes |
StarRocks Insight | Data Tables | Bucketing skew | Bucketing stored amount | Bucketing stored amount skew deviation exceeds 1% | Yes |
| | Index Optimization | Non-index queries/Number of queries | Queries with non-index ratio of 1% Query count/day threshold: 2 | Yes |
Dimension | Type | Insight Item | Severity Level | Insight Default Rule (Configurable) | Default Status | Whether Disablement Is Supported |
Resource Insights | Cluster | Cluster resource CPU sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | Cluster resource CPU sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| | Cluster resource memory sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | Cluster resource memory sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| Yarn queue | Yarn queue physical CPU overused | High | CPU over-allocation ratio 10% Cluster overall CPU utilization greater than 90% | Enabled | Yes |
| | Yarn queue virtual CPU wasted | Low | CPU wastage ratio 10% Cluster overall CPU utilization greater than 90% | Disabled | Yes |
| | Yarn queue physical memory overused | High | Memory over-allocation ratio 10% Cluster overall memory utilization greater than 90% | Enabled | Yes |
| | Yarn queue virtual memory wasted | Low | Memory wastage ratio 10% Cluster overall memory utilization greater than 90% | Disabled | Yes |
| | Yarn queue virtual CPU sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | Yarn queue virtual CPU sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| | Yarn queue virtual memory sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | Yarn queue virtual memory sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| StarRocks | StarRocksBe CPU sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | StarRocksBe CPU sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| | StarRocksBe memory sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | StarRocksBe memory sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| | StarRocksFe CPU sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | StarRocksFe CPU sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
| | StarRocksFe memory sustained idle | Low | Maximum idle utilization 10% Duration: 30 minutes | Disabled | Yes |
| | StarRocksFe memory sustained full load | High | Minimum full load utilization 90% Duration: 30 minutes | Enabled | Yes |
フィードバック