You can view job logs on the Logs page in Job Details, including the startup logs and runtime logs.
Viewing Startup Logs
The startup logs refer to logs generated by the control services during the period when a job is submitted to the cluster and starts running after the job is initiated in the console. The startup logs are used to locate issues encountered during the job submission. You can view the startup logs of each job for free. In Job Management, click the job name to view logs, open the Job Details page, and click Logs. Then, select Startup Logs from the drop-down list in the top left corner to view the startup logs.
On the Logs tab, you can switch between different instances using the second drop-down list in the top left corner. Each time a job version is submitted and run, an instance is generated. Stopping or pausing a job terminates its instance, and restarting or resuming the job creates a new instance. Startup logs are retained for 30 days.
Viewing Runtime Logs
The runtime logs refer to logs generated after a job is submitted to the cluster and starts running, and are used to locate issues during job running. Runtime log collection for the job is enabled by default and logs will be collected into the logset and log topic of the cluster where the job is located. Users can also disable job runtime log collection in Job Parameters of Development and Debugging. Using the Cloud Log Service (CLS) may incur fees. For details, see Cloud Log Service (CLS).
When runtime log collection for a job is enabled, you can view the job's runtime logs by switching to Runtime Logs in the first drop-down list in the top left corner of the Logs tab. Job instances can be switched using the second drop-down list, and the concept of an instance is the same as that in the startup logs. Containers can be switched using the third drop-down list, and each container is a JobManager or a TaskManager.
Auto Refreshing and Auto Wrapping
When viewing log information in text format, you can enable or disable the Auto Refresh and Auto Wrap buttons.
When Auto Refresh is enabled, the system will pull the latest log information in real-time and display the pulled latest logs at the bottom.
When Auto Wrap is enabled, excessively long logs will wrap to a new line. If Auto Wrap is disabled, you need to swipe left or right to view the complete log.
Dynamic Adjustment of Log Level
The job log output level can be dynamically adjusted. This adjustment does not affect running jobs and applies only to the current job instance, affecting the log level of subsequent outputs from that instance. The available levels are: DEBUG, INFO, WARN, and ERROR. You can temporarily adjust the log level for online jobs to facilitate troubleshooting. If log level selection is not supported, you can submit a ticket to request an upgrade.
Log Keyword Search
You can enter keywords in the search box to search log content. You can use the syntax supported by CLS to enter keywords. For example, you can connect keywords with "AND" or "OR" to perform conditional searches for "and" or "or". For more detailed syntax and rules, refer to Syntax and Rules.
After entering keywords, click Search. The search results will be displayed in a list. The search scope is the logs within the selected instances and the specified time range. If you search runtime logs, the search results will list all containers containing the keywords on the left side. Each log can be expanded to view its complete content, and the contained keywords will be highlighted.
Click Text in the top right corner to switch back to the text format for viewing full log information.
Log Quick Diagnosis
Quick Diagnosis provides a feature for quickly selecting common keywords to perform searches. Click the orange Quick Diagnosis button to view common keywords and their descriptions in the pop-up dialog box. You can select multiple keyword combinations. After selecting, click OK to trigger a search using the selected keywords.
