tencent cloud

DokumentasiData Lake Compute

dlcutils Tool Library and Magic Commands Usage Guide

Unduh
Mode fokus
Ukuran font
Terakhir diperbarui: 2026-05-26 11:22:18

Feature overview

dlcutils is a built-in utility library provided by DLC, designed to assist users in performing common tasks such as file operations, parameter management, Notebook invocation, and key acquisition within Notebooks and jobs.

Prerequisites

DLC service has been activated. For details, see Quick Start.
The machine learning resource groups have been created and started.
Engine image version requirement: Need to upgrade to the image version released after March 12, 2026 for the dlcutils tool library to become available.

File System (dlcutils.fs)

Feature Overview

dlcutils.fs module provides file system operation capabilities, supporting common operations such as listing, copying, moving, and deleting files.

API List

Methodology
Parameter
Feature Description
dlcutils.fs.ls(path, recurse)
path: str - directory path
List files and subdirectories in the specified directory.
dlcutils.fs.cp(src, dest)
src: str - source path
dest: str - destination path
recurse: bool - whether to copy recursively, default False
Copy files or entire directories
dlcutils.fs.head(path, maxBytes)
path: str - file path
maxBytes: int - maximum number of bytes to read, default 1024
View the beginning of the file.
dlcutils.fs.mkdir(path)
path: str - directory path
Create a directory, ignoring if it already exists.
dlcutils.fs.mv(src, dest)
src: str - source path
dest: str - destination path
Move or rename files/directories.
dlcutils.fs.rm(path)
path: str - file or directory path
Delete files or directories.
dlcutils.fs.rsync(src, dest)
src: str - source directory
dest: str - destination directory
Synchronize files from the source directory to the destination directory.

interactive widget (dlcutils.widgets)

Feature Overview

The dlcutils.widgets module provides the capability to create interactive widgets within Notebooks, supporting interaction forms such as text input and dropdown selection, enabling users to dynamically pass parameters.

API List

Methodology
Parameter
Feature Description
dlcutils.widgets.text(name, default, label)
name: str - widget name
default: str - default value
label: str - Display Tag, optional
Create a text input widget.
dlcutils.widgets.get(name)
name: str - widget name
Obtain the current value of the widget.
dlcutils.widgets.getAll()
-
Obtain the list of key-value pairs for all widgets.
dlcutils.widgets.remove(name)
name: str - widget name
Delete the specified widget.
dlcutils.widgets.removeAll()
-
Delete all widgets.
dlcutils.widgets.dropdown(name, choices, default)
name: str - widget name
choices: List[str] - list of options
default: str - default value
Create a dropdown.
dlcutils.widgets.combobox(name, choices, default)
name: str - widget name
choices: List[str] - list of options
default: str - default value
Create a dropdown that allows input or selection.
Note:
dropdown and combobox controls are currently only available in WeData interactive Notebook and community edition Notebook, and are temporarily not supported in WeData scheduling scenarios.

Notebook Control (dlcutils.notebook)

Feature Overview

dlcutils.notebook module supports invoking and executing other Notebooks within the current Notebook, allowing parameter passing and return value retrieval. It applies to building modular data processing workflows.

API List

Methodology
Parameter
Feature Description
dlcutils.notebook.run(path, timeout, arguments)
path: str - Notebook file path
timeout: int - timeout period (sec)
arguments: str - passed parameters
Call other Notebooks and execute with parameters
dlcutils.notebook.exit(value)
value: Any - return value
Stop Notebook execution and return the result value.
dlcutils.library.restartPython()

Restart the current Kernel.

Secrets Management (dlcutils.secrets)

Feature Overview

dlcutils.secrets module provides the capability to obtain keys from Tencent Cloud KMS (SSM), helping users securely manage sensitive information such as database passwords and API Keys, avoiding hardcoding in code.

API List

Methodology
Parameter
Feature Description
dlcutils.secrets.get(secretName, secretVersion, region)
secretName: str - secret name (required)
secretVersion: str - secret version (required)
region: str - region, defaults to the engine's region
Obtain the secret value from Secret Scope.

Job parameter passing (dlcutils.jobs)

Feature Overview

The dlcutils.jobs module provides the capability to pass data between different tasks in Spark jobs, which applies to complex workflow scenarios that require sharing intermediate results across tasks.

API List

Methodology
Parameter
Feature Description
dlcutils.jobs.taskValues.set(key, value)
key: str - key name
value: Any - value
Set the output value for the current task.
dlcutils.jobs.taskValues.get(taskKey, key, default, debugValue)
taskKey: str - task key
key: str - key name
default: Any - default value, optional
debugValue: Any - debug value, optional
Obtain the output value of the specified task.

multi-language magic commands

Feature Overview

DLC Notebook supports switching execution languages using magic commands in cells, enabling users to mix languages such as Python, SQL, and Scala within the same Notebook.

Supported magic commands

Command
Feature Description
Supported resource types
%sql / %%sql
Execute SQL queries in the cell.
machine learning resource group (Spark MLlib type)
%scala / %%scala
Execute Scala code in the cell.
machine learning resource group (Spark MLlib type)
%python / %%python
Execute Python code in the cell.
all machine learning resource group types
%restart_python
Restarting the current Kernel.
machine learning resource group (Spark MLlib type)

Bantuan dan Dukungan

Apakah halaman ini membantu?

masukan