Information about visits to the engine
Used by actions: DescribeDataEngine.
| Name | Type | Required | Description |
|---|---|---|---|
| AccessType | String | No | Methods to visit the engine Note: This field may return null, indicating that no valid values can be obtained. |
| AccessConnectionInfos | Array of String | No | URL that visits the engine and part of internal parameters should be taken place according to the actual situation. Note: This field may return null, indicating that no valid values can be obtained. |
Basic metadata object
Used by actions: AlterDMSDatabase, CreateDMSDatabase, DescribeDMSDatabase, DescribeDMSDatabaseList.
| Name | Type | Required | Description |
|---|---|---|---|
| Id | Integer | No | Primary key Note: This field may return null, indicating that no valid values can be obtained. |
| Name | String | No | Name Note: This field may return null, indicating that no valid values can be obtained. |
| Guid | String | No | Object GUID value Note: This field may return null, indicating that no valid values can be obtained. |
| Catalog | String | No | Data directory Note: This field may return null, indicating that no valid values can be obtained. |
| Description | String | No | Description information |
| Owner | String | No | Object owner |
| OwnerAccount | String | No | Object owner account |
| PermValues | Array of KVPair | No | Permission |
| Params | Array of KVPair | No | Additional attributes |
| BizParams | Array of KVPair | No | Additional business attributes |
| DataVersion | Integer | No | Data version |
| CreateTime | Timestamp ISO8601 | No | Creation time |
| ModifiedTime | Timestamp ISO8601 | No | Modification time |
| DatasourceId | Integer | No | Data source primary key |
VPC information of CHDFS products
Used by actions: DescribeOtherCHDFSBindingList.
| Name | Type | Required | Description |
|---|---|---|---|
| VpcId | String | No | vpc id Note: This field may return null, indicating that no valid values can be obtained. |
| VpcName | String | No | VPC name Note: This field may return null, indicating that no valid values can be obtained. |
| VpcCidrBlock | Array of VpcCidrBlock | No | VPC subnet information list Note: This field may return null, indicating that no valid values can be obtained. |
| RuleId | Integer | No | Rule ID Note: This field may return null, indicating that no valid values can be obtained. |
| AccessGroupId | String | No | Permission group ID Note: This field may return null, indicating that no valid values can be obtained. |
Check permission information
Used by actions: CheckGrantedPermission.
| Name | Type | Required | Description |
|---|---|---|---|
| AccessType | String | Yes | Permission type Note: This field may return null, indicating that no valid values can be obtained. |
| ResourceBaseInfo | ResourceBaseInfo | Yes | Resource information Note: This field may return null, indicating that no valid values can be obtained. |
Column information of the data table.
Used by actions: DescribeTaskResult, QueryResult.
| Name | Type | Required | Description |
|---|---|---|---|
| Name | String | Yes | Column name, which is case-insensitive and can contain up to 25 characters. |
| Type | String | Yes | Column type. Valid values: string|tinyint|smallint|int|bigint|boolean|float|double|decimal|timestamp|date|binary|array |
| Comment | String | No | Class comment. Note: This field may return null, indicating that no valid values can be obtained. |
| Precision | Integer | No | Length of the entire numeric value Note: This field may return null, indicating that no valid values can be obtained. |
| Scale | Integer | No | Length of the decimal part Note: This field may return null, indicating that no valid values can be obtained. |
| Nullable | String | No | Whether the column is null. Note: This field may return null, indicating that no valid values can be obtained. |
| Position | Integer | No | Field position Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | String | No | Field creation time Note: This field may return null, indicating that no valid values can be obtained. |
| ModifiedTime | String | No | Field modification time Note: This field may return null, indicating that no valid values can be obtained. |
| IsPartition | Boolean | No | Whether the column is the partition field. Note: This field may return null, indicating that no valid values can be obtained. |
Common task metrics
Used by actions: DescribeSparkAppTasks, DescribeTasks.
| Name | Type | Description |
|---|---|---|
| CreateTaskTime | Float | Note: This field may return null, indicating that no valid values can be obtained. |
| ProcessTime | Float | Note: This field may return null, indicating that no valid values can be obtained. |
| QueueTime | Float | Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutionTime | Float | Note: This field may return null, indicating that no valid values can be obtained. |
| IsResultCacheHit | Boolean | Note: This field may return null, indicating that no valid values can be obtained. |
| MatchedMVBytes | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| MatchedMVs | String | Note: This field may return null, indicating that no valid values can be obtained. |
| AffectedBytes | String | Note: This field may return null, indicating that no valid values can be obtained. |
| AffectedRows | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| ProcessedBytes | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| ProcessedRows | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
COS permissions
Used by actions: DescribeUserRoles.
| Name | Type | Required | Description |
|---|---|---|---|
| CosPath | String | No | Note: This field may return null, indicating that no valid values can be obtained. |
| Permissions | Array of String | No | Note: This field may return null, indicating that no valid values can be obtained. |
Scheduled start and suspension information
Used by actions: CreateDataEngine, DescribeDataEngine, UpdateDataEngine.
| Name | Type | Required | Description |
|---|---|---|---|
| ResumeTime | String | No | Scheduled starting time, such as 8: 00 a.m. on Monday and Wednesday Note: This field may return null, indicating that no valid values can be obtained. |
| SuspendTime | String | No | Scheduled suspension time, such as 8: 00 p.m. on Monday and Wednesday Note: This field may return null, indicating that no valid values can be obtained. |
| SuspendStrategy | Integer | No | The suspension setting. Valid values: 0 (suspension after task end, default) and 1 (force suspension).Note: This field may return null, indicating that no valid values can be obtained. |
Permission to visit DLC data logs
Used by actions: DescribeDLCCatalogAccess.
| Name | Type | Required | Description |
|---|---|---|---|
| VpcId | String | Yes | VPCID Note: This field may return null, indicating that no valid values can be obtained. |
| Product | String | Yes | Product type Note: This field may return null, indicating that no valid values can be obtained. |
| Description | String | No | Description Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | String | No | Creation time Note: This field may return null, indicating that no valid values can be obtained. |
Query table information object
Used by actions: CreateDLCTable, DescribeDLCTable, DescribeDLCTableList.
| Name | Type | Description |
|---|---|---|
| TableBaseInfo | TableBaseInfo | Basic information of the data table Note: This field may return null, indicating that no valid values can be obtained. |
| Columns | Array of TColumn | Data table column information Note: This field may return null, indicating that no valid values can be obtained. |
| Partitions | Array of TPartition | Data table partition information Note: This field may return null, indicating that no valid values can be obtained. |
| Location | String | Data storage path Note: This field may return null, indicating that no valid values can be obtained. |
| Properties | Array of Property | Data table attribute information Note: This field may return null, indicating that no valid values can be obtained. |
| ModifiedTime | String | Data table update information Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | String | Database creation time Note: This field may return null, indicating that no valid values can be obtained. |
| InputFormat | String | Data format Note: This field may return null, indicating that no valid values can be obtained. |
| StorageSize | Integer | Data storage size Note: This field may return null, indicating that no valid values can be obtained. |
| RecordCount | Integer | Number of rows in the data table Note: This field may return null, indicating that no valid values can be obtained. |
| MapMaterializedViewName | String | Materialized view name Note: This field may return null, indicating that no valid values can be obtained. |
| HeatValue | Integer | Visit hot spots Note: This field may return null, indicating that no valid values can be obtained. |
| ExternalDataConfiguration | ExternalDataConfiguration | External table configuration information Note: This field may return null, indicating that no valid values can be obtained. |
Migrating column objects
Used by actions: CreateDLCTable, DescribeDLCTable, DescribeDLCTableList.
| Name | Type | Required | Description |
|---|---|---|---|
| Name | String | No | Name Note: This field may return null, indicating that no valid values can be obtained. |
| Description | String | No | Description Note: This field may return null, indicating that no valid values can be obtained. |
| Type | String | No | Type Note: This field may return null, indicating that no valid values can be obtained. |
| Position | Integer | No | Sort Note: This field may return null, indicating that no valid values can be obtained. |
| Params | Array of KVPair | No | Additional Parameters Note: This field may return null, indicating that no valid values can be obtained. |
| BizParams | Array of KVPair | No | Business parameters Note: This field may return null, indicating that no valid values can be obtained. |
| IsPartition | Boolean | No | Whether partitioned Note: This field may return null, indicating that no valid values can be obtained. |
Column sorting object
Used by actions: CreateDLCTable, DescribeDLCTable, DescribeDLCTableList.
| Name | Type | Required | Description |
|---|---|---|---|
| Col | String | No | Column name Note: This field may return null, indicating that no valid values can be obtained. |
| Order | Integer | No | Sort Note: This field may return null, indicating that no valid values can be obtained. |
Metadata database objects
Used by actions: DescribeDMSDatabaseList.
| Name | Type | Description |
|---|---|---|
| Name | String | Database name Note: This field may return null, indicating that no valid values can be obtained. |
| SchemaName | String | Schema name Note: This field may return null, indicating that no valid values can be obtained. |
| Location | String | Storage address Note: This field may return null, indicating that no valid values can be obtained. |
| Asset | Asset | Basic object information Note: This field may return null, indicating that no valid values can be obtained. |
Partition objects of the migrating metadata
Used by actions: CreateDLCTable, DescribeDLCTable, DescribeDLCTableList.
| Name | Type | Required | Description |
|---|---|---|---|
| DatabaseName | String | No | Database name |
| SchemaName | String | No | Data directory name |
| TableName | String | No | Table name |
| DataVersion | Integer | No | Data version |
| Name | String | No | Partition name |
| Values | Array of String | No | Value List |
| StorageSize | Integer | No | Storage size |
| RecordCount | Integer | No | Number of records |
| CreateTime | Timestamp ISO8601 | No | Creation time |
| ModifiedTime | Timestamp ISO8601 | No | Modification time |
| LastAccessTime | Timestamp ISO8601 | No | Last access time |
| Params | Array of KVPair | No | Additional attribute |
| Sds | DMSSds | No | Storage object |
Description of metadata storage attributes
Used by actions: CreateDLCTable, DescribeDLCTable, DescribeDLCTableList.
| Name | Type | Required | Description |
|---|---|---|---|
| Location | String | No | Storage address Note: This field may return null, indicating that no valid values can be obtained. |
| InputFormat | String | No | Input format Note: This field may return null, indicating that no valid values can be obtained. |
| OutputFormat | String | No | Output format Note: This field may return null, indicating that no valid values can be obtained. |
| NumBuckets | Integer | No | Number of buckets Note: This field may return null, indicating that no valid values can be obtained. |
| Compressed | Boolean | No | Whether it is compressed Note: This field may return null, indicating that no valid values can be obtained. |
| StoredAsSubDirectories | Boolean | No | Whether it has a subdirectory Note: This field may return null, indicating that no valid values can be obtained. |
| SerdeLib | String | No | Serialization lib Note: This field may return null, indicating that no valid values can be obtained. |
| SerdeName | String | No | Serialization name Note: This field may return null, indicating that no valid values can be obtained. |
| BucketCols | Array of String | No | Bucket name Note: This field may return null, indicating that no valid values can be obtained. |
| SerdeParams | Array of KVPair | No | Serialization parameters Note: This field may return null, indicating that no valid values can be obtained. |
| Params | Array of KVPair | No | Additional Parameters Note: This field may return null, indicating that no valid values can be obtained. |
| SortCols | DMSColumnOrder | No | Column sorting (Expired) Note: This field may return null, indicating that no valid values can be obtained. |
| Cols | Array of DMSColumn | No | Column Note: This field may return null, indicating that no valid values can be obtained. |
| SortColumns | Array of DMSColumnOrder | No | Column sorting field Note: This field may return null, indicating that no valid values can be obtained. |
Source database description
Used by actions: CopyDLCTable.
| Name | Type | Required | Description |
|---|---|---|---|
| SourceDatabaseName | String | Yes | Source database name Note: This field may return null, indicating that no valid values can be obtained. |
| SourceTableNameList | Array of String | Yes | Array of table names in the source database Note: This field may return null, indicating that no valid values can be obtained. |
Basic information of DataEngine
Used by actions: DescribeUpdatableDataEngines.
| Name | Type | Description |
|---|---|---|
| DataEngineName | String | DataEngine name |
| State | Integer | Data engine status: -2: deleted; -1: failed; 0: initializing; 1: suspended; 2: running; 3: ready to delete; 4: deleting |
| CreateTime | Integer | Creation time |
| UpdateTime | Integer | Update time |
| Message | String | Returned information Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineId | String | Engine ID |
| DataEngineType | String | Engine types, and the valid values are PrestoSQL, SparkSQL, and SparkBatch. Note: This field may return null, indicating that no valid values can be obtained. |
| AppId | Integer | User ID Note: This field may return null, indicating that no valid values can be obtained. |
| UserUin | String | Account ID Note: This field may return null, indicating that no valid values can be obtained. |
Engine configuration information
Used by actions: DescribeUserDataEngineConfig.
| Name | Type | Description |
|---|---|---|
| DataEngineId | String | Engine ID Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineConfigPairs | Array of DataEngineConfigPair | Collection of user-defined configuration items |
| SessionResourceTemplate | SessionResourceTemplate | Template of the resource parameter configuration of the job cluster |
Engine configurations
Used by actions: CheckDataEngineConfigPairsValidity, CreateDataEngine, DescribeUserDataEngineConfig, UpdateUserDataEngineConfig.
| Name | Type | Required | Description |
|---|---|---|---|
| ConfigItem | String | Yes | Configuration items Note: This field may return null, indicating that no valid values can be obtained. |
| ConfigValue | String | Yes | Configuration values Note: This field may return null, indicating that no valid values can be obtained. |
Information about the major version of the cluster image
Used by actions: DescribeDataEngineImageVersions.
| Name | Type | Description |
|---|---|---|
| ImageVersionId | String | ID of the major version of the image |
| ImageVersion | String | Name of the major version of the image |
| Description | String | Description of the major version of the image Note: This field may return null, indicating that no valid values can be obtained. |
| IsPublic | Integer | Whether it is a public version: 1: public version; 2: private version |
| EngineType | String | Cluster types: SparkSQL, PrestoSQL, and SparkBatch |
| IsSharedEngine | Integer | Version status. 1: initializing; 2: online; 3: offline |
| State | Integer | Version status. 1: initializing; 2: online; 3: offline |
| InsertTime | String | Insert time |
| UpdateTime | String | Update time |
DataEngine details
Used by actions: DescribeDataEngine.
| Name | Type | Required | Description |
|---|---|---|---|
| DataEngineName | String | Yes | DataEngine name |
| EngineType | String | Yes | Engine type: spark/presto |
| ClusterType | String | Yes | Cluster resource type spark_private/presto_private/presto_cu/spark_cu |
| QuotaId | String | No | Quota ID |
| State | Integer | No | Data engine status -2 deleted, -1 failed, 0 initializing, 1 suspended, 2 running, 3 ready to delete, and 4 deleting |
| CreateTime | Integer | No | Creation time |
| UpdateTime | Integer | No | Update time |
| Size | Integer | No | Cluster specifications Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| Mode | Integer | No | Billing mode: 0 shared mode, 1 pay-as-you-go, and 2 monthly subscription |
| MinClusters | Integer | No | Minimum number of clusters Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| MaxClusters | Integer | No | Maximum number of clusters Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| AutoResume | Boolean | No | Whether to recover automatically Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| SpendAfter | Integer | No | Automatic recovery time Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| CidrBlock | String | No | Cluster IP range Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| DefaultDataEngine | Boolean | No | Whether it is the default engine Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| Message | String | No | Returned Message Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| DataEngineId | String | No | Engine ID |
| SubAccountUin | String | No | Operator |
| ExpireTime | String | No | Expiration time |
| IsolatedTime | String | No | Isolation time |
| ReversalTime | String | No | Rectification time |
| UserAlias | String | No | Username Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| TagList | Array of TagInfo | No | Key-value pair collection Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| Permissions | Array of String | No | Permissions owned by the engine Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| AutoSuspend | Boolean | No | Whether to automatically suspend the cluster: False (by default) means that the cluster is not automatically suspended; true means that the cluster is automatically suspended. Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| CrontabResumeSuspend | Integer | No | Scheduling policy of starting and ending the cluster: 0 (by default) indicates turning off the scheduling policy; 1 indicates turning on the scheduling policy. (Note: The scheduling policy of starting and ending the cluster and the automatic suspension policy are mutually exclusive.) Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| CrontabResumeSuspendStrategy | CrontabResumeSuspendStrategy | No | Scheduled start and end policy. Complex type: Include start and end time and the cluster suspension strategy. Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineExecType | String | No | Type of the task executed by the engine. Valid value: SQL/BATCH Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| RenewFlag | Integer | No | Automatic renewal flag: 0 means the initial state and there is no automatic renewal by default. If the user has the privilege to retain services with advanced payment, there will be an automatic renewal. 1 means that there is an automatic renewal. 2 means that there is surely no automatic renewal. Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| AutoSuspendTime | Integer | No | Cluster automatic suspension time Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| NetworkConnectionSet | Array of NetworkConnection | No | Network connection configuration Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| UiURL | String | No | Jump address of ui Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| ResourceType | String | No | Type of the engine resource Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| ImageVersionId | String | No | Cluster image version ID Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| ChildImageVersionId | String | No | Minor cluster image version ID Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| ImageVersionName | String | No | Cluster image version name Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| StartStandbyCluster | Boolean | No | Whether to enable the standby cluster Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| ElasticSwitch | Boolean | No | Whether the elastic scaling of the spark jar monthly subscription cluster is enabled Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| ElasticLimit | Integer | No | Upper limit of the spark jar's monthly subscribed cluster elasticity Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| DefaultHouse | Boolean | No | Whether it is the default engine Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| MaxConcurrency | Integer | No | Maximum concurrency number of a single cluster Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| TolerableQueueTime | Integer | No | Maximum time for the task to queue Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| UserAppId | Integer | No | User AppID Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| UserUin | String | No | User uin Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| SessionResourceTemplate | SessionResourceTemplate | No | SessionResourceTemplate Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| AutoAuthorization | Boolean | No | Automatic authorization switch Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineGeneration | String | No | Engine version, supporting Native/SuperSQL Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineTypeDetail | String | No | Detailed engine type, supporting SparkSQL/SparkBatch/PrestoSQL/Kyuubi Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineNetworkId | String | No | Engine network ID Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineResourceGroupCount | Integer | No | Number of resource groups associated with the standard engine Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineResourceUsedCU | Integer | No | Current usage of the engine (Cu) Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| AccessInfos | Array of AccessInfo | No | Access information of the engine Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| EngineNetworkName | String | No | Name of the network where the engine is located Note: The returned value of this field may be null, indicating that no valid value is obtained. |
Engine specification details
Used by actions: DescribeDataEnginesScaleDetail.
| Name | Type | Description |
|---|---|---|
| DataEngineId | String | Engine ID Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineName | String | Engine name Note: This field may return null, indicating that no valid values can be obtained. |
| ScaleDetail | Array of DataEngineScaleInfoDetail | Engine specification details Note: This field may return null, indicating that no valid values can be obtained. |
Engine specification details
Used by actions: DescribeDataEnginesScaleDetail.
| Name | Type | Description |
|---|---|---|
| StartTime | String | Start time of counting in the format of yyyy-MM-dd HH:mm:ss Note: This field may return null, indicating that no valid values can be obtained. |
| EndTime | String | End time of counting in the format of yyyy-MM-dd HH:mm:ss Note: This field may return null, indicating that no valid values can be obtained. |
| CU | Integer | Current statistical time period and engine specifications Note: This field may return null, indicating that no valid values can be obtained. |
DMask policy information.
Used by actions: DescribeDataMaskStrategies.
| Name | Type | Required | Description |
|---|---|---|---|
| StrategyId | String | No | Policy ID Note: This field may return null, indicating that no valid values can be obtained. |
| UserAppId | String | No | User AppId Note: This field may return null, indicating that no valid values can be obtained. |
| Uin | String | No | User UIN Note: This field may return null, indicating that no valid values can be obtained. |
| SubAccountUin | String | No | Specifies the uin of the user sub-account to operate. Note: This field may return null, indicating that no valid values can be obtained. |
| StrategyName | String | No | Policy name Note: This field may return null, indicating that no valid values can be obtained. |
| StrategyType | String | No | MASK_SHOW_FIRST_4, MASK_SHOW_LAST_4, MASK_HASH, MASK_DATE_SHOW_YEAR, MASK_NULL, MASK_DEFAULT. Note: This field may return null, indicating that no valid values can be obtained. |
| StrategyDesc | String | No | Policy description Note: This field may return null, indicating that no valid values can be obtained. |
| Groups | Array of GroupInfo | No | User group policy list. Note: This field may return null, indicating that no valid values can be obtained. |
| Users | String | No | User sub-account uin list, concatenated by ;. Note: This field may return null, indicating that no valid values can be obtained. |
| State | Integer | No | 1: in effect; 0: deleted. Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | Integer | No | Policy creation time. timestamp in milliseconds. Note: This field may return null, indicating that no valid values can be obtained. |
| UpdateTime | Integer | No | Policy update time. timestamp in milliseconds. Note: This field may return null, indicating that no valid values can be obtained. |
DMask policy information.
Used by actions: CreateDataMaskStrategy, UpdateDataMaskStrategy.
| Name | Type | Required | Description |
|---|---|---|---|
| StrategyName | String | No | Policy name |
| StrategyType | String | No | MASK_SHOW_FIRST_4, MASK_SHOW_LAST_4, MASK_HASH, MASK_DATE_SHOW_YEAR, MASK_NULL, MASK_DEFAULT. |
| StrategyDesc | String | No | Policy description |
| Groups | Array of GroupInfo | No | User group policy list. Note: This field may return null, indicating that no valid values can be obtained. |
| Users | String | No | User sub-account uin list, concatenated by ;. |
| StrategyId | String | No | Policy ID |
Masking policy permission object.
Used by actions: AttachDataMaskPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| PolicyInfo | Policy | No | Masking permission object. Note: This field may return null, indicating that no valid values can be obtained. |
| DataMaskStrategyId | String | No | Masking policy ID. specifies the ID of the data masking policy. Note: This field may return null, indicating that no valid values can be obtained. |
| ColumnType | String | No | Bound field type. Note: This field may return null, indicating that no valid values can be obtained. |
TensorFlow, Pytorch, SK-learn image information list.
Used by actions: DescribeSessionImageVersion.
| Name | Type | Required | Description |
|---|---|---|---|
| SparkImageId | String | No | Specifies the unique id of the Spark mirror. Note: This field may return null, indicating that no valid values can be obtained. |
| SparkImageVersion | String | No | Spark image version name. Note: This field may return null, indicating that no valid values can be obtained. |
| SparkImageType | Integer | No | Minor version image type. 1: TensorFlow, 2: Pytorch, 3: SK-learn. Note: This field may return null, indicating that no valid values can be obtained. |
| SparkImageTag | String | No | Image address. Note: This field may return null, indicating that no valid values can be obtained. |
SQL statement objects
Used by actions: GenerateCreateMangedTableSql.
| Name | Type | Description |
|---|---|---|
| SQL | String | The automatically generated SQL statements. |
External table configuration
Used by actions: CreateDLCTable, DescribeDLCTable, DescribeDLCTableList.
| Name | Type | Required | Description |
|---|---|---|---|
| Sds | DMSSds | No | Metadata storage object Note: This field may return null, indicating that no valid values can be obtained. |
| ViewOriginalText | String | No | View text Note: This field may return null, indicating that no valid values can be obtained. |
| ViewExpandedText | String | No | View text Note: This field may return null, indicating that no valid values can be obtained. |
| Retention | Integer | No | Hive maintenance version Note: This field may return null, indicating that no valid values can be obtained. |
| LifeTime | Integer | No | Lifecycle Note: This field may return null, indicating that no valid values can be obtained. |
| Partitions | Array of DMSPartition | No | Partition Note: This field may return null, indicating that no valid values can be obtained. |
FavorInfo
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| Priority | Integer | No | Prioritized items Note: This field may return null, indicating that no valid values can be obtained. |
| Catalog | String | No | Catalog name Note: This field may return null, indicating that no valid values can be obtained. |
| DataBase | String | No | DataBase name Note: This field may return null, indicating that no valid values can be obtained. |
| Table | String | No | Table name Note: This field may return null, indicating that no valid values can be obtained. |
Query list filter parameter
Used by actions: DescribeDLCCatalogAccess, DescribeDLCTableList, DescribeDataMaskStrategies, DescribeSparkAppJobs, DescribeSparkAppTasks, DescribeTablesName, DescribeTasks, DescribeUserDataEngineConfig, DescribeUserInfo, DescribeUsers, DescribeWorkGroupInfo, DescribeWorkGroups, QueryTaskCostDetail.
| Name | Type | Required | Description |
|---|---|---|---|
| Name | String | Yes | Attribute name. If more than one filter exists, the logical relationship between these filters is OR. |
| Values | Array of String | Yes | Attribute value. If multiple values exist in one filter, the logical relationship between these values is OR. |
DMask user group information.
Used by actions: CreateDataMaskStrategy, DescribeDataMaskStrategies, UpdateDataMaskStrategy.
| Name | Type | Required | Description |
|---|---|---|---|
| WorkGroupId | Integer | No | User group ID. |
| StrategyType | String | No | Policy type. |
Job information
Used by actions: DescribeJobs.
| Name | Type | Required | Description |
|---|---|---|---|
| JobName | String | No | Job name Note: This field may return null, indicating that no valid values can be obtained. |
| StatisticInfo | StatisticInfo | No | Statistics Note: This field may return null, indicating that no valid values can be obtained. |
| JobConfiguration | String | No | Job configuration Note: This field may return null, indicating that no valid values can be obtained. |
| JobStatus | Integer | No | The current status of the job: 0: initializing; 1: running; 2: executed; 3: writing data; 4: queuing; -1: execution failed; -3: manually terminated. The job execution result is returned only when the job is successfully executed. Note: This field may return null, indicating that no valid values can be obtained. |
Configuration format
Used by actions: AlterDMSDatabase, CreateDLCTable, CreateDMSDatabase, CreateSparkSessionBatchSQL, CreateTask, CreateTasks, DescribeDLCTable, DescribeDLCTableList, DescribeDMSDatabase, DescribeSparkSessionBatchSqlLog, DropDMSTable.
| Name | Type | Required | Description |
|---|---|---|---|
| Key | String | Yes | Configured key Note: This field may return null, indicating that no valid values can be obtained. |
| Value | String | Yes | Configured value Note: This field may return null, indicating that no valid values can be obtained. |
Describe basic information about DLC managed storage
Used by actions: DescribeLakeFsInfo.
| Name | Type | Description |
|---|---|---|
| Name | String | Managed storage name |
| Type | String | Managed storage type |
| SpaceUsedSize | Float | Capacity |
| CreateTimeStamp | Integer | Timestamp of the creation time |
| DefaultBucket | Integer | Whether it is the user's default bucket: 0: default bucket, 1: non-default bucket Note: This field may return null, indicating that no valid values can be obtained. |
| ShortName | String | Managed storage short name Note: This field may return null, indicating that no valid values can be obtained. |
| Description | String | Description information of the bucket Note: This field may return null, indicating that no valid values can be obtained. |
| Status | String | The status of the managed bucket. Current values: creating, bind, readOnly, and isolate Note: This field may return null, indicating that no valid values can be obtained. |
Bind fusion bucket information
Used by actions: CreateCHDFSBindingProduct.
| Name | Type | Description |
|---|---|---|
| BucketId | String | Bucket ID Note: This field may return null, indicating that no valid values can be obtained. |
| VpcId | String | vpcId Note: This field may return null, indicating that no valid values can be obtained. |
| VpcCidrBlock | String | Subnet address Note: This field may return null, indicating that no valid values can be obtained. |
| AccessGroupId | String | Permission group ID Note: This field may return null, indicating that no valid values can be obtained. |
| AccessRuleId | Integer | Permission rule ID Note: This field may return null, indicating that no valid values can be obtained. |
Network configuration
Used by actions: DescribeDataEngine.
| Name | Type | Required | Description |
|---|---|---|---|
| Id | Integer | No | Network configuration ID Note: This field may return null, indicating that no valid values can be obtained. |
| AssociateId | String | No | Unique identifier of network configuration Note: This field may return null, indicating that no valid values can be obtained. |
| HouseId | String | No | Computing engine ID Note: This field may return null, indicating that no valid values can be obtained. |
| DatasourceConnectionId | String | No | Data source ID (discarded) Note: This field may return null, indicating that no valid values can be obtained. |
| State | Integer | No | Network configuration status (0 - initializing; 1 - normal) Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | Integer | No | Creation time Note: This field may return null, indicating that no valid values can be obtained. |
| UpdateTime | Integer | No | Modification time Note: This field may return null, indicating that no valid values can be obtained. |
| Appid | Integer | No | Create user Appid. Note: This field may return null, indicating that no valid values can be obtained. |
| HouseName | String | No | Computing engine name Note: This field may return null, indicating that no valid values can be obtained. |
| DatasourceConnectionName | String | No | Network configuration name Note: This field may return null, indicating that no valid values can be obtained. |
| NetworkConnectionType | Integer | No | Network configuration type Note: This field may return null, indicating that no valid values can be obtained. |
| Uin | String | No | Create user UIN. Note: This field may return null, indicating that no valid values can be obtained. |
| SubAccountUin | String | No | Create user SubAccountUin. Note: This field may return null, indicating that no valid values can be obtained. |
| NetworkConnectionDesc | String | No | Network configuration description Note: This field may return null, indicating that no valid values can be obtained. |
| DatasourceConnectionVpcId | String | No | Data source vpcid Note: This field may return null, indicating that no valid values can be obtained. |
| DatasourceConnectionSubnetId | String | No | Data source SubnetId Note: This field may return null, indicating that no valid values can be obtained. |
| DatasourceConnectionCidrBlock | String | No | Data source SubnetId Note: This field may return null, indicating that no valid values can be obtained. |
| DatasourceConnectionSubnetCidrBlock | String | No | Data source SubnetCidrBlock Note: This field may return null, indicating that no valid values can be obtained. |
Information about the users accessing through the third-party
Used by actions: DescribeThirdPartyAccessUser.
| Name | Type | Description |
|---|---|---|
| Id | Integer | ID information Note: This field may return null, indicating that no valid values can be obtained. |
| Uin | String | User primary UIN Note: This field may return null, indicating that no valid values can be obtained. |
| AppId | String | User AppID Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | String | Activation time Note: This field may return null, indicating that no valid values can be obtained. |
Binding Self-purchased CHDFS to DLC
Used by actions: DescribeOtherCHDFSBindingList.
| Name | Type | Description |
|---|---|---|
| ProductName | String | Product name Note: This field may return null, indicating that no valid values can be obtained. |
| SuperUser | Array of String | User name (a discarded field) Note: This field may return null, indicating that no valid values can be obtained. |
| VpcInfo | Array of CHDFSProductVpcInfo | VPC configuration information Note: This field may return null, indicating that no valid values can be obtained. |
| IsBind | Boolean | Whether to bind the bucket (a discarded field) Note: This field may return null, indicating that no valid values can be obtained. |
Permission information
Used by actions: CheckGrantedPermission.
| Name | Type | Required | Description |
|---|---|---|---|
| AccessType | String | No | Permission type Note: This field may return null, indicating that no valid values can be obtained. |
| ResourceBaseInfo | ResourceBaseInfo | No | Resource information Note: This field may return null, indicating that no valid values can be obtained. |
| Allowed | Boolean | No | Authentication result Note: This field may return null, indicating that no valid values can be obtained. |
Permission objects
Used by actions: AttachDataMaskPolicy, AttachUserPolicy, AttachWorkGroupPolicy, CreateUser, CreateWorkGroup, DescribeUserInfo, DescribeUsers, DescribeWorkGroupInfo, DescribeWorkGroups, DetachUserPolicy, DetachWorkGroupPolicy, UpdateRowFilter.
| Name | Type | Required | Description |
|---|---|---|---|
| Database | String | Yes | The database name requiring authorization. use * to represent all databases under the current Catalog. for administrator level authorization type, only * is allowed. for data connection level authorization type, leave it blank. for other types, any specified database is allowed. |
| Catalog | String | Yes | Specifies the name of the data source requiring authorization. at administrator level, only * is supported (representing all resources at this level). for database-level or data source-level authentication, only COSDataCatalog or * is supported. for table-level authentication, user-defined data sources can be filled. defaults to DataLakeCatalog if left blank. note: if authenticating a user-defined data source, the permissions DLC can manage are a subset of the account provided when the user accesses the data source. |
| Table | String | Yes | Specifies the table name requiring authorization. use * to represent all tables in the current Database. for administrator-level authorization type, only * is allowed. for data connection level or Database-level authorization type, leave it blank. for other types, any specific data table can be specified. |
| Operation | String | Yes | The target permissions, which vary by permission level. Admin: ALL (default); data connection: CREATE; database: ALL, CREATE, ALTER, and DROP; table: ALL, SELECT, INSERT, ALTER, DELETE, DROP, and UPDATE. Note: For table permissions, if a data source other than COSDataCatalog is specified, only the SELECT permission can be granted here. |
| PolicyType | String | No | The permission type. Valid values: ADMIN, DATASOURCE, DATABASE, TABLE, VIEW, FUNCTION, COLUMN, and ENGINE. Note: If it is left empty, ADMIN is used. |
| Function | String | No | Name of the function requiring authorization. use * to represent all functions in the current Catalog. for administrator-level authorization type, only * is allowed. for data connection-level authorization type, leave it blank. for other types, any function can be specified. Note: This field may return null, indicating that no valid values can be obtained. |
| View | String | No | Authorization is required for the view. fill in * to represent all views under the current Database. when the authorization type is administrator level, only * is allowed. when the authorization type is data connection level or Database level, only blank is allowed. for other types, any view can be specified. Note: This field may return null, indicating that no valid values can be obtained. |
| Column | String | No | Columns requiring authorization. use * to represent all current columns. when the authorization type is administrator level, only * is allowed. Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngine | String | No | The data engine requiring authorization. use * to represent all current engines. when the authorization type is administrator level, only * is allowed. Note: This field may return null, indicating that no valid values can be obtained. |
| ReAuth | Boolean | No | Whether the grantee is allowed to further grant the permissions. Valid values: false (default) and true (the grantee can grant permissions gained here to other sub-users).Note: This field may return null, indicating that no valid values can be obtained. |
| Source | String | No | The permission source, which is not required when input parameters are passed in. Valid values: USER (from the user) and WORKGROUP (from one or more associated work groups).Note: This field may return null, indicating that no valid values can be obtained. |
| Mode | String | No | The grant mode, which is not required as an input parameter. Valid values: COMMON and SENIOR.Note: This field may return null, indicating that no valid values can be obtained. |
| Operator | String | No | The operator, which is not required as an input parameter. Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | String | No | The permission policy creation time, which is not required as an input parameter. Note: This field may return null, indicating that no valid values can be obtained. |
| SourceId | Integer | No | The ID of the work group, which applies only when the value of the Source field is WORKGROUP.Note: This field may return null, indicating that no valid values can be obtained. |
| SourceName | String | No | The name of the work group, which applies only when the value of the Source field is WORKGROUP.Note: This field may return null, indicating that no valid values can be obtained. |
| Id | Integer | No | The policy ID. Note: This field may return null, indicating that no valid values can be obtained. |
| EngineGeneration | String | No | Specifies the engine type. |
Collection of policies
Used by actions: DescribeUserInfo, DescribeWorkGroupInfo.
| Name | Type | Description |
|---|---|---|
| PolicySet | Array of Policy | Collection of policies Note: This field may return null, indicating that no valid values can be obtained. |
| TotalCount | Integer | Total policies Note: This field may return null, indicating that no valid values can be obtained. |
Presto monitoring metrics
Used by actions: DescribeSparkAppTasks, DescribeTasks.
| Name | Type | Description |
|---|---|---|
| LocalCacheHitRate | Float | Note: This field may return null, indicating that no valid values can be obtained. |
| FragmentCacheHitRate | Float | Note: This field may return null, indicating that no valid values can be obtained. |
Properties of database and table
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql.
| Name | Type | Required | Description |
|---|---|---|---|
| Key | String | Yes | The property key name. |
| Value | String | Yes | The property value. |
Python-spark image information.
Used by actions: DescribeDataEnginePythonSparkImages.
| Name | Type | Description |
|---|---|---|
| SparkImageId | String | Unique ID of the spark image |
| ChildImageVersionId | String | ID of the cluster image of the minor version |
| SparkImageVersion | String | Name of the spark image |
| Description | String | Description of the spark image Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | String | Creation time |
| UpdateTime | String | Update time |
Resource basic information
Used by actions: CheckGrantedPermission.
| Name | Type | Required | Description |
|---|---|---|---|
| Catalog | String | No | Catalog name Note: This field may return null, indicating that no valid values can be obtained. |
| Schema | String | No | Schema name Note: This field may return null, indicating that no valid values can be obtained. |
| Table | String | No | Table name Note: This field may return null, indicating that no valid values can be obtained. |
| View | String | No | View name Note: This field may return null, indicating that no valid values can be obtained. |
| Database | String | No | Database name Note: This field may return null, indicating that no valid values can be obtained. |
| Function | String | No | Function name Note: This field may return null, indicating that no valid values can be obtained. |
ResourceInfo
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| AttributionType | String | No | Affiliated type Note: This field may return null, indicating that no valid values can be obtained. |
| ResourceType | String | No | Resource type Note: This field may return null, indicating that no valid values can be obtained. |
| Name | String | No | Engine name Note: This field may return null, indicating that no valid values can be obtained. |
| Instance | String | No | If the resource type is spark-sql, the value is Name; if the resource type is spark-batch, the value is session app_name. Note: This field may return null, indicating that no valid values can be obtained. |
| Favor | Array of FavorInfo | No | Affinity Note: This field may return null, indicating that no valid values can be obtained. |
| Status | Integer | No | Status Note: This field may return null, indicating that no valid values can be obtained. |
SQL query task
Used by actions: CreateTask.
| Name | Type | Required | Description |
|---|---|---|---|
| SQL | String | Yes | Base64-encrypted SQL statement |
| Config | Array of KVPair | No | Task configuration information |
The session resource configuration template for a Spark cluster.
Used by actions: CreateDataEngine, DescribeDataEngine, DescribeUserDataEngineConfig, UpdateDataEngine, UpdateUserDataEngineConfig.
| Name | Type | Required | Description |
|---|---|---|---|
| DriverSize | String | No | The driver size. Valid values for the standard resource type: small, medium, large, and xlarge.Valid values for the memory resource type: m.small, m.medium, m.large, and m.xlarge.Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutorSize | String | No | The executor size. Valid values for the standard resource type: small, medium, large, and xlarge.Valid values for the memory resource type: m.small, m.medium, m.large, and m.xlarge.Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutorNums | Integer | No | The executor count. The minimum value is 1 and the maximum value is less than the cluster specification. Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutorMaxNumbers | Integer | No | The maximum executor count (in dynamic mode). The minimum value is 1 and the maximum value is less than the cluster specification. If you set ExecutorMaxNumbers to a value smaller than that of ExecutorNums, the value of ExecutorMaxNumbers is automatically changed to that of ExecutorNums.Note: This field may return null, indicating that no valid values can be obtained. |
SmartOptimizerIndexPolicy
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| IndexEnable | String | No | Enable index. Note: This field may return null, indicating that no valid values can be obtained. |
SmartOptimizerLifecyclePolicy
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| LifecycleEnable | String | No | Enable the lifecycle Note: This field may return null, indicating that no valid values can be obtained. |
| Expiration | Integer | No | Expiration time Note: This field may return null, indicating that no valid values can be obtained. |
| DropTable | Boolean | No | Whether to delete the table Note: This field may return null, indicating that no valid values can be obtained. |
| ExpiredField | String | No | Expired fields Note: This field may return null, indicating that no valid values can be obtained. |
| ExpiredFieldFormat | String | No | Expired field format Note: This field may return null, indicating that no valid values can be obtained. |
SmartOptimizerPolicy
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| Inherit | String | No | Whether to inherit it Note: This field may return null, indicating that no valid values can be obtained. |
| Resources | Array of ResourceInfo | No | ResourceInfo Note: This field may return null, indicating that no valid values can be obtained. |
| Written | SmartOptimizerWrittenPolicy | No | SmartOptimizerWrittenPolicy Note: This field may return null, indicating that no valid values can be obtained. |
| Lifecycle | SmartOptimizerLifecyclePolicy | No | SmartOptimizerLifecyclePolicy Note: This field may return null, indicating that no valid values can be obtained. |
| Index | SmartOptimizerIndexPolicy | No | SmartOptimizerIndexPolicy Note: This field may return null, indicating that no valid values can be obtained. |
SmartOptimizerWrittenPolicy
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| WrittenEnable | String | No | none/enable/disable/default Note: This field may return null, indicating that no valid values can be obtained. |
SmartPolicyRequest
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| BaseInfo | SmartPolicyBaseInfo | No | Basic information Note: This field may return null, indicating that no valid values can be obtained. |
| Policy | SmartOptimizerPolicy | No | Policy description Note: This field may return null, indicating that no valid values can be obtained. |
SmartPolicyBaseInfo
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql, GetOptimizerPolicy.
| Name | Type | Required | Description |
|---|---|---|---|
| Uin | String | Yes | User UIN Note: This field may return null, indicating that no valid values can be obtained. |
| PolicyType | String | No | Catalog/Database/Table Note: This field may return null, indicating that no valid values can be obtained. |
| Catalog | String | No | Catalog name Note: This field may return null, indicating that no valid values can be obtained. |
| Database | String | No | Database name Note: This field may return null, indicating that no valid values can be obtained. |
| Table | String | No | Table name Note: This field may return null, indicating that no valid values can be obtained. |
| AppId | String | No | User AppID Note: This field may return null, indicating that no valid values can be obtained. |
Spark job details
Used by actions: DescribeSparkAppJob, DescribeSparkAppJobs.
| Name | Type | Description |
|---|---|---|
| JobId | String | Spark job ID |
| JobName | String | Spark job name |
| JobType | Integer | Spark job type. Valid values: 1 (batch job), 2 (streaming job). |
| DataEngine | String | Engine name |
| Eni | String | This field has been disused. Use the Datasource field instead. |
| IsLocal | String | Whether the program package is uploaded locally. Valid values: cos, lakefs. |
| JobFile | String | Program package path |
| RoleArn | Integer | Role ID |
| MainClass | String | Main class of Spark job execution |
| CmdArgs | String | Command line parameters of the Spark job separated by space |
| JobConf | String | Native Spark configurations separated by line break |
| IsLocalJars | String | Whether the dependency JAR packages are uploaded locally. Valid values: cos, lakefs. |
| JobJars | String | Dependency JAR packages of the Spark job separated by comma |
| IsLocalFiles | String | Whether the dependency file is uploaded locally. Valid values: cos, lakefs. |
| JobFiles | String | Dependency files of the Spark job separated by comma |
| JobDriverSize | String | Driver resource size of the Spark job |
| JobExecutorSize | String | Executor resource size of the Spark job |
| JobExecutorNums | Integer | Number of Spark job executors |
| JobMaxAttempts | Integer | Maximum number of retries of the Spark flow task |
| JobCreator | String | Spark job creator |
| JobCreateTime | Integer | Spark job creation time |
| JobUpdateTime | Integer | Spark job update time |
| CurrentTaskId | String | Last task ID of the Spark job |
| JobStatus | Integer | Last status of the Spark job |
| StreamingStat | StreamingStatistics | Note: This field may return null, indicating that no valid values can be obtained. |
| DataSource | String | Note: This field may return null, indicating that no valid values can be obtained. |
| IsLocalPythonFiles | String | Note: This field may return null, indicating that no valid values can be obtained. |
| AppPythonFiles | String | Note: This field may return null, indicating that no valid values can be obtained. |
| IsLocalArchives | String | Note: This field may return null, indicating that no valid values can be obtained. |
| JobArchives | String | Note: This field may return null, indicating that no valid values can be obtained. |
| SparkImage | String | Note: This field may return null, indicating that no valid values can be obtained. |
| JobPythonFiles | String | Note: This field may return null, indicating that no valid values can be obtained. |
| TaskNum | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineStatus | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| JobExecutorMaxNumbers | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| SparkImageVersion | String | Note: This field may return null, indicating that no valid values can be obtained. |
| SessionId | String | Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineClusterType | String | Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineImageVersion | String | Note: This field may return null, indicating that no valid values can be obtained. |
| IsInherit | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| IsSessionStarted | Boolean | Note: This field may return null, indicating that no valid values can be obtained. |
| EngineTypeDetail | String | Note: This field may return null, indicating that no valid values can be obtained. |
Spark monitoring metrics
Used by actions: DescribeSparkAppTasks, DescribeTasks.
| Name | Type | Description |
|---|---|---|
| ShuffleWriteBytesCos | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| ShuffleWriteBytesTotal | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
Running logs of a Spark SQL batch job
Used by actions: DescribeSparkSessionBatchSqlLog.
| Name | Type | Description |
|---|---|---|
| Step | String | The log step. Valid values: BEG, CS, DS, DSS, DSF, FINF, RTO, CANCEL, CT, DT, DTS, DTF, FINT, and EXCE.Note: This field may return null, indicating that no valid values can be obtained. |
| Time | String | Time. Note: This field may return null, indicating that no valid values can be obtained. |
| Message | String | The log message. Note: This field may return null, indicating that no valid values can be obtained. |
| Operate | Array of SparkSessionBatchLogOperate | The operation. Note: This field may return null, indicating that no valid values can be obtained. |
Operation information in the logs of a Spark SQL batch job
Used by actions: DescribeSparkSessionBatchSqlLog.
| Name | Type | Description |
|---|---|---|
| Text | String | The operation message. Note: This field may return null, indicating that no valid values can be obtained. |
| Operate | String | The operation type. Valid values: COPY, LOG, UI, RESULT, List, and TAB.Note: This field may return null, indicating that no valid values can be obtained. |
| Supplement | Array of KVPair | Additional information, such as taskid, sessionid, and sparkui. Note: This field may return null, indicating that no valid values can be obtained. |
Statement information
Used by actions: CreateSparkSessionBatchSQL.
| Name | Type | Description |
|---|---|---|
| TaskId | String | Unique ID of the SQL task Note: This field may return null, indicating that no valid values can be obtained. |
| SQL | String | SQL content Note: This field may return null, indicating that no valid values can be obtained. |
Task statistics information.
Used by actions: DescribeJob, DescribeJobs, DescribeTaskStatistics.
| Name | Type | Required | Description |
|---|---|---|---|
| TaskId | String | No | Task ID Note: This field may return null, indicating that no valid values can be obtained. |
| TotalProcessedBytes | Integer | No | Amount of data scanned Note: This field may return null, indicating that no valid values can be obtained. |
| UsedTime | Integer | No | Calculation time: ms Note: This field may return null, indicating that no valid values can be obtained. |
| CreateTime | Integer | No | Creation Time Note: This field may return null, indicating that no valid values can be obtained. |
| EndTime | Integer | No | Task end time Note: This field may return null, indicating that no valid values can be obtained. |
| StartTime | Integer | No | Task start time Note: This field may return null, indicating that no valid values can be obtained. |
| RowsAffect | Integer | No | Number of rows affected Note: This field may return null, indicating that no valid values can be obtained. |
| TotalTime | Integer | No | Task duration Note: This field may return null, indicating that no valid values can be obtained. |
Statistics of the Spark flow task
Used by actions: DescribeSparkAppJob, DescribeSparkAppJobs.
| Name | Type | Description |
|---|---|---|
| StartTime | String | Task start time |
| Receivers | Integer | Number of data receivers |
| NumActiveReceivers | Integer | Number of receivers in service |
| NumInactiveReceivers | Integer | Number of inactive receivers |
| NumActiveBatches | Integer | Number of running batches |
| NumRetainedCompletedBatches | Integer | Number of batches to be processed |
| NumTotalCompletedBatches | Integer | Number of completed batches |
| AverageInputRate | Float | Average input speed |
| AverageSchedulingDelay | Float | Average queue time |
| AverageProcessingTime | Float | Average processing time |
| AverageTotalDelay | Float | Average latency |
Table field information
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql.
| Name | Type | Required | Description |
|---|---|---|---|
| Name | String | Yes | Field name Note: This field may return null, indicating that no valid values can be obtained. |
| Type | String | Yes | Field type Note: This field may return null, indicating that no valid values can be obtained. |
| Comment | String | No | Field description Note: This field may return null, indicating that no valid values can be obtained. |
| Default | String | No | Field default values Note: This field may return null, indicating that no valid values can be obtained. |
| NotNull | Boolean | No | Is the field non-empty? Note: This field may return null, indicating that no valid values can be obtained. |
| Precision | Integer | No | Indicate the length of the entire numeric with a value of 1-38 Note: This field may return null, indicating that no valid values can be obtained. |
| Scale | Integer | No | Indicates the length of the decimal part. Scale is smaller than Precision. Note: This field may return null, indicating that no valid values can be obtained. |
| Position | Integer | No | Field position: Fields with smaller values need to be placed first. Note: This field may return null, indicating that no valid values can be obtained. |
| IsPartition | Boolean | No | Whether it is a partition field Note: This field may return null, indicating that no valid values can be obtained. |
Table partition information
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql.
| Name | Type | Required | Description |
|---|---|---|---|
| Name | String | Yes | The field name. |
| Type | String | No | The field type. |
| Comment | String | No | The field description. |
| PartitionType | String | No | The partition type. |
| PartitionFormat | String | No | The partition format. |
| PartitionDot | Integer | No | The separator count of the partition conversion policy. |
| Transform | String | No | The partition conversion policy. |
| TransformArgs | Array of String | No | The policy parameters. |
Table configurations
Used by actions: CreateDLCTable, CreateInternalTable, DescribeDLCTable, DescribeDLCTableList, GenerateCreateMangedTableSql.
| Name | Type | Required | Description |
|---|---|---|---|
| DatabaseName | String | Yes | The database name. |
| TableName | String | Yes | The table name. |
| DatasourceConnectionName | String | No | The data source name. Note: This field may return null, indicating that no valid values can be obtained. |
| TableComment | String | No | The table remarks. Note: This field may return null, indicating that no valid values can be obtained. |
| Type | String | No | The specific type: table or view.Note: This field may return null, indicating that no valid values can be obtained. |
| TableFormat | String | No | The data format type, such as hive and iceberg.Note: This field may return null, indicating that no valid values can be obtained. |
| UserAlias | String | No | The table creator name. Note: This field may return null, indicating that no valid values can be obtained. |
| UserSubUin | String | No | The table creator ID. Note: This field may return null, indicating that no valid values can be obtained. |
| SmartPolicy | SmartPolicy | No | Smart data governance configuration items Note: This field may return null, indicating that no valid values can be obtained. |
Tag pair info
Used by actions: CreateDataEngine, DescribeDataEngine.
| Name | Type | Required | Description |
|---|---|---|---|
| TagKey | String | No | The tag key. Note: This field may return null, indicating that no valid values can be obtained. |
| TagValue | String | No | The tag value. Note: This field may return null, indicating that no valid values can be obtained. |
Task type, such as SQL query.
Used by actions: CreateTask.
| Name | Type | Required | Description |
|---|---|---|---|
| SQLTask | SQLTask | No | SQL query task |
| SparkSQLTask | SQLTask | No | Spark SQL query task |
The task instance.
Used by actions: DescribeSparkAppTasks, DescribeTasks.
| Name | Type | Description |
|---|---|---|
| DatabaseName | String | Database name of the task |
| DataAmount | Integer | Data volume of the task |
| Id | String | Task ID |
| UsedTime | Integer | The compute time in ms. |
| OutputPath | String | Task output path |
| CreateTime | String | Task creation time |
| State | Integer | The task status. Valid values: 0 (initializing), 1 (executing), 2 (executed), 3 (writing data), 4 (queuing), -1 (failed), and -3 (canceled). |
| SQLType | String | SQL statement type of the task, such as DDL and DML. |
| SQL | String | SQL statement of the task |
| ResultExpired | Boolean | Whether the result has expired |
| RowAffectInfo | String | Number of affected data rows |
| DataSet | String | Note: This field may return null, indicating that no valid values can be obtained. |
| Error | String | Failure information, such as errorMessage. This field has been disused. |
| Percentage | Integer | Task progress (%) |
| OutputMessage | String | Output information of task execution |
| TaskType | String | Type of the engine executing the SQL statement |
| ProgressDetail | String | Note: This field may return null, indicating that no valid values can be obtained. |
| UpdateTime | String | Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineId | String | Note: This field may return null, indicating that no valid values can be obtained. |
| OperateUin | String | Note: This field may return null, indicating that no valid values can be obtained. |
| DataEngineName | String | Note: This field may return null, indicating that no valid values can be obtained. |
| InputType | String | Note: This field may return null, indicating that no valid values can be obtained. |
| InputConf | String | Note: This field may return null, indicating that no valid values can be obtained. |
| DataNumber | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| CanDownload | Boolean | Note: This field may return null, indicating that no valid values can be obtained. |
| UserAlias | String | Note: This field may return null, indicating that no valid values can be obtained. |
| SparkJobName | String | Note: This field may return null, indicating that no valid values can be obtained. |
| SparkJobId | String | Note: This field may return null, indicating that no valid values can be obtained. |
| SparkJobFile | String | Note: This field may return null, indicating that no valid values can be obtained. |
| UiUrl | String | Note: This field may return null, indicating that no valid values can be obtained. |
| TotalTime | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| CmdArgs | String | Note: This field may return null, indicating that no valid values can be obtained. |
| ImageVersion | String | Note: This field may return null, indicating that no valid values can be obtained. |
| DriverSize | String | Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutorSize | String | Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutorNums | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| ExecutorMaxNumbers | Integer | Note: This field may return null, indicating that no valid values can be obtained. |
| CommonMetrics | CommonMetrics | Note: This field may return null, indicating that no valid values can be obtained. |
| SparkMonitorMetrics | SparkMonitorMetrics | Note: This field may return null, indicating that no valid values can be obtained. |
| PrestoMonitorMetrics | PrestoMonitorMetrics | Note: This field may return null, indicating that no valid values can be obtained. |
| ResultFormat | String | Note: This field may return null, indicating that no valid values can be obtained. |
| EngineTypeDetail | String | Note: This field may return null, indicating that no valid values can be obtained. |
| ResourceGroupName | String | Note: This field may return null, indicating that no valid values can be obtained. |
The task result information.
Used by actions: DescribeTaskResult.
| Name | Type | Description |
|---|---|---|
| TaskId | String | Unique task ID |
| DatasourceConnectionName | String | Name of the default selected data source when the current job is executed Note: This field may return null, indicating that no valid values can be obtained. |
| DatabaseName | String | Name of the default selected database when the current job is executed Note: This field may return null, indicating that no valid values can be obtained. |
| SQL | String | The currently executed SQL statement. Each task contains one SQL statement. |
| SQLType | String | Type of the executed task. Valid values: DDL, DML, DQL. |
| State | Integer | u200cThe current task status. Valid values: 0 (initializing), 1 (executing), 2 (executed), 3 (writing data), 4 (queuing), u200c-1 (failed), and -3 (canceled). Only when the task is successfully executed, a task execution result will be returned. |
| DataAmount | Integer | Amount of the data scanned in bytes |
| UsedTime | Integer | The compute time in ms. |
| OutputPath | String | Address of the COS bucket for storing the task result |
| CreateTime | String | Task creation timestamp |
| OutputMessage | String | Task execution information. success will be returned if the task succeeds; otherwise, the failure cause will be returned. |
| RowAffectInfo | String | Number of affected rows |
| ResultSchema | Array of Column | Schema information of the result Note: This field may return null, indicating that no valid values can be obtained. |
| ResultSet | String | Result information. After it is unescaped, each element of the outer array is a data row. Note: This field may return null, indicating that no valid values can be obtained. |
| NextToken | String | Pagination information. If there is no more result data, nextToken will be empty. |
| Percentage | Integer | Task progress (%) |
| ProgressDetail | String | Task progress details |
| DisplayFormat | String | Console display format. Valid values: table, text. |
| TotalTime | Integer | The task time in ms. |
| QueryResultTime | Float | Time consumed to get results Note: This field may return null, indicating that no valid values can be obtained. |
Collection of tasks executed sequentially in batches
Used by actions: CreateTasks.
| Name | Type | Required | Description |
|---|---|---|---|
| TaskType | String | Yes | Task type. Valid values: SQLTask (SQL query task), SparkSQLTask (Spark SQL query task). |
| FailureTolerance | String | Yes | Fault tolerance policy. Proceed: continues to execute subsequent tasks after the current task fails or is canceled. Terminate: terminates the execution of subsequent tasks after the current task fails or is canceled, and marks all subsequent tasks as canceled. |
| SQL | String | Yes | Base64-encrypted SQL statements separated by ";". Up to 50 tasks can be submitted at a time, and they will be executed strictly in sequence. |
| Config | Array of KVPair | No | Configuration information of the task. Currently, only SparkSQLTask tasks are supported. |
| Params | Array of KVPair | No | User-defined parameters of the task |
The task overview.
Used by actions: DescribeTasks.
| Name | Type | Description |
|---|---|---|
| TaskQueuedCount | Integer | The number of tasks in queue. |
| TaskInitCount | Integer | The number of initialized tasks. |
| TaskRunningCount | Integer | The number of tasks in progress. |
| TotalTaskCount | Integer | The total number of tasks in this time range. |
Detailed user information
Used by actions: DescribeUserInfo.
| Name | Type | Description |
|---|---|---|
| UserId | String | User ID Note: This field may return null, indicating that no valid values can be obtained. |
| Type | String | Types of returned information. Group: returned information about the working group where the current user is; DataAuth: returned information about the current user's data permission; EngineAuth: returned information about the current user's engine permission Note: This field may return null, indicating that no valid values can be obtained. |
| UserType | String | Types of users. ADMIN: administrators; COMMON: general users Note: This field may return null, indicating that no valid values can be obtained. |
| UserDescription | String | User description Note: This field may return null, indicating that no valid values can be obtained. |
| DataPolicyInfo | Policys | Collection of data permission information Note: This field may return null, indicating that no valid values can be obtained. |
| EnginePolicyInfo | Policys | Collection of engine permissions Note: This field may return null, indicating that no valid values can be obtained. |
| WorkGroupInfo | WorkGroups | Information about collections of working groups bound to the user Note: This field may return null, indicating that no valid values can be obtained. |
| UserAlias | String | User alias Note: This field may return null, indicating that no valid values can be obtained. |
| RowFilterInfo | Policys | Collection of filtered rows Note: This field may return null, indicating that no valid values can be obtained. |
Collection of IDs of users bound to the same working group
Used by actions: AddUsersToWorkGroup, DeleteUsersFromWorkGroup.
| Name | Type | Required | Description |
|---|---|---|---|
| WorkGroupId | Integer | Yes | Working group IDs |
| UserIds | Array of String | Yes | User ID which matches the Uin on the CAM side |
Information about the authorized user
Used by actions: DescribeUsers.
| Name | Type | Description |
|---|---|---|
| UserId | String | User ID, which is the same as the sub-user UIN |
| UserDescription | String | User description, which can make it easy to identify different users. Note: This field may return null, indicating that no valid values can be obtained. |
| PolicySet | Array of Policy | Collection of permissions solely bound to users Note: This field may return null, indicating that no valid values can be obtained. |
| Creator | String | Creator of the current user |
| CreateTime | String | Creation time, and the example of the format is at 16:19:32 on Jul 28, 2021. |
| WorkGroupSet | Array of WorkGroupMessage | Collection of associated working groups Note: This field may return null, indicating that no valid values can be obtained. |
| IsOwner | Boolean | Whether it is the root account Note: This field may return null, indicating that no valid values can be obtained. |
| UserType | String | Types of users. ADMIN: administrators; COMMON: general users Note: This field may return null, indicating that no valid values can be obtained. |
| UserAlias | String | User alias Note: This field may return null, indicating that no valid values can be obtained. |
Partial information of the user
Used by actions: DescribeWorkGroupInfo, DescribeWorkGroups.
| Name | Type | Description |
|---|---|---|
| UserId | String | User Id which matches the sub-user UIN on the CAM side |
| UserDescription | String | User description Note: The returned value of this field may be null, indicating that no valid value is obtained. |
| Creator | String | The creator of the current user |
| CreateTime | String | The creation time of the current user, e.g. 16:19:32, July 28, 2021 |
| UserAlias | String | User alias |
User role
Used by actions: DescribeUserRoles.
| Name | Type | Description |
|---|---|---|
| RoleId | Integer | The role ID. |
| AppId | String | The user's app ID. |
| Uin | String | The user ID. |
| Arn | String | The role permission. |
| ModifyTime | Integer | The last modified timestamp. |
| Desc | String | The role description. |
| RoleName | String | Note: This field may return null, indicating that no valid values can be obtained. |
| Creator | String | Note: This field may return null, indicating that no valid values can be obtained. |
| CosPermissionList | Array of CosPermission | Note: This field may return null, indicating that no valid values can be obtained. |
| PermissionJson | String | Note: This field may return null, indicating that no valid values can be obtained. |
Collection of user information
Used by actions: DescribeWorkGroupInfo.
| Name | Type | Description |
|---|---|---|
| UserSet | Array of UserMessage | Collection of user information Note: This field may return null, indicating that no valid values can be obtained. |
| TotalCount | Integer | Total users Note: This field may return null, indicating that no valid values can be obtained. |
VPC subnet information
Used by actions: DescribeOtherCHDFSBindingList.
| Name | Type | Required | Description |
|---|---|---|---|
| CidrId | String | No | Subnet ID Note: This field may return null, indicating that no valid values can be obtained. |
| CidrAddr | String | No | Subnet IP range Note: This field may return null, indicating that no valid values can be obtained. |
VPC information
Used by actions: CreateCHDFSBindingProduct, DeleteCHDFSBindingProduct.
| Name | Type | Required | Description |
|---|---|---|---|
| VpcId | String | No | vpc Id Note: This field may return null, indicating that no valid values can be obtained. |
| VpcCidrBlock | String | No | VPC subnet Note: This field may return null, indicating that no valid values can be obtained. |
| RuleId | Integer | No | Rule ID Note: This field may return null, indicating that no valid values can be obtained. |
| AccessGroupId | String | No | Permission group ID Note: This field may return null, indicating that no valid values can be obtained. |
Details about working groups
Used by actions: DescribeWorkGroupInfo.
| Name | Type | Description |
|---|---|---|
| WorkGroupId | Integer | Working group ID Note: This field may return null, indicating that no valid values can be obtained. |
| WorkGroupName | String | Working group name Note: This field may return null, indicating that no valid values can be obtained. |
| Type | String | Types of information included. User: user information; DataAuth: data permissions; EngineAuth: engine permissions Note: This field may return null, indicating that no valid values can be obtained. |
| UserInfo | Users | Collection of users bound to working groups Note: This field may return null, indicating that no valid values can be obtained. |
| DataPolicyInfo | Policys | Collection of data permissions Note: This field may return null, indicating that no valid values can be obtained. |
| EnginePolicyInfo | Policys | Collection of engine permissions Note: This field may return null, indicating that no valid values can be obtained. |
| WorkGroupDescription | String | Working group description Note: This field may return null, indicating that no valid values can be obtained. |
| RowFilterInfo | Policys | Collection of information about filtered rows Note: This field may return null, indicating that no valid values can be obtained. |
Collections of the working groups bound to the same user
Used by actions: BindWorkGroupsToUser, UnbindWorkGroupsFromUser.
| Name | Type | Required | Description |
|---|---|---|---|
| UserId | String | Yes | User ID, which matches Uin on the CAM side. |
| WorkGroupIds | Array of Integer | Yes | Collections of IDs of working groups |
Working group information
Used by actions: DescribeWorkGroups.
| Name | Type | Description |
|---|---|---|
| WorkGroupId | Integer | Unique ID of the working group that have been queried |
| WorkGroupName | String | Working group name |
| WorkGroupDescription | String | Working group description Note: This field may return null, indicating that no valid values can be obtained. |
| UserNum | Integer | Number of users associated with working groups |
| UserSet | Array of UserMessage | Collection of users associated with working groups Note: This field may return null, indicating that no valid values can be obtained. |
| PolicySet | Array of Policy | Collections of permissions bound to working groups Note: This field may return null, indicating that no valid values can be obtained. |
| Creator | String | Creators of working groups |
| CreateTime | String | The creation time of the working group, e.g. at 16:19:32 on Jul 28, 2021 |
Partial information about working groups
Used by actions: DescribeUserInfo, DescribeUsers.
| Name | Type | Description |
|---|---|---|
| WorkGroupId | Integer | Unique ID of the working group |
| WorkGroupName | String | Working group name |
| WorkGroupDescription | String | Working group description Note: This field may return null, indicating that no valid values can be obtained. |
| Creator | String | Creator |
| CreateTime | String | The creation time of the working group, e.g. at 16:19:32 on Jul 28, 2021 |
Collections of working groups
Used by actions: DescribeUserInfo.
| Name | Type | Description |
|---|---|---|
| WorkGroupSet | Array of WorkGroupMessage | Collection of working group information Note: This field may return null, indicating that no valid values can be obtained. |
| TotalCount | Integer | Total working groups Note: This field may return null, indicating that no valid values can be obtained. |
Feedback