tencent cloud

Tencent Cloud WeData

DocumentationTencent Cloud WeDataOperation GuideData IntegrationOffline Synchronization Task Configuration and Operation and MaintenanceData Sources Supported by Offline SynchronizationData Source ListBigQuery data source

BigQuery data source

PDF
Focus Mode
Font Size
Last updated: 2025-07-25 17:18:59

Use Limits

BigQuery data source is only supported in Hong Kong (China), Singapore, usa (Virginia), and germany (Frankfurt). Other regions are not supported yet.
The split key only supports INT64, FLOAT64, TIMESTAMP, BIGNUMERIC, NUMERIC, DATE, and DATETIME.

BigQuery offline single table read node configuration




Parameters
Description
Data Source
Select the configured BigQuery data source from the source end.
Dataset
Support selecting or manually entering the dataset name to be read.
The database bound to the data source is set as the default library by default, other databases require manual input of the dataset name.
When the data source network is not connected, causing database information to be unavailable for direct fetching, you can manually enter the dataset name. Data synchronization can still be performed given network connection in DataInLong.
Table
Supports selecting or manually entering the table name to be read.
If the data source network is not connected and table information cannot be fetched directly, you can manually enter the table name. Data synchronization can still be performed when the Data Integration network is connected.
Split Key
Specify the field for data sharding. After specifying, concurrent tasks will be launched for data synchronization. You can use a column in the source data table as the partition key. It is recommended to use the primary key or indexed column as the partition key.
Supported split key types: INT64, FLOAT64, TIMESTAMP, BIGNUMERIC, NUMERIC, DATE, DATETIME.
Filter Conditions (Optional)
Fill in the corresponding filter statement based on the data type. This statement will serve as the filter condition for the data to be synchronized.

Data type conversion support

Read

BigQuery data types.
Internal Types
BOOLEAN
BOOL
INTEGER
LONG
FLOAT、BIGNUMERIC、NUMERIC
DOUBLE
STRING、TIMESTAMP、DATE、TIME、GEOGRAPHY、JSON、INTERVAL、RECORD
STRING

BigQuery script demo

If you configure offline tasks using script mode, you need to write the reader parameters in the task script according to the unified script format requirements.


        
            
                
                    
                        "databaseName": "source_database", //source database
                        "credential": { //bigquery data source configuration information
                            
                            
                            
                            
                            
                            
                            
                            
                            
                            
                            
                        
                        "column": [ //column name 
                            
                            
                        
                         
                        "tableName": "source_table" // Source table
                    
                    
                
                
            
        
        
            "memory": { // Memory settings
                
                
                
                
                
            
            "errorLimit": { // Dirty data threshold
                
            
            
                "byte": -1, // No limit on sync speed, positive integer indicates maximum transfer speed in byte/s
                "channel": 1 // Concurrency limit
            
        
    


Previous Topic: TBase Data SourceNext Topic: GBase Data Source

Help and Support

Was this page helpful?

Help us improve! Rate your documentation experience in 5 mins.

Feedback