WeData Data Development provides an efficient, collaborative, and intelligent data development platform for data engineers and related personnel. The WeData Data Development module is designed for various engines (EMR, DLC, etc.) by offering capabilities such as script development, visual development, task orchestration, task publishing, and task operation and maintenance. It helps organizations and enterprises efficiently build data warehouses.
Use Limits
1. The computing tasks available in WeData Data Development depend on the compute engine type bound to the project. For EMR engines, you need to start Hive and Spark services, allowing the use of HiveSQL, SparkSQL, and Spark task types for development. Other tasks in WeData based on the EMR engine, such as Trino, also require the corresponding EMR services to be enabled. For DLC engines, you can use DLC SQL and DLC Spark task types. For other types of relational databases, if data development is required in WeData, a data source must be created in WeData, and SQL scripts must be created using JDBC tasks. WeData Data Development is responsible for the execution of these scripts.
2. Each compute engine has its own data permission control system. WeData does not perform specific data permission control but interfaces with the permissions of various engines through account interoperability. For example, permissions regarding EMR data in WeData are realized through account mapping.