产品概述
基本概念
集群架构
产品优势
应用场景
doriswriter/doriswriter/doc。init-env.shdoriswriter/ 目录软链到 DataX/doriswriter 目录。DataX/pom.xml 文件中添加 <module>doriswriter</module> 模块。DataX/core/pom.xml 文件中的 httpclient 版本从 4.5 改为 4.5.13。DataX/ 目录开始开发或编译了。因为做了软链，所以任何对 DataX/doriswriter 目录中文件的修改，都会反映到 doriswriter/ 目录中，方便开发者提交代码。init-env.sh。DataX/doriswriter 中的代码。mvn clean install -pl plugin-rdbms-util,doriswriter -DskipTests
mvn package assembly:assembly -Dmaven.test.skip=true
target/datax/datax/.DataX/pom.xml 中删除这些插件的模块。Could not find artifact com.alibaba.datax:datax-all:pom:0.0.1-SNAPSHOT ...
alibaba/datax/ 目录，拷贝到所使用的 maven 对应的 .m2/repository/com/alibaba/ 下。CREATE TABLE `t_test`(`id`bigint(30) NOT NULL,`order_code` varchar(30) DEFAULT NULL COMMENT '',`line_code` varchar(30) DEFAULT NULL COMMENT '',`remark` varchar(30) DEFAULT NULL COMMENT '',`unit_no` varchar(30) DEFAULT NULL COMMENT '',`unit_name` varchar(30) DEFAULT NULL COMMENT '',`price` decimal(12,2) DEFAULT NULL COMMENT '',PRIMARY KEY(`id`) USING BTREE)ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC COMMENT='';
CREATE TABLE `ods_t_test` (`id`bigint(30) NOT NULL,`order_code` varchar(30) DEFAULT NULL COMMENT '',`line_code` varchar(30) DEFAULT NULL COMMENT '',`remark` varchar(30) DEFAULT NULL COMMENT '',`unit_no` varchar(30) DEFAULT NULL COMMENT '',`unit_name` varchar(30) DEFAULT NULL COMMENT '',`price` decimal(12,2) DEFAULT NULL COMMENT '')ENGINE=OLAPUNIQUE KEY(`id`, `order_code`)DISTRIBUTED BY HASH(`order_code`) BUCKETS 1PROPERTIES ("replication_allocation" = "tag.location.default: 3","in_memory" = "false","storage_format" = "V2");
import_t_test.json{"job": {"setting": {"speed": {"channel": 1},"errorLimit": {"record": 0,"percentage": 0}},"content": [{"reader": {"name": "mysqlreader","parameter": {"username": "xxx","password": "xxx","column": ["id","order_code","line_code","remark","unit_no","unit_name","price"],"connection": [ { "table": [ "t_test" ], "jdbcUrl": [ "jdbc:mysql://10.10.10.1:3306/demo" ] } ] }},"writer": {"name": "doriswriter","parameter": {"feLoadUrl": ["127.0.0.1:8030","127.0.0.2:8030"],"beLoadUrl": ["127.0.0.3:8040","127.0.0.4:8040","127.0.0.5:8040"],"jdbcUrl": "jdbc:mysql://127.0.0.1:9030/","database": "demo","table": "ods_t_test","column": ["id","order_code","line_code","remark","unit_no","unit_name","price"],"username": "xxx","password": "xxx","postSql": [],"preSql": [],"loadProps": {},"maxBatchRows" : 300000,"maxBatchByteSize" : 20971520}}}]}}
文档反馈