You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
|
|
|
|
# V2.3.5
|
|
|
|
|
|
### 1.doris自动建表错误 （不要使用自动建表功能）
|
|
|
|
|
|
配置如下
|
|
|
|
|
|
```hocon
|
|
|
|
|
|
env {
|
|
|
|
|
|
execution.parallelism = 1
|
|
|
|
|
|
job.mode = "BATCH"
|
|
|
|
|
|
}
|
|
|
|
|
|
source {
|
|
|
|
|
|
Jdbc {
|
|
|
|
|
|
url = "jdbc:mysql://172.31.51.134:3306/dcm_zhongshan?characterEncoding=UTF-8&connectTimeout=5000&useSSL=false&allowPublicKeyRetrieval=true&zeroDateTimeBehavior=convertToNull"
|
|
|
|
|
|
driver = "com.mysql.cj.jdbc.Driver"
|
|
|
|
|
|
user = "root"
|
|
|
|
|
|
password = "Admin123"
|
|
|
|
|
|
query = "select * from acc_d_sjzd"
|
|
|
|
|
|
connection_check_timeout_sec = 100
|
|
|
|
|
|
result_table_name = "source_result"
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
transform{
|
|
|
|
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
sink {
|
|
|
|
|
|
jdbc {
|
|
|
|
|
|
url = "jdbc:mysql://172.31.51.142:9030/bjbj"
|
|
|
|
|
|
driver = "com.mysql.cj.jdbc.Driver"
|
|
|
|
|
|
user = "admin"
|
|
|
|
|
|
password = "6G_FahdUxAh@K"
|
|
|
|
|
|
database="bjbj"
|
|
|
|
|
|
table="bjbj.test_acc_d_sjzd"
|
|
|
|
|
|
generate_sink_sql="true"
|
|
|
|
|
|
schema_save_mode="CREATE_SCHEMA_WHEN_NOT_EXIST"
|
|
|
|
|
|
}
|
|
|
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
使用了自动建表功能，从mysql导入数据到doris报错，查看日志发现生成的建表语句字段类型映射有问题，doris没有TEXT和LONGTEXT类型并且没有指定分桶键
|
|
|
|
|
|
![[Pasted image 20240926165648.png]]
|
|
|
|
|
|
|
|
|
|
|
|
### 2.日期时间格式参数替换问题（-i 传递的参数不能带空格）
|
|
|
|
|
|
官方文档写的变量替换方式，传入日期时间自动替换后时间丢失
|
|
|
|
|
|
![[Pasted image 20240926172216.png]]
|
|
|
|
|
|
![[Pasted image 20240926172157.png]]
|
|
|
|
|
|
![[Pasted image 20240926172306.png]]
|
