Commit f389005
committed
chore(connector-file,transforms-v2): adapt to upstream/dev API changes after rebase
Mechanical fixups so the rebased PR branch compiles and all schema-evolution tests pass
on top of latest upstream/dev. No behaviour change in the schema-evolution feature itself.
Adapted to upstream API changes:
- CanalJsonSerializationSchema / DebeziumJsonSerializationSchema /
MaxWellJsonSerializationSchema constructors now require a third arg (mergeUpdateEventFlag)
when called with a Charset. onSchemaChanged() in each strategy now passes the field that
was already stored in the writer.
- FileSinkConfig constructor signature changed to take ReadonlyConfig (was Config).
Test sites in FileSchemaEvolutionTest, ParquetWriteStrategyEvolutionTest, and
ParquetTypeCoercionTest wrapped with ReadonlyConfig.fromConfig(...). Imports added.
- FieldFieldMultiCatalogTransform was renamed to FilterFieldMultiCatalogTransform upstream.
ChainTimestampPreservationTest and ProductionPipelineSchemaChangeTest updated accordingly.
Test contract updates following the partition_by guard removal in this PR:
- testPartitionByWithSchemaEvolutionEnabledThrowsAtConfig replaced with
testPartitionByWithSchemaEvolutionEnabledIsAccepted — partition_by + schema_evolution
is now supported (writer-local partitionFieldsIndexInRow rebuilds on every ALTER via
name-based lookup; drop/rename of partition column is the only case rejected, with an
explicit IllegalStateException at rebuild time).
Restored applySchemaChange's silent no-op behavior when schema_evolution_enabled=false.
The fail-fast variant is left as an open design question for maintainers — current
behavior preserves backward compatibility but leaves a latent ClassCastException trap
when schema-changes.enabled=true at source + schema_evolution_enabled=false at sink
(documented inline). Awaiting maintainer guidance on the preferred UX.
All 30 schema-evolution tests pass:
- FileSchemaEvolutionTest (19)
- ParquetWriteStrategyEvolutionTest (2), ParquetTypeCoercionTest (3)
- ChainTimestampPreservationTest, MetadataMultiCatalogSchemaChangeTest,
ProductionPipelineSchemaChangeTest, TransformChainLiveAlterTest,
SQLMultiCatalogSchemaChangeTest1 parent 7a1b8fc commit f389005
10 files changed
Lines changed: 43 additions & 39 deletions
File tree
- seatunnel-connectors-v2/connector-file/connector-file-base/src
- main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer
- test/java/org/apache/seatunnel/connectors/seatunnel/file/writer
- seatunnel-transforms-v2/src
- main/java/org/apache/seatunnel/transform/common
- test/java/org/apache/seatunnel/transform/metadata
Lines changed: 6 additions & 17 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
44 | 43 | | |
45 | 44 | | |
46 | 45 | | |
| |||
193 | 192 | | |
194 | 193 | | |
195 | 194 | | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | | - | |
200 | | - | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
206 | | - | |
207 | | - | |
208 | | - | |
209 | | - | |
210 | | - | |
211 | | - | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
212 | 201 | | |
213 | 202 | | |
214 | 203 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
75 | 77 | | |
76 | 78 | | |
77 | 79 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
75 | 77 | | |
76 | 78 | | |
77 | 79 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
75 | 77 | | |
76 | 78 | | |
77 | 79 | | |
| |||
Lines changed: 11 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
488 | 488 | | |
489 | 489 | | |
490 | 490 | | |
491 | | - | |
| 491 | + | |
| 492 | + | |
492 | 493 | | |
493 | 494 | | |
494 | 495 | | |
| |||
654 | 655 | | |
655 | 656 | | |
656 | 657 | | |
657 | | - | |
| 658 | + | |
658 | 659 | | |
659 | 660 | | |
660 | 661 | | |
661 | 662 | | |
662 | | - | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
663 | 668 | | |
664 | 669 | | |
665 | 670 | | |
666 | 671 | | |
667 | 672 | | |
668 | 673 | | |
669 | | - | |
670 | | - | |
671 | | - | |
672 | | - | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
673 | 677 | | |
674 | 678 | | |
675 | 679 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
214 | 215 | | |
215 | 216 | | |
216 | 217 | | |
217 | | - | |
| 218 | + | |
| 219 | + | |
218 | 220 | | |
219 | 221 | | |
220 | 222 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
89 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
90 | 93 | | |
91 | 94 | | |
92 | 95 | | |
| |||
187 | 190 | | |
188 | 191 | | |
189 | 192 | | |
190 | | - | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
191 | 196 | | |
192 | 197 | | |
193 | 198 | | |
| |||
Lines changed: 0 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
23 | | - | |
24 | 22 | | |
25 | 23 | | |
26 | 24 | | |
| |||
Lines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| |||
162 | 162 | | |
163 | 163 | | |
164 | 164 | | |
165 | | - | |
166 | | - | |
| 165 | + | |
| 166 | + | |
167 | 167 | | |
168 | 168 | | |
169 | 169 | | |
| |||
Lines changed: 4 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
| 36 | + | |
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
151 | | - | |
| 151 | + | |
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
155 | | - | |
156 | | - | |
| 155 | + | |
| 156 | + | |
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
| |||
0 commit comments