fix(kafka): regenerate config constraints#3002
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## release-1.1 #3002 +/- ##
===========================================
Coverage 0.00% 0.00%
===========================================
Files 73 73
Lines 9259 9259
===========================================
Misses 9259 9259 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Early review while this is still draft (classes 1–8 walked on the current head). The direction — regenerating constraints from Kafka's own 1. Empty body / no issue. Repo rule: every code PR needs an issue describing what it fixes, the failing scenario, and reproduction. Please state what the old hand-written constraints broke (rejected valid reconfigure values? missing keys? wrong types?) with one concrete example. 2. Scope: the JMX_PORT move is a second root cause bundled in. Moving 3. Upgrade path for the ParametersDefinition rename. 4. Closed-struct validation per role needs runtime evidence. Nice touches: generator README documents the DistributedConfig boundary; pinned |
|
@weicao don't modify this PR. It is still under development. |
1b80419 to
715bc77
Compare
Below are AI generated description.
Fixes #3027.
Problem
This draft PR covers two separate Kafka 1.1 problems. They are related only because both affect Kafka config/reconfigure behavior.
1. Kafka 3.x config constraints drifted from Kafka runtime configs
The old hand-written
ParametersDefinitionschema was stale and not role-aware.Concrete examples from the old schema:
listeners,controller.quorum.voters, andgroup.consumer.session.timeout.mswere absent;2.
JMX_PORTwas set as a container-wide env varThe old component definitions put
JMX_PORT=5555in the container env. That means ordinary Kafka CLI commands started inside the container can inheritJMX_PORTand try to open the same JMX port as the Kafka process.This PR moves
JMX_PORTinto the Kafka startup script path so the Kafka process still gets JMX, but ad-hoc CLI commands do not inherit it from the container env.Changes
ConfigDefwithtools/gen-cue.ParamConfigRendererto its matching role schema.string | [...string]for LIST-valued external keys while keeping non-LIST external keys asstring.JMX_PORT=5555out of component env and intoset_jvm_configuration().allow.everyone.if.no.acl.found, which is not present in the generated Kafka 3.9 schema.configs/kafka-server.prop.tplremain covered and LIST/non-LIST external contracts cannot diverge again.Local and repository validation
Current head:
6e6dc0d9f5417c1d0b180943d60edea95dc7fcaf.Passed:
helm dependency build addons/kafkahelm lint addons/kafkahelm template kafka addons/kafka --namespace kafka-pr3002configs/kafka-server.prop.tplis present inconfigs/kafka-server-constraint.cueJMX_PORT; only the Kafka startup script exports itrendersScopedSchemas: 1 test, 0 failures, 0 errorsThe full generator module is 6 tests / 1 failure at the existing
log.retention.msexpectation inrendersRealKafkaSources. The untouched parent head reproduces the same logical failure, so this is a pre-existing baseline red, not a regression introduced by this patch.These results close repository CI and the local CUE scalar/list contract only. They do not prove Controller/Broker runtime behavior.
Runtime validation still required before leaving draft
Fresh runtime validation at the current head has not started: Kubernetes invocation is 0 and runtime N=0. The current environment blocker is the fresh release-1.1 vcluster handoff owned by Mason.
The runtime target must use KubeBlocks source pin
apecloud/kubeblocks release-1.1@00a42539bc879c0748371fd51fecfb62dca77461with matching CRDs and full controller imageID/digest readback. Prior r11/r12 scenes are cleanup-closed and must not be reused.Required fresh runtime matrix:
kafka-pd-> role-specific PD rename does not cause unexpected config re-render/restart and reconfigure still works.This PR is not release-ready until the fresh environment handoff, reviewed invocation-0 launch packet, and runtime evidence are complete.