Track cluster-wide Citus version - method 4 - (shmem cache + daemon refresh)#8688
Draft
alperkocatas wants to merge 1 commit into
Draft
Track cluster-wide Citus version - method 4 - (shmem cache + daemon refresh)#8688alperkocatas wants to merge 1 commit into
alperkocatas wants to merge 1 commit into
Conversation
alperkocatas
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds cluster-wide Citus version introspection by introducing a node-local shared-memory cache of the minimum Citus version across active primaries, plus a maintenance-daemon-driven refresh loop to keep the cache warm.
Changes:
- Adds
citus_version_num()andcitus_minimum_cluster_version()SQL UDFs (plus upgrade/downgrade plumbing). - Introduces shared-memory state + LWLock-protected cache and invalidates it on
pg_dist_noderelcache invalidations. - Extends the maintenance daemon with a periodic refresh controlled by
citus.cluster_version_refresh_interval.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/include/distributed/cluster_version.h | Public API for the cluster-version shmem cache and refresh/invalidate hooks |
| src/backend/distributed/utils/maintenanced.c | Adds periodic daemon refresh of the cached minimum cluster version |
| src/backend/distributed/utils/citus_version.c | Adds citus_version_num() C implementation returning CITUS_VERSION_NUM |
| src/backend/distributed/sql/udfs/citus_version_num/latest.sql | Creates pg_catalog.citus_version_num() UDF for latest installs |
| src/backend/distributed/sql/udfs/citus_version_num/15.0-1.sql | Creates pg_catalog.citus_version_num() UDF for 15.0-1 upgrade path |
| src/backend/distributed/sql/udfs/citus_minimum_cluster_version/latest.sql | Creates pg_catalog.citus_minimum_cluster_version() UDF for latest installs |
| src/backend/distributed/sql/udfs/citus_minimum_cluster_version/15.0-1.sql | Creates pg_catalog.citus_minimum_cluster_version() UDF for 15.0-1 upgrade path |
| src/backend/distributed/sql/downgrades/citus--15.0-1--14.0-1.sql | Drops the new UDFs on downgrade |
| src/backend/distributed/sql/citus--14.0-1--15.0-1.sql | Includes the new UDF SQL files during upgrade |
| src/backend/distributed/shared_library_init.c | Registers new GUC + requests shmem space / initializes the shmem hook |
| src/backend/distributed/operations/cluster_version.c | Implements shmem cache, fan-out compute, and citus_minimum_cluster_version() |
| src/backend/distributed/metadata/metadata_cache.c | Invalidates the version cache when pg_dist_node changes |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+46
to
+50
| * GUC controlling how often the maintenance daemon recomputes and caches the | ||
| * cluster minimum version via fan-out, in milliseconds. Because the daemon | ||
| * refreshes the cache proactively, on-demand reads are always served from the | ||
| * cache and never trigger a fan-out themselves. -1 disables the periodic | ||
| * refresh. |
Comment on lines
+718
to
+722
| * so that on-demand reads are always served from a warm cache and the | ||
| * value reflects in-place version changes (which don't touch | ||
| * pg_dist_node). RefreshClusterVersionCache is best-effort: it leaves the | ||
| * cache untouched if a node is unreachable, so it cannot crash the daemon. | ||
| */ |
Comment on lines
+1118
to
+1122
| gettext_noop("The maintenance daemon periodically recomputes, via fan-out, " | ||
| "the value returned by citus_minimum_cluster_version() and stores " | ||
| "it in shared memory, so on-demand reads are always served from a " | ||
| "warm cache. This also lets in-place version changes be picked up. " | ||
| "Use -1 to disable."), |
| @@ -0,0 +1,6 @@ | |||
| CREATE OR REPLACE FUNCTION pg_catalog.citus_minimum_cluster_version() | |||
| RETURNS text | |||
| LANGUAGE C STRICT | |||
| @@ -0,0 +1,6 @@ | |||
| CREATE OR REPLACE FUNCTION pg_catalog.citus_minimum_cluster_version() | |||
| RETURNS text | |||
| LANGUAGE C STRICT | |||
Comment on lines
+89
to
+92
| Datum | ||
| citus_minimum_cluster_version(PG_FUNCTION_ARGS) | ||
| { | ||
| CheckCitusVersion(ERROR); |
Comment on lines
+158
to
+161
| ereport(ERROR, (errmsg("could not get Citus version from node \"%s:%d\"", | ||
| workerNode->workerName, workerNode->workerPort), | ||
| errhint("Ensure the node is reachable and running Citus."))); | ||
| } |
Add citus_version_num() and citus_minimum_cluster_version() backed by a node-local shared-memory cache. The maintenance daemon periodically recomputes the minimum via fan-out and stores it (citus.cluster_version_refresh_interval), so on-demand reads are served from a warm cache. If any node is unreachable the daemon invalidates the cache instead of storing a partial minimum or serving a stale value, so the next read surfaces the error, matching the strict on-demand path. It never errors on a node being down, so the daemon stays alive. No catalog changes.
alperkocatas
force-pushed
the
alperkocatas/method-4-min-cluster-version-daemon-refresh
branch
from
July 22, 2026 12:16
791cd3f to
536b212
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (2)
src/backend/distributed/sql/udfs/citus_minimum_cluster_version/latest.sql:4
- This function is missing a volatility annotation. Leaving it VOLATILE means it may be executed repeatedly within a query (e.g., per-row), which is undesirable given it can fan-out when the cache is invalid.
RETURNS text
LANGUAGE C STRICT
AS 'MODULE_PATHNAME', $$citus_minimum_cluster_version$$;
src/backend/distributed/sql/udfs/citus_minimum_cluster_version/15.0-1.sql:4
- This function is missing a volatility annotation. Leaving it VOLATILE means it may be executed repeatedly within a query (e.g., per-row), which is undesirable given it can fan-out when the cache is invalid.
RETURNS text
LANGUAGE C STRICT
AS 'MODULE_PATHNAME', $$citus_minimum_cluster_version$$;
Comment on lines
+92
to
+98
| CheckCitusVersion(ERROR); | ||
|
|
||
| /* fast path: return the cached value if it is still valid */ | ||
| LWLockAcquire(&ClusterVersionShmem->lock, LW_SHARED); | ||
| bool cacheValid = ClusterVersionShmem->cacheValid; | ||
| int32 cachedVersion = ClusterVersionShmem->cachedMinVersionNum; | ||
| LWLockRelease(&ClusterVersionShmem->lock); |
Comment on lines
+23
to
+25
| -- cluster-wide Citus version tracking UDFs | ||
| #include "udfs/citus_version_num/15.0-1.sql" | ||
| #include "udfs/citus_minimum_cluster_version/15.0-1.sql" |
Comment on lines
+293
to
+296
| * If any node is unreachable we do not store a partial (and therefore possibly | ||
| * too-high) minimum; instead we invalidate the cache so the next read surfaces | ||
| * the "node unreachable" error rather than returning a stale value, keeping this | ||
| * consistent with the strict on-demand path. This never errors on a node being |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add citus_version_num() and citus_minimum_cluster_version() backed by a node-local shared-memory cache. The maintenance daemon periodically recomputes the minimum via fan-out and stores it (citus.cluster_version_refresh_interval), so on-demand reads are always served from a warm cache.
If any node is unreachable, the cache is invalidated. No catalog changes.
DESCRIPTION: PR description that will go into the change log, up to 78 characters