Replace Env Generator Tool and EnvConfigGateway with Zod Schema-Based Env Validation

[maany]

Description

The project includes a standalone CLI tool (tools/env-generator/) that uses liquidjs to template .env files from RUCIO_WEBUI_* environment variables, and an EnvConfigGateway class that wraps raw process.env reads behind a port interface. Together, these create several problems:

1. Unnecessary dependency tree: liquidjs (+ commander) and yargs are production dependencies of the tool, each with their own transitive dependency graph. liquidjs has required repeated security patch bumps (e.g., f8c7ff49), adding maintenance burden for a tool that renders a single template.

2. Redundant validation: tools/env-generator/src/api/base.ts manually validates required/optional env vars, checks conditional requirements (e.g., SERVER_CA_BUNDLE required when NODE_TLS_REJECT_UNAUTHORIZED=1), and validates OIDC provider / VO sub-configurations. This is exactly what Zod does, but hand-rolled and less composable.

3. Templating is overkill: The .env.liquid template uses Liquid loops and conditionals to generate dynamic OIDC/VO sections. This could be a simple TypeScript function that iterates config and writes key-value lines, no templating engine needed.

4. Separate build/install step: The tool has its own package.json, tsconfig.json, and build process (tsc && cp -rf src/templates dist/). Users must cd tools/env-generator && npm install && npm run build before they can generate an env file.

5. Drift risk: The env vars read by the app (via EnvConfigGateway) and the vars validated/generated by this tool are defined in two separate places. Nothing enforces they stay in sync.

6. EnvConfigGateway is a pass-through: The gateway reads process.env in each method, applies ad-hoc defaults and string parsing, and returns values. Once a Zod schema validates and types the config at startup, this class becomes a redundant wrapper. Every method (e.g., oidcEnabled(), rucioHost()) is just returning a property from what would already be a typed object. The port/gateway indirection adds boilerplate without abstracting over multiple config sources or providing any transformation logic that the schema doesn't already handle.

Proposed Solution

Replace both tools/env-generator/ and EnvConfigGateway with a Zod schema that produces a single typed config object, imported directly wherever config is needed. No new dependencies: Zod is already used in the project for API route validation.

Summary of experiments with Zod and how they map to the environment management lifecycle

1. Server/Client Schema Separation

Split env vars into a client schema (for NEXT_PUBLIC_* vars inlined at build time) and a server schema that extends it:

// src/lib/infrastructure/config/env.client.ts
import { z } from 'zod';

export const clientEnvSchema = z.object({
  NEXT_PUBLIC_WEBUI_HOST: z.string().url(),
});

export type ClientEnv = z.infer<typeof clientEnvSchema>;

// src/lib/infrastructure/config/env.server.ts
import { z } from 'zod';
import { clientEnvSchema } from './env.client';

const oidcProviderSchema = z.object({
  name: z.string().min(1),
  client_id: z.string().min(1),
  client_secret: z.string().min(1),
  authorization_url: z.string().url(),
  token_url: z.string().url(),
  redirect_url: z.string().url(),
  issuer: z.string().url(),
  icon_url: z.string().url().optional(),
  refresh_token_url: z.string().url().optional(),
  userinfo_url: z.string().url().optional(),
  scopes: z.string().optional(),
  logout_url: z.string().url().optional(),
});

const voSchema = z.object({
  short_name: z.string().min(1),
  name: z.string().min(1),
  logo: z.string().optional(),
  oidc_enabled: z.coerce.boolean().default(false),
  oidc_providers: z.string().optional(),
});

export const serverEnvSchema = clientEnvSchema.merge(
  z.object({
    // Gateway
    RUCIO_HOST: z.string().url(),
    RUCIO_AUTH_HOST: z.string().url(),
    PARAMS_ENCODING_ENABLED: z.coerce.boolean().default(false),

    // Auth
    AUTH_SECRET: z.string().min(1),
    NEXTAUTH_SECRET: z.string().min(1),
    NEXTAUTH_URL: z.string().url(),

    // Login methods
    ENABLE_USERPASS_LOGIN: z.coerce.boolean().default(true),
    X509_ENABLED: z.coerce.boolean().default(true),

    // OIDC
    OIDC_ENABLED: z.coerce.boolean().default(false),
    OIDC_PROVIDERS: z.string().optional(),
    OIDC_EXPECTED_AUDIENCE_CLAIM: z.string().default('rucio'),

    // Multi-VO
    MULTIVO_ENABLED: z.coerce.boolean().default(false),
    VO_DEFAULT: z.string().default('def'),
    VO_LIST: z.string().default('def'),

    // TLS
    NODE_TLS_REJECT_UNAUTHORIZED: z.enum(['0', '1']).default('1'),
    SERVER_CA_BUNDLE: z.string().optional(),

    // Misc
    RULE_ACTIVITY: z.string().default('User Subscriptions'),
    LIST_DIDS_INITIAL_PATTERN: z.string().optional(),
    PROJECT_URL: z.string().url().optional(),
  })
).superRefine((data, ctx) => {
  if (data.NODE_TLS_REJECT_UNAUTHORIZED === '1' && !data.SERVER_CA_BUNDLE) {
    ctx.addIssue({
      code: z.ZodIssueCode.custom,
      message: 'SERVER_CA_BUNDLE is required when NODE_TLS_REJECT_UNAUTHORIZED=1',
      path: ['SERVER_CA_BUNDLE'],
    });
  }

  if (data.OIDC_ENABLED && !data.OIDC_PROVIDERS) {
    ctx.addIssue({
      code: z.ZodIssueCode.custom,
      message: 'OIDC_PROVIDERS is required when OIDC_ENABLED=true',
      path: ['OIDC_PROVIDERS'],
    });
  }
});

export type ServerEnv = z.infer<typeof serverEnvSchema>;

2. Boolean Coercion from Env Strings

Environment variables are always strings. Pre-process before parsing:

function preprocessBooleanEnv(key: string): boolean | undefined {
  const val = process.env[key]?.trim().toLowerCase();
  if (val === undefined) return undefined;
  return val === 'true';
}

const runtimeEnv = {
  OIDC_ENABLED: preprocessBooleanEnv('OIDC_ENABLED'),
  X509_ENABLED: preprocessBooleanEnv('X509_ENABLED'),
  ENABLE_USERPASS_LOGIN: preprocessBooleanEnv('ENABLE_USERPASS_LOGIN'),
  RUCIO_HOST: process.env.RUCIO_HOST,
  RUCIO_AUTH_HOST: process.env.RUCIO_AUTH_HOST,
  // ...
};

3. Build-Time Placeholder Support

For containerized deployments where env vars are injected at runtime (e.g., Docker/Helm), support a build-time placeholder mode so next build doesn't fail on missing vars:

const isBuildTime = process.env.NEXT_PHASE === 'phase-production-build';

const result = serverEnvSchema.safeParse(runtimeEnv);

if (!result.success) {
  if (isBuildTime) {
    console.warn('[env] Validation warnings during build (will be checked at runtime):');
    console.warn(result.error.format());
  } else {
    console.error('[env] Invalid environment configuration:');
    console.error(result.error.format());
    throw new Error('Invalid environment configuration. See errors above.');
  }
}

export const env = result.data!;

This gives fail-fast at runtime (server won't start with bad config) while allowing builds to complete in CI/CD pipelines where runtime secrets aren't available.

4. Dynamic OIDC/VO Sub-Configuration Parsing

The current tool loops over comma-separated provider names and reads RUCIO_WEBUI_OIDC_PROVIDER_{NAME}_{FIELD} vars. Replicate this with a parser that expands dynamic env vars into structured objects, then validate with Zod:

function parseOIDCProviders(providerList: string): unknown[] {
  return providerList.split(',').map(name => {
    const prefix = `OIDC_PROVIDER_${name.trim().toUpperCase()}`;
    return {
      name: name.trim(),
      client_id: process.env[`${prefix}_CLIENT_ID`],
      client_secret: process.env[`${prefix}_CLIENT_SECRET`],
      authorization_url: process.env[`${prefix}_AUTHORIZATION_URL`],
      token_url: process.env[`${prefix}_TOKEN_URL`],
      redirect_url: process.env[`${prefix}_REDIRECT_URL`],
      issuer: process.env[`${prefix}_ISSUER`],
      // ... optional fields
    };
  });
}

const providers = parseOIDCProviders(env.OIDC_PROVIDERS);
const parsed = z.array(oidcProviderSchema).parse(providers);

Same pattern for VO sub-configurations.

5. Direct Import Replaces EnvConfigGateway

Instead of injecting EnvConfigGateway through the IoC container and calling methods like gateway.oidcEnabled(), consumers import the typed config object directly:

// Before (with gateway)
const gateway = appContainer.get<EnvConfigGatewayOutputPort>(GATEWAYS.ENV_CONFIG);
const isOidc = gateway.oidcEnabled();
const host = gateway.rucioHost();

// After (direct import)
import { env } from '@/lib/infrastructure/config/env.server';
const isOidc = env.OIDC_ENABLED;
const host = env.RUCIO_HOST;

The EnvConfigGatewayOutputPort interface, EnvConfigGateway implementation, and its IoC binding can all be removed. The Zod-parsed env object is already validated, typed, and immutable. There is nothing left for a gateway class to do.

6. Generate .env.example from Schema (Optional)

A lightweight script can introspect the Zod schema to produce a documented template, no templating engine needed:

// scripts/generate-env-example.ts
// Reads schema keys + descriptions to produce .env.example
// Run with: npx tsx scripts/generate-env-example.ts

Migration Path

Current	Proposed
WebUIEnvTemplateCompiler.validateEnv()	serverEnvSchema.safeParse()
validateOIDCProvider() / validateVO()	oidcProviderSchema / voSchema with .superRefine()
.env.liquid template rendering	Direct process.env reading + Zod parse
RUCIO_WEBUI_* prefix stripping	Unnecessary, app reads env vars directly
generateRandomString() for secrets	Zod .default() with crypto.randomBytes()
Manual error/warning distinction	safeParse() + isBuildTime conditional
Separate tool build step	Schema lives in main project, no extra build
EnvConfigGateway + port interface	Direct import { env } from schema module
IoC binding for EnvConfigGatewayOutputPort	Removed
gateway.oidcEnabled() / gateway.rucioHost()	env.OIDC_ENABLED / env.RUCIO_HOST

Checklist for this task

• Server and client Zod env schemas defined in src/lib/infrastructure/config/
• Server schema extends client schema via .merge()
• App fails fast at startup with clear Zod error messages on invalid config
• Build-time placeholder mode allows next build without runtime secrets
• Conditional validations work via .superRefine() (e.g., CA bundle when TLS on, providers when OIDC on)
• Dynamic OIDC provider and VO sub-configs parsed from env and validated against sub-schemas
• All consumers of EnvConfigGateway migrated to direct env import
• EnvConfigGatewayOutputPort, EnvConfigGateway, and their IoC bindings removed
• tools/env-generator/ directory removed
• liquidjs removed from project dependencies
• .env.development.local.template updated to match schema
• No new dependencies introduced (Zod already in project)

Why we should do this

• Eliminates liquidjs dependency: no more security patch churn for a templating engine used to write key-value pairs
• Eliminates EnvConfigGateway boilerplate: no more port interface, implementation class, IoC binding, and per-variable accessor methods wrapping what is already a typed object
• Single source of truth: env var definitions, validation, types, and defaults all in one schema
• Type safety: consumers get a fully typed config object, no stringly-typed process.env access, no method-per-variable indirection
• Fail-fast: bad config caught at startup, not at runtime when a feature is first used
• Container-friendly: build-time placeholder mode supports Docker/Helm workflows where secrets are injected at runtime
• No separate tool build: schema lives in the main project
• Composable with Feature Flags: the feature flags subsystem (see related issue) can define its flags in the same schema

Additional Information

• The Rucio API needs to be changed.