Kubernetes Packaging Comparison Demo

This repository demonstrates three different approaches to deploying the same HTTP Echo application on Kubernetes:

1. Helm - Template-based package manager
2. Kustomize - Configuration management with overlays
3. Kro - Resource composition with custom resources

Architecture

  flowchart LR
    subgraph Dev["developers"]
      D(devs)
    end
    subgraph Git["GitHub repo"]
      H[Helm folder] -->|render| HR
      K[Kustomize folder] -->|build| KR
      R[Kro folder] -->|validate| RR
    end
    subgraph Cluster["Kubernetes"]
      DP[Deployment
      http-echo]
      SV[Service
      http-echo]
    end
    D --> Git
    HR --> Cluster
    KR --> Cluster
    RR --> Cluster

Loading

The HTTP Echo is a simple web service that demonstrates common Kubernetes deployment patterns including:

• Deployment with configurable replicas
• Service for internal communication
• Ingress for external access
• Environment-specific configurations

Prerequisites

• Docker
• kubectl
• kind
• Helm
• jq (for Kro operator installation)

Quick Start

1. Setup Cluster

# Create KinD cluster with ingress support
make setup

This will:

• Create a 3-node KinD cluster
• Install nginx-ingress controller
• Install Kro operator using Helm
• Configure port forwarding for local access

2. Deploy Using Different Methods

Helm Deployment

make helm

Deploys the application using Helm charts with:

• Templated Kubernetes manifests
• Configurable values via values.yaml
• Release management capabilities

Kustomize Deployment

make kustomize

Deploys using Kustomize with:

• Base configuration in kustomize/base/
• Dev overlay with environment-specific patches
• Declarative configuration management

Kro Deployment

make kro

Deploys using Kro ResourceGraphDefinition:

• Custom resource for application definition
• Parameterized resource composition
• Graph-based resource relationships

3. Check Deployment Status

make status

4. Access the Application

Add the following to your /etc/hosts file:

127.0.0.1 http-echo-helm.local
127.0.0.1 http-echo-kustomize.local
127.0.0.1 http-echo-kro.local

Then access:

• Helm deployment: http://http-echo-helm.local
• Kustomize deployment: http://http-echo-kustomize.local
• Kro deployment: http://http-echo-kro.local

5. Cleanup

# Remove all deployments
make teardown

# Delete the cluster
make clean

Repository Structure

k8s-packaging-comparison-demo/
├── helm/http-echo/              # Helm chart
│   ├── Chart.yaml             # Chart metadata
│   ├── values.yaml            # Default values
│   └── templates/             # Kubernetes templates
│       ├── deployment.yaml
│       ├── service.yaml
│       ├── ingress.yaml
│       ├── serviceaccount.yaml
│       └── _helpers.tpl
├── kustomize/                 # Kustomize configuration
│   ├── base/                  # Base resources
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   ├── ingress.yaml
│   │   └── kustomization.yaml
│   └── overlays/              # Environment overlays
│       ├── kustomization.yaml
│       ├── replica-count.yaml
│       └── env-config.yaml
├── kro/                       # Kro resources
│   ├── rgd/
│   │   └── http-echo-rgd.yaml   # ResourceGraphDefinition
│   └── examples/
│       └── http-echo.yaml      # Example custom resource
├── cluster/                   # Cluster setup
│   ├── kind-cluster.yaml      # KinD configuration
│   └── install-ingress.sh     # Ingress setup script
├── .github/workflows/         # CI/CD pipelines
│   ├── helm-lint.yml
│   ├── kustomize-build.yml
│   └── kro-validate.yml
└── Makefile                   # Automation targets

Deployment Method Comparison

Helm

Pros:

• Rich templating with Go templates
• Package management and versioning
• Large ecosystem of charts
• Release management (rollback, history)
• Hooks for lifecycle management

Cons:

• Template complexity can be hard to debug
• Learning curve for template syntax
• Tiller dependency (Helm 2, resolved in Helm 3)

Best for: Teams that need package management, complex templating, and release management.

Kustomize

Pros:

• Pure YAML, no templating language
• Built into kubectl
• Overlay system for environment management
• Git-friendly (plain YAML files)
• Simple patch mechanism

Cons:

• Limited programming constructs
• No release management
• Can become complex with many overlays

Best for: Teams that prefer declarative configuration and have GitOps workflows.

Kro

Pros:

• Custom resource abstraction
• Graph-based resource composition
• Parameterized deployments
• Cloud-native resource management
• Intuitive resource relationships

Cons:

• Newer tool with smaller ecosystem
• Requires Kro operator installation
• Learning curve for ResourceGraph concepts

Best for: Teams that want to create reusable, parameterized application definitions with clear resource relationships.

Configuration Examples

Environment Variables

Each method handles environment configuration differently:

Helm: Uses values.yaml and templates

env:
  - name: NODE_ENV
    value: {{ .Values.environment }}

Kustomize: Uses strategic merge patches

# env-config.yaml
spec:
  template:
    spec:
      containers:
      - name: http-echo
        env:
        - name: NODE_ENV
          value: "development"

Kro: Uses schema parameters

schema:
  properties:
    environment:
      type: string
      default: "production"

Resource Scaling

Helm: Template-based scaling

spec:
  replicas: {{ .Values.replicaCount }}

Kustomize: Patch-based scaling

# replica-count.yaml
spec:
  replicas: 3

Kro: Parameter-based scaling

schema:
  properties:
    replicaCount:
      type: integer
      default: 2

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: make test (if implemented)
5. Submit a pull request

Learn More

• Helm Documentation
• Kustomize Documentation
• Kro Documentation
• Kubernetes Documentation

Troubleshooting

Common Issues

1. Port conflicts: Ensure ports 80 and 443 are available
2. DNS resolution: Add entries to /etc/hosts as shown above
3. Resource limits: Ensure Docker has sufficient resources allocated

Getting Help

• Check the Issues page
• Review deployment logs: kubectl logs -n <namespace> <pod-name>
• Check ingress status: kubectl get ingress -A