# local-k3d A local Kubernetes development environment using k3d (lightweight Kubernetes in Docker). ## Overview This project provides a complete local Kubernetes development setup with: - k3d cluster with 1 server and 2 agent nodes - Local container registry on port 5949 - Tilt for rapid development iteration - Shared volumes for persistent data - Common Kubernetes tooling and utilities ## Prerequisites Install required tools using Homebrew: ```bash make setup ``` This will install: - Core tools: k3d, kubectl, helm, k9s, tilt, kustomize, arkade - Context management: kubectx, kubie - Utilities: jq, yq, gomplate, gum, stern - Additional tools: flux, podman, kind, httpie ## Quick Start ### 1. Create the cluster ```bash ./k3d-create-cluster.sh ``` This creates a cluster named `lk3d-cluster` with: - 1 server node - 2 agent nodes - Local registry at `lk3d-cluster-registry:5949` - Mounted volumes: `./tmp`, `./local`, `./work`, `./shared` ### 2. Switch to the cluster context ```bash ./k3d-use-context.sh ``` ### 3. Start Tilt (optional) ```bash tilt up ``` Tilt will automatically deploy manifests from the `manifests/` directory and provide a UI for managing your development workflow. ## Cluster Management Scripts - `./k3d-create-cluster.sh` - Create a new k3d cluster - `./k3d-destroy-cluster.sh` - Delete the cluster - `./k3d-start-cluster.sh` - Start a stopped cluster - `./k3d-stop-cluster.sh` - Stop the cluster (without deleting) - `./k3d-use-context.sh` - Switch kubectl context to the k3d cluster - `./build-push-deploy.sh` - Build, push, and deploy workflow - `./enter-shell.sh` - Enter the cluster shell environment ## Directory Structure ``` . � manifests/ # Kubernetes manifests (format: ..yaml) � k8s/ # Additional Kubernetes configs � helm-values/ # Helm chart value overrides � local/ # Mounted to /hl and /h in cluster nodes � tmp/ # Mounted to /ht in cluster nodes � work/ # Mounted to /wrk in cluster nodes � shared/ # Shared data across cluster � Tiltfile # Tilt configuration for dev workflow � Makefile # Build automation � Brewfile # Homebrew dependencies ``` ## Kubernetes Manifest Naming Convention Store manifests in the `manifests/` directory using the format: ``` ..yaml ``` Examples: - `shell.deployment.yaml` - `example-init.deployment.yaml` - `api.service.yaml` ## Accessing the Cluster Once the cluster is running: ```bash # Verify cluster is running k3d cluster list # Check nodes kubectl get nodes # Use k9s for a TUI k9s # Use Lens for a GUI open -a FreeLens ``` ## Local Registry The cluster includes a local registry for pushing images: ```bash # Tag an image for the local registry docker tag myimage:latest localhost:5949/myimage:latest # Push to local registry docker push localhost:5949/myimage:latest # Reference in Kubernetes manifests image: lk3d-cluster-registry:5949/myimage:latest ``` ### Build, Push, and Deploy Workflow Use the automated script for building and deploying: ```bash ./build-push-deploy.sh ``` This script handles the complete workflow of building images, pushing to the local registry, and deploying to the cluster. ## Makefile Targets - `make setup` - Install dependencies and create required directories - `make clean` - Clean the tmp directory - `make all` - Run clean target (default) ## Tilt Development Workflow The Tiltfile is configured to: - Watch and deploy manifests from `manifests/` - Provide a local resource for Homebrew installations - Support Tilt extensions (git_resource, helm_remote) - Optional Argo Workflows installation (manual trigger) Access the Tilt UI after running `tilt up` at: http://localhost:10350 ## Argo Workflows Argo Workflows is available as an optional component that can be installed via Tilt. ### Installation 1. Start Tilt: ```bash tilt up ``` 2. In the Tilt UI (http://localhost:10350), locate the `argo-workflows-server` resource 3. Click the trigger button to install Argo Workflows This will: - Create the `argo` namespace - Install Argo Workflows via Helm - Configure server auth mode for easier local development - Port forward the UI to `localhost:2746` ### Accessing Argo UI Once installed, access the Argo Workflows UI at: http://localhost:2746 ### Using Argo CLI ```bash # Set the Argo server address export ARGO_SERVER='localhost:2746' export ARGO_HTTP1=true export ARGO_SECURE=false # Submit a workflow argo submit -n argo examples/hello-world.yaml # List workflows argo list -n argo # Get workflow logs argo logs -n argo ``` ## Cleanup To completely remove the cluster and free up resources: ```bash ./k3d-destroy-cluster.sh ``` ## Troubleshooting ### Context not found If `kubectl` can't find the context, ensure the cluster is created: ```bash k3d cluster list ./k3d-create-cluster.sh ``` ### Port conflicts If port 5949 is already in use, modify the registry port in `k3d-create-cluster.sh` ### Cluster won't start Check Docker is running and has sufficient resources allocated ### Shell Access Use the enter-shell script to access the cluster environment: ```bash ./enter-shell.sh ```