fix(DR-1170): fix ComfyUI startup crash (server not reachable)

ComfyUI is installed by comfy-cli into its own /comfyui/.venv, but start.sh
launches it with /opt/venv's python. The launch venv was missing ComfyUI's
runtime deps (sqlalchemy from the new asset DB, etc.), so ComfyUI crashed at
startup — surfacing as the misleading 'ComfyUI server not reachable' error.
Nothing in this repo changed; the breakage rode in via COMFYUI_VERSION=latest.

- Mirror ComfyUI's full dependency set (core + custom-node requirements) into
  /opt/venv so the launch venv is complete.
- Pin transformers<5 / huggingface-hub<1 (both unbounded upstream).
- Add a build-time smoke test (main.py --quick-test-for-ci --cpu) so a
  startup-breaking dep fails the build instead of a live worker.
- start.sh pre-flight now launches a real kernel + prints sm/torch/cuda, so a
  GPU/kernel mismatch fails loudly at boot instead of as 'not reachable'.

Base-image stays cu126; NGC base migration is a separate branch/PR.

Author: TimPietrusky

Dockerfile

@@ -62,6 +62,31 @@ RUN if [ "$ENABLE_PYTORCH_UPGRADE" = "true" ]; then \
       uv pip install --force-reinstall torch torchvision torchaudio --index-url ${PYTORCH_INDEX_URL}; \
     fi
 
+# comfy-cli installs ComfyUI into its own workspace venv (/comfyui/.venv), but
+# start.sh launches ComfyUI with /opt/venv's python. That mismatch leaves the
+# launch venv missing ComfyUI's runtime deps (e.g. sqlalchemy, pulled in by
+# ComfyUI's asset DB), so ComfyUI crashes at startup and surfaces as the
+# misleading "ComfyUI server (127.0.0.1:8188) not reachable" error. Mirror
+# ComfyUI's full dependency set (core + custom nodes) into /opt/venv so the
+# launch venv is complete. Root-cause fix for DR-1170.
+RUN uv pip install -r /comfyui/requirements.txt \
+    && for r in /comfyui/custom_nodes/*/requirements.txt; do \
+         [ -f "$r" ] && uv pip install -r "$r" || true; \
+       done
+
+# Pin ComfyUI's unbounded ML dependencies to their last known-good majors.
+# ComfyUI declares transformers>=4.50.3 and huggingface-hub with NO upper bound,
+# so a fresh install can pull transformers 5.x / huggingface-hub 1.x whose
+# breaking API changes also crash ComfyUI at startup. Keep them on the last
+# known-good major.
+RUN uv pip install "transformers>=4.50.3,<5" "huggingface-hub<1.0"
+
+# Build-time smoke test: actually start ComfyUI (imports the full node graph) so
+# a startup-breaking dependency is caught HERE, at build time, instead of as a
+# runtime "server not reachable" failure on a live worker. Runs on CPU — no GPU
+# needed to exercise the import graph.
+RUN cd /comfyui && timeout 300 python main.py --quick-test-for-ci --cpu
+
 # Change working directory to ComfyUI
 WORKDIR /comfyui
 

src/start.sh

@@ -34,15 +34,24 @@ import torch
 try:
     torch.cuda.init()
     name = torch.cuda.get_device_name(0)
-    print(f'OK: {name}')
+    cap = torch.cuda.get_device_capability(0)
+    # Launch a real kernel. The driver-only calls above succeed even when this
+    # PyTorch build has no compiled kernels for the GPU architecture (e.g. an
+    # older torch on a newer GPU). Without this, the worker boots, ComfyUI dies
+    # on the first GPU op, and it surfaces as the misleading 'server not
+    # reachable' error instead of a clear cause here.
+    _ = (torch.zeros(8, device='cuda') + 1).sum().item()
+    torch.cuda.synchronize()
+    print(f'OK: {name} (sm_{cap[0]}{cap[1]}), torch {torch.__version__}, cuda {torch.version.cuda}')
 except Exception as e:
     print(f'FAIL: {e}')
     exit(1)
 " 2>&1); then
-    echo "worker-comfyui: GPU is not available. PyTorch CUDA init failed:"
+    echo "worker-comfyui: GPU is not available or incompatible with this PyTorch build:"
     echo "worker-comfyui: $GPU_CHECK"
-    echo "worker-comfyui: This usually means the GPU on this machine is not properly initialized."
-    echo "worker-comfyui: Please contact RunPod support and report this machine."
+    echo "worker-comfyui: A 'no kernel image is available' error means this torch build"
+    echo "worker-comfyui: lacks kernels for this GPU. Otherwise the GPU may not be"
+    echo "worker-comfyui: properly initialized — please contact RunPod support."
     exit 1
 fi
 echo "worker-comfyui: GPU available — $GPU_CHECK"