Skip to content

Add setup command and fix DNS resolution with NVIDIA GPU passthrough - #264

Open
ViniAman01 wants to merge 2 commits into
Jip-Hop:mainfrom
ViniAman01:main
Open

Add setup command and fix DNS resolution with NVIDIA GPU passthrough#264
ViniAman01 wants to merge 2 commits into
Jip-Hop:mainfrom
ViniAman01:main

Conversation

@ViniAman01

Copy link
Copy Markdown

Proposed fix for DNS resolution failure when gpu_passthrough_nvidia=1 is enabled.

Note: This is untested code, meant as a starting point for discussion. I encountered these issues in my setup and thought this approach might help others.

The issue was caused by mounting broad system library directories (like /usr/lib/x86_64-linux-gnu) which shadowed the jail's libc/NSS libraries, breaking name resolution.

Changes:

  • Filter system library dirs from NVIDIA bind mounts to preserve jail DNS
  • Add setup command to re-run initial_setup in existing jails
  • Make docker template idempotent for safe re-execution
  • Warn when NVIDIA Container Toolkit is missing in jail

vsa added 2 commits February 17, 2026 17:49
- Add 'setup' command to re-run initial_setup in existing jails
- Filter broad system library dirs from NVIDIA mounts to prevent
  shadowing jail libc/NSS which breaks DNS resolution
- Warn when NVIDIA Container Toolkit is missing in jail
@kuya1284

kuya1284 commented Jul 12, 2026

Copy link
Copy Markdown

If you were running into issues while using TrueNAS 25.10 or newer, the issue was due to a change made by iXsystems. More specifically, they replaced the NVIDIA Proprietary drivers with the NVIDIA Open Kernel drivers. With the proprietary drivers, the library files used to be located in /usr/lib/x86_64-linux-gnu/video. However, with the open drivers, they're now located directly in /usr/lib/x86_64-linux-gnu. Because of this significant change, when Jailmaker attempts to build the list of NVIDIA bind mounts (by parsing the list returned by the nvidia-container-cli list command), as well as the dynamic linking library config file, the /usr/lib/x86_64-linux-gnu directory ends up getting placed in the config file and when ldconfig gets executed in the jail, it effectively clobbers the libraries inside the jail, especially /usr/lib/x86_64-linux-gnu/systemd/.

So I don't think the issue is necessarily a DNS resolution issue per se. It's just a side effect because /usr/lib/x86_64-linux-gnu is getting bind mounted when it shouldn't be.

I've created a new fork and just created a new v2.2.0 release, if you're willing to give it a try. It should hopefully prevent the issue you ran into without having to re-run the initial setup in existing jails.

Also, the NVIDIA Container Toolkit shouldn't be a requirement for a jail. IMHO, that should be handled via the config template, because it isn't always going to be needed. It'll probably be needed for apps like Beszel, but not other apps that won't be monitoring the GPU. So I don't think it's necessary to warn when that particular package doesn't exist in the jail.

Here's a sample of what I have in my config template:

  # Add NVIDIA Container Toolkit repository
  curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
    && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
      sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
      tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

  apt update

  # ................................. other stuff, like installing Docker and creating users...

  # Install NVIDIA Container Toolkit packages
  # @link https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
  echo '-----------------------------'
  echo 'Installing NVIDIA Container Toolkit'
  echo '-----------------------------'
  export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.19.1-1
  apt-get install -y \
    nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}

  nvidia-ctk runtime configure --runtime=docker
  systemctl restart docker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants