Skip to content

README.md

Latest commit

 

History

History
123 lines (96 loc) · 6.61 KB

README.md

File metadata and controls

123 lines (96 loc) · 6.61 KB

Steps to enable and launch an ubuntu-based SEV-SNP guest (applied in our NixOs servers - TUM Cluster) :

BIOS preparation

  • You have to enable the SME and SNP options in your BIOS settings. To do so in ryan and graham, access their mgmt interface via ssh and run the following:
set BIOS.ProcSettings.Sme Enabled
set BIOS.ProcSettings.Snp Enabled
jobqueue create BIOS.Setup.1-1

and then reboot the server.

  • For SEV ES you have to enable IOMMU support and set the minimum SEV ASIDs value in the BIOS. To do so in ryan and graham, run the following after accessing their mgmt interface:
set BIOS.ProcSettings.IommuSupport Enabled
set BIOS.ProcSettings.CpuMinSevAsid 128
jobqueue create BIOS.Setup.1-1

and then reboot the server. For more information regarding the parameter for CPU mininmum SEV ASIDs specifically in our machines, look here.

1. Use the dedicated host kernel and enable SEV-SNP in the desired server:

Import the amd_sev_snp.nix. An example configuration is shown here.

This module sets the appropriate kernel version and parameters, and adds the mandatory kernel modules for SME, SEV-SNP, and, optionally, svsm .

Note: this setup has been tested with (currently only SEV-ES)

  • kernel 6.6.0-rc1 sev-snp version provided by AMD (link) -- the latest tested commit is here

2. Verify that SME, SEV and SEV-ES are enabled:

  • dmesg | grep sev should include sev enabled in its output.
  • dmesg | grep -i SEV-ES should indicate that SEV-ES is supported and the number of SEV ASIDs.
  • dmesg | grep -i SEV-SNP should indicate that SEV-SNP is enabled and the number of ASIDs.

3. Prepare the host toolchain (OVMF)

Compile the custom OVMF provided by AMD:

$ cd AMDSEV
$ bash build.sh ovmf

Note:

For SNP, this setup has been tested with

4. Prepare an AMD SEV-SNP guest.

  • You need to have a network-config file (.yaml) for your VM, similar to those in the network_configs folder.
  • The prepare_net_cfg.sh script takes as a parameter the virtual bridge where the VMs will be connected to and modifies the IP prefix in the network configuration (given as a secord parameter) appropriately.

Follow the next set of commands from the AMD_SEV_SNP directory to launch an SEV-SNP guest (tested with ubuntu 22.04 cloud img).

$ wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

$ mkdir images

$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH qemu-img convert noble-server-cloudimg-amd64.img ./images/gdpr.img

$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH qemu-img resize ./images/gdpr.img +20G

$ bash prepare_net_cfg.sh -br virbr0 -cfg ./network_configs/netplan-gdpr.yaml

$ mkdir -p firmware/gdpr

$ cp ./AMDSEV/usr/local/share/qemu/OVMF.fd ./firmware/gdpr/OVMF.fd

For convenience, we wrap these operations in a single script (GDPRuler_VMs_setup.sh) to setup a gdpr and passthrough controller image.

Important note:

  • Each VM requires a separate .img and OVMF.fd files.
  • To avoid any problems, you have to use a distro with text-based installer, otherwise your launched VM might stuck (issue).

5. Launch an AMD SEV-SNP guest.

$ sudo LD_LIBRARY_PATH=$LD_LIBRARY_PATH bash AMDSEV/launch-qemu.sh \
-hda images/gdpr.img \
-sev-snp \
-bridge virbr0 \
-bios firmware/gdpr

IMPORTANT: As of 25/1/2024 the synchronized versions of linux kernel, qemu and ovmf do not allow to run SNP VMs on a host kernel lower than version 6.7. Therefore, if you have an older kernel (<=6.6), change the -sev-snp parameter above to -sev-es till you update the host kernel to a version >=6.7.

Important notes:

  • Follow the same process for the creation of a client vm (if you want/need to). You need a different .img, and to adapt the network configuration appropriately to reserve a different IP. Network configuration examples are given in the network_configs folder.

6. Inside the guest VM, verify that AMD SEV-SNP is enabled:

sudo dmesg | grep snp -i should indicate Memory Encryption Features active: AMD SEV SEV-ES SEV-SNP

7. Networking:

In step 5 above, we use the parameter -bridge virbr0, so that our VMs use the virtual network bridge virbr0. Typically, this is set up if you are using libvirt. If it does not exist, you can create and configure it by using the bridge-utils package. An example is shown below:

sudo brctl addbr virbr0
sudo brctl stp virbr0 on
sudo ifconfig virbr0 up
sudo ifconfig virbr0 192.168.122.1 netmask 255.255.255.0

Our script prepare_net_cfg.sh checks the given virtual bridge and adjusts the prefix of the IP declared in the network configuration file. Example configuration files are given in the network_configs folder. They are used mainly to pre-determine the IPs of the VMs in the network.

Manual ssh connection setup

  • After you make sure that networking works fine and you can reach the VM guest from the host, you can log-in the VM using ssh (after placing your ssh keys in the ~/.ssh/autorhized_keys file of the guest VM).

Useful links