Sunshine Encode Performance Windows vs. CachyOS

[audiocrush]

Select Topic Area

Question

Body

Hi there

I've been using Sunshine for a while now with my virtualized Windows gaming machine.
However I finally "want to break free" from redmond so to speak and gave CachyOS a try because it looked kinda promising.

Now I built a vm on my proxmox host with the exact same specs, turned the Windows VM off so that the CachyOS VM could use the GPU via PCIe passthrough, everything went smooth and stuff worked right out of the box. I just had to figure out the creation of the virtual display using an edid file but that was easy enough once I knew that I had to do that.

In case anyone asks these are the specs of the VM:
It may use 8 vcores of the Ryzen 7 5700G
It may use 16GB of DDR4 RAM
It has a 150GB virtio disk on a nvme zfs mirror
And the GPU is a RX9060 XT

The game runs well in 4k with vsync @60Hz and frame pacing turned on in moonlight and it looks quite good when i turn the bitrate to 80-100MBit/s below it gets atrocious and horrible to watch

This worked very well in Windows.

When I use CachyOS on the gameserver though, things seem to go south.
I get anything from horrible stutter to slideshow-mode when I turn Moonlight above 25MBit/s.
The game itself reports to be running at an easy 40-80FPS though.

So I really don't understand what is happening.

I noticed one major difference in sunshine though:
On the WebUI on Windows, the one tab was called "AMD AMF" and on CachyOS it is called "VA-API"
Could that be a hint?
There is also a looooot less settings to do...

Greetings

audiocrush

[bjarnando]

Same experience on 30-series Nvidia on Cachy as well. Installed through octopi, not a problem on windows or fedora

[audiocrush]

I finished the setup and ust did a quick testrun.
I don't think the settings are the same on windows/cachyOS/bazzite but after fiddleing around and testing I now get !subjectively! better results with bazzite than I even got on windows
HOWEVER I had to turn vsync off in the game but enable it on the moonlight client, but disable frame pacing on the moonlight client.
When I do that I get pretty solid 60FPS at 60Hz vsync 4k in Enshouded at settings that I deem to be visually pleasing but far from the best. Bitrate I tested last was 50MBit/s. Tomorrow is gamenight with my friends so I'll leave it at that for now so we can have a good time and not waste it to try testing.
I was the last one of the group not playing on linux so I'm quite happy with that now.
We also migrated away from discord to teamspeak 6

It is a bit of a hit and miss experience so far... but I feel like we're getting there fast and the pace is picking up by the day.

Bill should get some AI friends or something, because it will get lonely up there soon :D

[audiocrush]

nevermind, I must have been lucky...
today I got stutters and hangs, sometimes up to 5 seconds where the game ran just fine at avg 60 FPS but the video stopped completely delivering a message I should reduce my bitrate... this is insane
I really am at a loss right now... and it does not make any sense at all
there is no load spikes on the proxmox host, no congestion on the open vswitch, no congestion on the HP gbit switch
I also tried a separate nic for the VM alone not using the 4x1gbit intel nic bridge that does LACP TCP-LB
I bet cachyOS and bazzite do something ridiculously stupid internally that causes a scheduler to misbehave at irregular intervals or certain events, that causes critical realtime applications like sunshine to hang under certain circumstances

but apparently this seems to be so rare that there is almost no one talking about it

[ReenigneArcher]

Have you seen this? LizardByte/Sunshine#4126

TLDR: adjust MTU size

[audiocrush]

No MTU size is the same across this subnet on all interfaces

doesn't matter anyways I found the culprit, this was such a damn pain to find I swear
when I found it I remembered I once had a similar problem with a remote desktop server farm on a couple of VMWare clusters and the customer had issues with the sessions lagging and hanging and at one point we found out the ring buffers on the NICs were overflowing and thus packets were discarded

So now, all that damn time, it was overflowing tx buffers in proxmox...

By default, for some insanely dumb reasons they were set on all my NICs to 256kb
I set them to 4096kb and now I can crank the bitrate all the way up as far as it goes and I get buttery smooth 4k 60Hz Enshrouded goodness on my bazzite VM :D

Ok so here is the deal for anyone else experiencing performance troubles with sunshine playing in a VM:

If you run your bazzite gaming VM on proxmox make sure you check the TX and RX buffer size with ethtool on the proxmox cli

!!! **this is crucial and not a typo** !!! the small g means read the config, the large G means write the config

ethtool -g
like for instance
ethtool -g enp8s0f0
you might get something like this:

Ring parameters for enp8s0f0:
Pre-set maximums:
RX:			4096
RX Mini:		n/a
RX Jumbo:		n/a
TX:			4096
TX push buff len:	n/a
Current hardware settings:
RX:			256
RX Mini:		n/a
RX Jumbo:		n/a
TX:			256
RX Buf Len:		n/a
CQE Size:		n/a
TX Push:		off
RX Push:		off
TX push buff len:	n/a
TCP data split:		n/a

if you do you can set it to a larger size
ethtool -G enp8s0f0 tx 4096 rx 4096
to try it out
ifdown enp8s0f0 && ifup enp8s0f0

if you are happy with the result you can add this to /etc/network/interfaces to make it permanent
you have to add it in the section of the specific interface you want to edit like so

before

auto enp14s0
iface enp14s0 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr0

after

auto enp14s0
iface enp14s0 inet manual
        pre-up ethtool -G enp14s0 tx 4096 rx 4096
        ovs_type OVSPort
        ovs_bridge vmbr0

IF your interface is part of a bridge that has multiple NICs on it
I'd highly recommend you do it to all of the NICs because you will never know which NIC your traffic will pass if you for instance loadbalance

you might need to install ifupdown2 using apt
apt install ifupdown2
otherwise the pre-up command might not work in /etc/network/interfaces

Hope this helps

cheers and good night 😆