Dedicated Server: "Game paused" after joining but actually running

I recently setup a dedicated server on my local machine where everything is working just fine and I can connect normally.

When moving the instance files to my server (Windows Server 22 VM on proxmox hypervisor) I can connect fine but as soon as the world loads (i.e. I can see my base from my cryo pod) the client displays "Game paused" at the top of the screen while the server reports that it is running with a simulation speed of 1 and unchanged cpu usage (around 10-20 %). I can see my player as online in the remote client and all pages from grids, characters and so forth are working. Some grids are even moving. The console is also working, I can see myself joining and leaving the game.

The same happens on the server with a fresh game with standard configuration and without mods, The server is installed with steamcmd (I also tried with the steam client but the same thing happened). It is a fresh windows install with nothing but space engineers. The port is opened in the windows firewall.

The only thing I noticed was different is the message "2025-01-02 13:45:14.171: Loaded 506 Steam Inventory item definitions" after joining. It is missing on the non working instance.

Attached is my client and server log (server with verbose network logging activated), the time should be pretty much the same as both use the same NTP server. (In this particular client log it took two tries to connect to the server. This did not happen before)

Please let my know if you need anything else. It should also be possible for you to remote into the server if needed