MP - Local UPS/SimSpeed Drops

So I noticed with this new update local UPS and Sim Speeds, and I repeat, ONLY local ones drop out of nowhere, causing massive rubberbanding locally.

It seems like it stays stable for like 3-4 seconds, then UPS drops from 60 to ~46 and sim speed dips into ~0.83 range. On the server side everything's completely fine, sim speed 1.0 and no CPU/Storage bottlenecks. (Juicy CPU and SSD storage, CPU usage never above 20%) (Dedicated server rig)

Before this update you it ran butter smooth as per dedicated hardware should, but now this happens. I am 100% certain this is a local issue, considering the F11 statistics and server side statistics.

I myself am running a Ryzen 5 5600X with a 3060 Ti and a NVMe 7GB R/W drive, so pretty sure that ain't the issue either, considering the juice it has. (This thing can run at 200-300 FPS uncapped in grid populated spaces, so pretty sure that ain't it chief)

Changing graphics settings to anything else (lowest to high) changes nothing.

Like I said, this is only with this update, I really hope keen can look into this, multiplayer is kind of a lifeblood for this thing as well.

P.S: Noticed that disabling 'Adaptive Sim Speed Quality' in the server settings seems to have solved it for now, but now there is lots of increased CPU usage than before... not really an ideal solution. (Now at max 35%)

EDIT2: Nope, the 'Adaptive Sim Speed Quality' does nothing. Fully testing it affects nothing. It seems to be related to traveling to new areas. Even if they are done loading/generating, staying in them even afk constantly has this issue. Tho, after a server restart, the issue dissapears when still in the same location, but upon moving elsewhere (outside of a chuck per say) the issue starts up again.

This is standing absolutely still on an empty planet for a pro longed time. Barely any grids in the world.

EDIT3: Also confirmed this on a completely empty world, on a single planet over time.