I'm also running into this issue. We have a dedicated server running on G-portal and my friends can play the game without having any issues but me.
Without VPN, I simply get "Failed to download map from the server" error end of the map chunk load. like E.g: 278/279 chunks loaded.
I tried using VPN but at some point interaction with the world becomes unresponsive and other players becomes invisible (assuming that my connection drops).
I also tried running packet loss test and I have 0.00% package loss w/wo VPN. My connection speed is 250mb/ps.
The server has 8 GB RAM. My computer has 16 GB RAM. We are 4 people in total and even the server is empty I run into this problem.
I tried disabling all mods on the server and the issue persists.
I tried mostly everything you listed above but still no luck. I'm unable to play the game, it happens in every server I try to connect.