Jump to content
  • 0

loaded2=zombie.network.ServerMap$ServerCell


nwr

Question

I have a Linux x64 server(3.2x2 GHz) and after some time(~2 days) I start to recieve A LOT of huge messages like that:

loaded2=[zombie.network.ServerMap$ServerCell@427f1590, zombie.network.ServerMap$ServerCell@7422b0de, zombie.network.ServerMap$ServerCell@1189fa63, zombie.network.ServerMap$ServerCell@79958293, zombie.network.ServerMap$ServerCell@46afbf41, zombie.network.ServerMap$ServerCell@92affad, zombie.network.ServerMap$ServerCell@65f7bb8e, zombie.network.ServerMap$ServerCell@6b3a9bee, zombie.network.ServerMap$ServerCell@54d35d41, zombie.network.ServerMap$ServerCell@8f81e9, zombie.network.ServerMap$ServerCell@4d99e848, zombie.network.ServerMap$ServerCell@3360f6c5, zombie.network.ServerMap$ServerCell@22e919dd, zombie.network.ServerMap$ServerCell@75a10e12, zombie.network.ServerMap$ServerCell@20a2fc13, zombie.network.ServerMap$ServerCell@2c535422, zombie.network.ServerMap$ServerCell@12b89aea, zombie.network.ServerMap$ServerCell@15c28240, zombie.network.ServerMap$ServerCell@14f542e9, zombie.network.ServerMap$ServerCell@71fbd9be, zombie.network.ServerMap$ServerCell@373305ba, zombie.network.ServerMap$ServerCell@76bd12a5, zombie.network.ServerMap$ServerCell@77134b89, zombie.network.ServerMap$ServerCell@13c44c72, zombie.network.ServerMap$ServerCell@3c76da81, zombie.network.ServerMap$ServerCell@1f036a2a, zombie.network.ServerMap$ServerCell@3b2fcb7f, zombie.network.ServerMap$ServerCell@3ed91d34]
finish loading cell 155,134 ms=5365.96
loaded2=[zombie.network.ServerMap$ServerCell@7422b0de, zombie.network.ServerMap$ServerCell@1189fa63, zombie.network.ServerMap$ServerCell@79958293, zombie.network.ServerMap$ServerCell@46afbf41, zombie.network.ServerMap$ServerCell@92affad, zombie.network.ServerMap$ServerCell@65f7bb8e, zombie.network.ServerMap$ServerCell@6b3a9bee, zombie.network.ServerMap$ServerCell@54d35d41, zombie.network.ServerMap$ServerCell@8f81e9, zombie.network.ServerMap$ServerCell@4d99e848, zombie.network.ServerMap$ServerCell@3360f6c5, zombie.network.ServerMap$ServerCell@22e919dd, zombie.network.ServerMap$ServerCell@75a10e12, zombie.network.ServerMap$ServerCell@20a2fc13, zombie.network.ServerMap$ServerCell@2c535422, zombie.network.ServerMap$ServerCell@12b89aea, zombie.network.ServerMap$ServerCell@15c28240, zombie.network.ServerMap$ServerCell@14f542e9, zombie.network.ServerMap$ServerCell@71fbd9be, zombie.network.ServerMap$ServerCell@373305ba, zombie.network.ServerMap$ServerCell@76bd12a5, zombie.network.ServerMap$ServerCell@77134b89, zombie.network.ServerMap$ServerCell@13c44c72, zombie.network.ServerMap$ServerCell@3c76da81, zombie.network.ServerMap$ServerCell@1f036a2a, zombie.network.ServerMap$ServerCell@3b2fcb7f, zombie.network.ServerMap$ServerCell@3ed91d34]
finish loading cell 156,134 ms=3543.8767
loaded2=[zombie.network.ServerMap$ServerCell@1189fa63, zombie.network.ServerMap$ServerCell@79958293, zombie.network.ServerMap$ServerCell@46afbf41, zombie.network.ServerMap$ServerCell@92affad, zombie.network.ServerMap$ServerCell@65f7bb8e, zombie.network.ServerMap$ServerCell@6b3a9bee, zombie.network.ServerMap$ServerCell@54d35d41, zombie.network.ServerMap$ServerCell@8f81e9, zombie.network.ServerMap$ServerCell@4d99e848, zombie.network.ServerMap$ServerCell@3360f6c5, zombie.network.ServerMap$ServerCell@22e919dd, zombie.network.ServerMap$ServerCell@75a10e12, zombie.network.ServerMap$ServerCell@20a2fc13, zombie.network.ServerMap$ServerCell@2c535422, zombie.network.ServerMap$ServerCell@12b89aea, zombie.network.ServerMap$ServerCell@15c28240, zombie.network.ServerMap$ServerCell@14f542e9, zombie.network.ServerMap$ServerCell@71fbd9be, zombie.network.ServerMap$ServerCell@373305ba, zombie.network.ServerMap$ServerCell@76bd12a5, zombie.network.ServerMap$ServerCell@77134b89, zombie.network.ServerMap$ServerCell@13c44c72, zombie.network.ServerMap$ServerCell@3c76da81, zombie.network.ServerMap$ServerCell@1f036a2a, zombie.network.ServerMap$ServerCell@3b2fcb7f, zombie.network.ServerMap$ServerCell@3ed91d34]
finish loading cell 170,97 ms=4001.465
loaded2=[zombie.network.ServerMap$ServerCell@79958293, zombie.network.ServerMap$ServerCell@46afbf41, zombie.network.ServerMap$ServerCell@92affad, zombie.network.ServerMap$ServerCell@65f7bb8e, zombie.network.ServerMap$ServerCell@6b3a9bee, zombie.network.ServerMap$ServerCell@54d35d41, zombie.network.ServerMap$ServerCell@8f81e9, zombie.network.ServerMap$ServerCell@4d99e848, zombie.network.ServerMap$ServerCell@3360f6c5, zombie.network.ServerMap$ServerCell@22e919dd, zombie.network.ServerMap$ServerCell@75a10e12, zombie.network.ServerMap$ServerCell@20a2fc13, zombie.network.ServerMap$ServerCell@2c535422, zombie.network.ServerMap$ServerCell@12b89aea, zombie.network.ServerMap$ServerCell@15c28240, zombie.network.ServerMap$ServerCell@14f542e9, zombie.network.ServerMap$ServerCell@71fbd9be, zombie.network.ServerMap$ServerCell@373305ba, zombie.network.ServerMap$ServerCell@76bd12a5, zombie.network.ServerMap$ServerCell@77134b89, zombie.network.ServerMap$ServerCell@13c44c72, zombie.network.ServerMap$ServerCell@3c76da81, zombie.network.ServerMap$ServerCell@1f036a2a, zombie.network.ServerMap$ServerCell@3b2fcb7f, zombie.network.ServerMap$ServerCell@3ed91d34]
finish loading cell 171,97 ms=3854.692

Game zones loading become VERY slow and CPU at 100%, but Ping stay low.

PZ v30.15

Link to comment
Share on other sites

18 answers to this question

Recommended Posts

If you want to try a fix I've been working on for this:

  • Download this ZIP file and extract it into your server's ProjectZomboid directory (wherever the media/ and zombie/ directories are).
  • This will change some of the Lua files, so clients connecting will get "Lua files don't match" errors.  Set DoLuaChecksum=false in the server's .ini file if you want to test this with clients joining.

This should reduce the "finish loading cell" times a lot, although there might be other causes I haven't discovered yet.  Lots of Erosion objects does increase the load time, but I haven't seen 5 second load times yet.

Link to comment
Share on other sites

Does the server use any mods?  If so, try a server without mods enabled.  The main thing that slows down the "finish loading cell" stage is all the LoadGridsquare callbacks.  Those are calls from Java into Lua for every single map location after they are loaded, which is quite slow.  Even a simple mod like the Spraypaint mod adds a LoadGridsquare callback, which is bad news for the server.  The game itself uses LoadGridsquare callbacks for farming, trapping, rainbarrels, and camping which is something of a concern without mods adding any extra work.  On my laptop the "finish loading cell" stage took 1093, 384, 282, and 242 ms from a cold start just now.

Link to comment
Share on other sites

If you see that error with GameTime.save() again, can you post your server's map_t.bin file?  There's too much data being written for some reason.

Error with packet of type: 6 for MaksTaker

java.nio.BufferOverflowException

        at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:183)

        at java.nio.ByteBuffer.put(ByteBuffer.java:832)

        at zombie.GameWindow.WriteStringUTF(GameWindow.java:1991)

        at zombie.GameWindow.WriteString(GameWindow.java:1977)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:231)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:222)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:237)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:223)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:237)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:223)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:237)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:223)

        at zombie.GameTime.save(GameTime.java:1748)

        at zombie.network.GameServer.sendPlayerConnect(GameServer.java:1853)

        at zombie.network.GameServer.receivePlayerConnect(GameServer.java:3826)

        at zombie.network.GameServer.mainLoopDealWithNetData(GameServer.java:1210)

        at zombie.network.GameServer.main(GameServer.java:357)

Connection Lost for id=1 username=RayWong

Connection Lost for id=2 username=MaksTaker

Connection Lost for id=2 username=null

Connection Lost for id=1 username=null

Connection Lost for id=2 username=null

Connection Lost for id=2 username=null

Connection Lost for id=3 username=null

Connection Lost for id=3 username=null

 

http://rghost.ru/8hmdZbsgG

Link to comment
Share on other sites

If you want to try a fix I've been working on for this:

  • Download this ZIP file and extract it into your server's ProjectZomboid directory (wherever the media/ and zombie/ directories are).
  • This will change some of the Lua files, so clients connecting will get "Lua files don't match" errors.  Set DoLuaChecksum=false in the server's .ini file if you want to test this with clients joining.

This should reduce the "finish loading cell" times a lot, although there might be other causes I haven't discovered yet.  Lots of Erosion objects does increase the load time, but I haven't seen 5 second load times yet.

 

That's definitely better, now finish cell loading ~700ms:

*** SERVER STARTED ****

server is listening on port 16261

connecting to public server list

User lacee is trying to connect.

Connected new client lacee ID # 0 and assigned DL port 16262

testing TCP download port 16262

client connected to TCP download port 16262 ok

loaded chunks for cell 166,96 ms=418.67654

loaded chunks for cell 166,97 ms=362.45187

RecalcAll for cell 166,96 ms=1997.573

RecalcAll for cell 166,97 ms=2416.9265

loaded2=[]

finish loading cell 166,96 ms=2540.2986

loaded2=[]

finish loading cell 166,97 ms=700.2321

loaded chunks for cell 165,97 ms=44.976143

RecalcAll for cell 165,97 ms=232.57248

loaded2=[]

finish loading cell 165,97 ms=584.5851

loaded chunks for cell 165,96 ms=138.4806

RecalcAll for cell 165,96 ms=177.67143

loaded2=[]

finish loading cell 165,96 ms=615.5104

loaded chunks for cell 165,98 ms=211.89096

loaded chunks for cell 166,98 ms=237.16252

RecalcAll for cell 165,98 ms=465.6067

RecalcAll for cell 166,98 ms=523.2783

loaded2=[]

finish loading cell 165,98 ms=1250.3823

loaded2=[]

finish loading cell 166,98 ms=681.87646

loaded chunks for cell 164,97 ms=57.149204

loaded chunks for cell 164,98 ms=98.28331

RecalcAll for cell 164,97 ms=256.88007

RecalcAll for cell 164,98 ms=359.16537

loaded2=[]

finish loading cell 164,97 ms=830.9981

loaded2=[]

finish loading cell 164,98 ms=627.8606

 

 

But I've got some:

 

loaded2=[]

finish loading cell 170,98 ms=969.4551

loaded chunks for cell 171,98 ms=26.135971

RecalcAll for cell 171,98 ms=233.1984

loaded2=[]

finish loading cell 171,98 ms=809.5926

янв 23, 2015 2:56:03 PM zombie.ai.astar.AStarPathFinder findPath

SEVERE: null

java.lang.ArrayIndexOutOfBoundsException: 4548

        at zombie.core.utils.ExpandableBooleanList.setValue(ExpandableBooleanList.java:71)

        at zombie.ai.astar.AStarPathFinder.addToClosed(AStarPathFinder.java:1014)

        at zombie.ai.astar.AStarPathFinder.findPath(AStarPathFinder.java:625)

        at zombie.ai.astar.AStarPathFinder.Cycle(AStarPathFinder.java:247)

        at zombie.PathfindManager$PathfindJob.Process(PathfindManager.java:291)

        at zombie.PathfindManager$PathfindJob.access$0(PathfindManager.java:259)

        at zombie.PathfindManager.run(PathfindManager.java:110)

        at zombie.PathfindManager.access$0(PathfindManager.java:58)

        at zombie.PathfindManager$2.run(PathfindManager.java:375)

        at java.lang.Thread.run(Thread.java:745)

янв 23, 2015 2:56:03 PM zombie.ai.astar.AStarPathFinder findPath

SEVERE: null

java.lang.ArrayIndexOutOfBoundsException: 4548

        at zombie.core.utils.ExpandableBooleanList.setValue(ExpandableBooleanList.java:71)

        at zombie.ai.astar.AStarPathFinder.addToClosed(AStarPathFinder.java:1014)

        at zombie.ai.astar.AStarPathFinder.findPath(AStarPathFinder.java:625)

        at zombie.ai.astar.AStarPathFinder.Cycle(AStarPathFinder.java:247)

        at zombie.PathfindManager$PathfindJob.Process(PathfindManager.java:291)

        at zombie.PathfindManager$PathfindJob.access$0(PathfindManager.java:259)

        at zombie.PathfindManager.run(PathfindManager.java:110)

        at zombie.PathfindManager.access$0(PathfindManager.java:58)

        at zombie.PathfindManager$2.run(PathfindManager.java:375)

        at java.lang.Thread.run(Thread.java:745)

Link to comment
Share on other sites

The server loads the map in blocks of 7x7 chunks at a time. (Each chunk is a 10x10 tile area).

When the server finishes loading a cell, it prints out the other cells it still has to load in.

In this case, it appears the server has a big back-log of cells to load in.

Also, the "finish loading cell" times are quite long, 5.3 seconds, 3.5 seconds, 4.0 seconds and 3.8 seconds.

After 2 days, I imagine the server is just running out of memory, you may want to keep an eye on that.

It could be a memory leak getting worse over the 2 days.

4 Players and 8Gb RAM

loaded2=[]

finish loading cell 149,102 ms=3952.8682

loaded chunks for cell 150,101 ms=41.700172

loaded chunks for cell 150,102 ms=37.380596

RecalcAll for cell 150,101 ms=124.82262

RecalcAll for cell 150,102 ms=123.9954

[loaded2=[]

finish loading cell 150,101 ms=3291.6377

loaded chunks for cell 150,100 ms=11.942501

players

loaded2=[]

finish loading cell 150,102 ms=3352.7864

Unknown command players

RecalcAll for cell 150,100 ms=126.89577

loaded2=[]

finish loading cell 150,100 ms=2855.4624

Players connected (3):

-Sazerac

-optional

-Rattlehead

loaded chunks for cell 183,72 ms=11.903647

loaded chunks for cell 184,72 ms=12.505292

RecalcAll for cell 183,72 ms=121.32469

RecalcAll for cell 184,72 ms=119.332

loaded2=[]

finish loading cell 183,72 ms=2957.7622

User Ara> is trying to connect.

Connected new client Ara> ID # 8 and assigned DL port 16264

testing TCP download port 16264

client connected to TCP download port 16264 ok

loaded2=[]

finish loading cell 184,72 ms=2869.4949

loaded chunks for cell 151,100 ms=10.210245

loaded chunks for cell 151,101 ms=11.580391

RecalcAll for cell 151,100 ms=119.640854

RecalcAll for cell 151,101 ms=120.96185

loaded2=[]

finish loading cell 151,100 ms=2825.705

loaded2=[]

finish loading cell 151,101 ms=2879.4504

loaded chunks for cell 185,70 ms=10.065434

loaded chunks for cell 185,71 ms=9.682536

loaded chunks for cell 185,72 ms=10.952149

RecalcAll for cell 185,70 ms=194.35239

RecalcAll for cell 185,71 ms=143.9832

RecalcAll for cell 185,72 ms=280.0478

loaded2=[]

finish loading cell 185,70 ms=3606.666

1.png?extra=JjOrPb336aj_jdBHRKtBydYFINs9

Link to comment
Share on other sites

If you want to try a fix I've been working on for this:

  • Download this ZIP file and extract it into your server's ProjectZomboid directory (wherever the media/ and zombie/ directories are).
  • This will change some of the Lua files, so clients connecting will get "Lua files don't match" errors.  Set DoLuaChecksum=false in the server's .ini file if you want to test this with clients joining.

This should reduce the "finish loading cell" times a lot, although there might be other causes I haven't discovered yet.  Lots of Erosion objects does increase the load time, but I haven't seen 5 second load times yet.

 

I've got a problem when I restarted the server with that fix:

*** SERVER STARTED ****

server is listening on port 16261

connecting to public server list

connecting to public server list

User Alexander Petrovich is trying to connect.

Connected new client Alexander Petrovich ID # 0 and assigned DL port 16262

testing TCP download port 16262

Error with packet of type: 2 for Alexander Petrovich

java.nio.BufferOverflowException

        at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:183)

        at java.nio.ByteBuffer.put(ByteBuffer.java:832)

        at zombie.GameWindow.WriteStringUTF(GameWindow.java:1991)

        at zombie.GameWindow.WriteString(GameWindow.java:1977)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:231)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:222)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:237)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:223)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:237)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:223)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:237)

        at se.krka.kahlua.j2se.KahluaTableImpl.save(KahluaTableImpl.java:223)

        at zombie.GameTime.save(GameTime.java:1748)

        at zombie.network.GameServer.receiveClientConnect(GameServer.java:3684)

        at zombie.network.GameServer.mainLoopDealWithNetData(GameServer.java:1329)

        at zombie.network.GameServer.main(GameServer.java:357)

TCP port test: accept() timed out, client will have to try again

 

 

and nobody can connect after that.

 

UPD: "rm -rf ./Multiplayer/servertest/" helped me

Link to comment
Share on other sites

It looks like there are 1192 farm plots, I guess that is the problem.

We'll have to use a different method of sending map_t.bin to the clients.

 

EDIT:

It turns out the client doesn't need the camping/farming/trapping modData, so this is fixable by not sending that stuff to the client.

 

But I found another problem, it takes the server 3.5 seconds to save map_t.bin with that much data, and that happens every 10 seconds on the server.

Link to comment
Share on other sites

Does the server use any mods?  If so, try a server without mods enabled.  The main thing that slows down the "finish loading cell" stage is all the LoadGridsquare callbacks.  Those are calls from Java into Lua for every single map location after they are loaded, which is quite slow.  Even a simple mod like the Spraypaint mod adds a LoadGridsquare callback, which is bad news for the server.  The game itself uses LoadGridsquare callbacks for farming, trapping, rainbarrels, and camping which is something of a concern without mods adding any extra work.  On my laptop the "finish loading cell" stage took 1093, 384, 282, and 242 ms from a cold start just now.

No mods, clean server. If I do a hard reset(rm -rf ./Multiplayer/servertest/) - cell loading become normal ~150ms, but after 2-3 real days(20 players online max) it will be 5+sec again, even if I restart the server. May be it's errosion or loot respawn?

Link to comment
Share on other sites

After 2 days, I imagine the server is just running out of memory, you may want to keep an eye on that.

It could be a memory leak getting worse over the 2 days.

I have 20 players online and -Xms6g/-Xmx6g. Ok, next time I'll try 8g/10g with the same conditions.

Link to comment
Share on other sites

If you want to try a fix I've been working on for this:

  • Download this ZIP file and extract it into your server's ProjectZomboid directory (wherever the media/ and zombie/ directories are).
  • This will change some of the Lua files, so clients connecting will get "Lua files don't match" errors.  Set DoLuaChecksum=false in the server's .ini file if you want to test this with clients joining.

This should reduce the "finish loading cell" times a lot, although there might be other causes I haven't discovered yet.  Lots of Erosion objects does increase the load time, but I haven't seen 5 second load times yet.

 

Hi! 

Can we expect these changes to the build of 31? Or the server part will not be affected?

Fix fine, but it does not allow to include verification Lua, which leads to a large number of cheaters.

Link to comment
Share on other sites

Server just started, 1 player, 14G RAM, 4x3.2GHz Xeon:

*** SERVER STARTED ****
server is listening on port 16261
connecting to public server list
User Dobby is trying to connect.
Connected new client Dobby ID # 0 and assigned DL port 16262
testing TCP download port 16262
client connected to TCP download port 16262 ok
loaded chunks for cell 151,140 ms=165.00122
loaded chunks for cell 152,140 ms=72.63654
loaded chunks for cell 151,141 ms=46.840588
loaded chunks for cell 152,141 ms=44.824715
loaded chunks for cell 151,142 ms=87.95642
loaded chunks for cell 152,142 ms=74.23387
RecalcAll for cell 151,140 ms=860.9991
RecalcAll for cell 152,140 ms=434.21915
RecalcAll for cell 151,141 ms=160.43575
RecalcAll for cell 152,141 ms=245.04771
RecalcAll for cell 151,142 ms=153.41873
RecalcAll for cell 152,142 ms=132.37213
loaded2=[]
finish loading cell 151,140 ms=6556.6655
loaded2=[zombie.network.ServerMap$ServerCell@3e2463a0, zombie.network.ServerMap$ServerCell@4708bd3f, zombie.network.ServerMap$ServerCell@ec09788, zombie.network.ServerMap$ServerCell@61c6949]
finish loading cell 152,140 ms=6283.6123
loaded2=[zombie.network.ServerMap$ServerCell@4708bd3f, zombie.network.ServerMap$ServerCell@ec09788, zombie.network.ServerMap$ServerCell@61c6949]
finish loading cell 151,141 ms=5720.8335
loaded2=[zombie.network.ServerMap$ServerCell@ec09788, zombie.network.ServerMap$ServerCell@61c6949]
finish loading cell 152,141 ms=5705.814
loaded2=[zombie.network.ServerMap$ServerCell@61c6949]
finish loading cell 151,142 ms=5885.9463
loaded2=[]
finish loading cell 152,142 ms=5306.5728

 

Each cell loading 5sec+

Link to comment
Share on other sites

The server loads the map in blocks of 7x7 chunks at a time. (Each chunk is a 10x10 tile area).

When the server finishes loading a cell, it prints out the other cells it still has to load in.

In this case, it appears the server has a big back-log of cells to load in.

Also, the "finish loading cell" times are quite long, 5.3 seconds, 3.5 seconds, 4.0 seconds and 3.8 seconds.

After 2 days, I imagine the server is just running out of memory, you may want to keep an eye on that.

It could be a memory leak getting worse over the 2 days.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...