Zeek Posted March 2, 2015 Share Posted March 2, 2015 These chunks are 10x10 tiles of inefficient space.... Cells are by far.. the best way to pre-load map data and alleviate server stress. Although requiring more bandwidth at any given moment to load up the area, it is a lot more easier to deal with. Having 2,500 505 cells ~100KB - ~1MB ~300KB - ~3MB to process vs 250,000 454,000 Chunks non-sequential and sizes of ~1KB - ~11KB with only around 2,000 chunks larger than that. Replace the mapxxxx_xxx.bin Chunks with mapxx_xx.bin Cells. It takes a total of 30 minutes on avg to process 100,000 chunks... Where the same process could be repeated at on 1,000 ~112 cells in a matter of 2 minutes or less... Reffering to Soft-Wipe command. The problem isn't that the size of the data changes, what changes is the amount of buffering required to process each file is exceeding less at the Cell level compared to the Chunk level. It's easier for computers to process LARGE files vs numerous small files equal to the same size of data. Please make this something that I or other server hosters can test. We've been messing around with this stuff long enough to know that this issue is just one of many and may be a big fix to most of our problems... especially with the time it takes backing up the files and this will make HDD's actually feasible for hosting as they're not good at performing the many IOPS that is required of a server with high population using chunks. This process is also healthier for SSDs.. Less read and write operations overall from this issue. Edit: More accurate numbers based on calculations made from the maps absolute values. RichCoconut and Jack Bower 2 Link to comment Share on other sites More sharing options...
ricky Posted March 2, 2015 Share Posted March 2, 2015 My map folder gets to 2-3GB with 168,000+ files. After 2 days of hosting.I can't keep a smooth server running for a week, and soft reset does not help.Zombies start to lag and then huge lag spikes happen. Coming from somebody that hosts allot of games and always testing methods and hardware to get the most performance out of them.Yes this game needs it's Map system scrapped and rewritten, to put it bluntly. Not even high-end server hardware can compensate for the performance losses that get introduced over time.When i speak to certain hosting company's about hosting this game they don't want to know. And i can see why. Zeek 1 Link to comment Share on other sites More sharing options...
EnigmaGrey Posted March 2, 2015 Share Posted March 2, 2015 Don't get how that's possible. If the chunks are 10 x 10, then there should only be a maximum of 10800 chunk files, unless the game is making a duplicate chunk for every player on the server. Link to comment Share on other sites More sharing options...
Zeek Posted March 2, 2015 Author Share Posted March 2, 2015 Don't get how that's possible. If the chunks are 10 x 10, then there should only be a maximum of 10800 chunk files, unless the game is making a duplicate chunk for every player on the server. Chunks are 10x10 tiles = 100 tiles and Cells are 30x30 chunks = 900 chunks or 90,000 Tiles. Muld is 20x20 Cells = 400 cells.. Muld = 360,000 Chunks or 36,000,000 tiles alone..... West Point is 15x7 Cells =105 and 10x5 Cells = 50(because of it's shape there are two measurements), 105+50 = 155 Cells or 94,000 Chunks or 9,400,000 Tiles. Total of 454,000 Chunks or 454,000 mapxxxx_xxx.bin files which comes to a total of 45,400,000 tiles(thankfully we don't have a bin file made for each tile)... This is a problem.. Not that all chunks will ever be loaded, but they stack up quickly and cause huge buffer overheads... Even wiping 80k out of 120k chunks makes a huge difference to zombie behavior and the lag spikes for a short while. Where doing a server restart does little to nothing. Jack Bower and RichCoconut 2 Link to comment Share on other sites More sharing options...
ricky Posted March 2, 2015 Share Posted March 2, 2015 I can upload a map save with 168,000+ files if you're interested. Just let me know and i'll have a link.This is a 20 slot server. Started on Friday 27th Feb. It's the same every Friday i reset it. Within a day or 2. 100,000+ files. Zeek 1 Link to comment Share on other sites More sharing options...
lemmy101 Posted March 2, 2015 Share Posted March 2, 2015 There are for sure likely better ways to manage the data sent to the clients, but trust me we tried loading entire cells at once, in fact this is exactly how it used to be, and the calcs required to do the collision / vision / lighting data for an entire cell would kill the shit out of the framerate in the game, the 10x10 chunks were chosen for that reason as on the client they can be loaded and processed quick enough for the streaming to work. This was before multiplayer. We're planning to reorganize how map data is sent to clients to get the vehicles to work etc, but upping the chunk size on clientside to entire cells isn't an option. The servers it would be more plausible but the data would need to work as 10x10 for the client to perform properly so I'm not sure how it'd be possible to organize this so servers worked with entire cells without any 10x10 segmentation. If we're just talking about how the actual files are stored on HD on the server maybe there's something we can change, but as I said above, its very difficult to get rid of the concept of 10x10 chunks without heavy rewriting of the game and I fear the extra load required to give the clients 10x10 chunk files they need would have a much bigger detrimental effect than the gain from server files operating on bigger sections of the map. Since these files are currently sent directly to the client as required, it would result in a LOT more data with large sections of the map requiring sending to the clients for the map stuff (potentially 450 tiles in every direction if a player is in the centre of a cell), which at present with much much less than that is already a strain. Cheers for the info we'll certainly bear it in mind when we rejig the multiplayer stuff. Ideally we would be able to optimize this a lot in future and its on the roadmap, but its by far far more complicated a problem than upping the chunk size dimension variables. Realmkeeper 1 Link to comment Share on other sites More sharing options...
Zeek Posted March 2, 2015 Author Share Posted March 2, 2015 Thank you for the response! They wouldn't need to process the lighting for that much though, perhaps limiting distance draw within the loaded cell? Food for thought. Is it possible to send segmentations of a Cell without saving each chunk individually in order to obtain this request and without drastically altering loaded sections? Cell 22x22 contains Chunks 1-30 x 1-30. Player see Cell 22x22 and chunks 1x1, 1x2, 1x3 of that Cell. Server sends Cell and Chunks as MapCell22x22.bin Check if exists, if not Create Cell and add Chunk data, else if update Chunk data to MapCell22x22.bin Client with chunks 1x1, 1x2, 1x3 from Server. Adding/Updating chunks from the Server's Cell file as they are within view. All while contained in just a single MapCellxx_xx.Bin file instead of individual chunk files. If the chunks are stored inside a larger file it'll streamline the chunks sequentially a lot faster and update the same unless I'm mistaken? I hope something like that wouldn't be a huge overhaul, but it would certainly help a lot on the server side of things. Link to comment Share on other sites More sharing options...
Kitt Frostpaws Posted March 2, 2015 Share Posted March 2, 2015 -snip- Is.. Is this a bot? Link to comment Share on other sites More sharing options...
Kirrus Posted March 2, 2015 Share Posted March 2, 2015 -snip- Is.. Is this a bot? Kitt, you should know better. Report spam, this, don't comment on them, unless you like the idea of all your posts being forever purged and being marked as a spammer by accident Kitt Frostpaws 1 Link to comment Share on other sites More sharing options...
Kitt Frostpaws Posted March 2, 2015 Share Posted March 2, 2015 -snip- Is.. Is this a bot? Kitt, you should know better. Report spam, this, don't comment on them, unless you like the idea of all your posts being forever purged and being marked as a spammer by accident So.. That's a yes? Link to comment Share on other sites More sharing options...
Zeek Posted March 3, 2015 Author Share Posted March 3, 2015 Please stop derailing my thread... Link to comment Share on other sites More sharing options...
Kirrus Posted March 3, 2015 Share Posted March 3, 2015 Please stop derailing my thread... We don't keep threads on hard-rails around here Tangents are accepted, even if this tangent was spam-bot triggered. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now