Favoriting the "Rip Clothing" recipe if you have too many clothing mods can corrupt your character data

[Trigg]

• Version?
   41.78.16
• Singleplayer/Multiplayer?
   Tested in singleplayer, but should happen in any environment.
• Mods?
   Yes, see a triggering combination below
• Old or new save?
   Any save.

• Reproduction steps:
  1. Be in a modpack with way too many clothing items.
  2. Open the crafting menu. Big lag spikes, but that's a known problem.
  3. Favorite the "Rip Clothing" recipe
  4. Save and quit
  5. Try to load again
  6. You will spawn where your character was, but with a zeroed-out character with the name "None None"
  
  7. Find the following error message in the logs:

ERROR: General     , 1699919034597> ExceptionLogger.logException> Exception thrown java.lang.RuntimeException: invalid lua table type 99 at KahluaTableImpl.load line:351.
ERROR: General     , 1699919034602> DebugLogStream.printException> Stack trace:
java.lang.RuntimeException: invalid lua table type 99
	at se.krka.kahlua.j2se.KahluaTableImpl.load(KahluaTableImpl.java:351)
	at se.krka.kahlua.j2se.KahluaTableImpl.load(KahluaTableImpl.java:325)
	at zombie.iso.IsoMovingObject.load(IsoMovingObject.java:631)
	at zombie.characters.IsoGameCharacter.load(IsoGameCharacter.java:5511)
	at zombie.characters.IsoPlayer.load(IsoPlayer.java:972)
	at zombie.iso.IsoObject.load(IsoObject.java:987)
	at zombie.savefile.PlayerDB.loadPlayer(PlayerDB.java:625)
	at zombie.savefile.PlayerDB.loadLocalPlayer(PlayerDB.java:656)
	at zombie.iso.IsoCell.LoadPlayer(IsoCell.java:6089)
	at zombie.iso.IsoWorld.init(IsoWorld.java:2654)
	at zombie.gameStates.GameLoadingState$1.runInner(GameLoadingState.java:268)
	at zombie.gameStates.GameLoadingState$1.run(GameLoadingState.java:225)
	at java.base/java.lang.Thread.run(Unknown Source)

 

 

I did a little digging.

First, here's my testing setup to confirm the problem isn't bound to a specific mod:

 

Test A: Just dakimakura mods (don't judge; they're very funny in multiplayer)

1. Create a new save with just the following mod list:
    ModManager,
    AlexDakis,
    WaifuDakis,
    ProfessionFramework,
    DakiAddonDDLC,
    DakiAddonTouhou,
    DucklingsDakis,
2. Load in, open crafting menu, favorite any regular recipe, like opening an umbrella. Just to make sure the table for favorited recipes exists, to compare the database files later.
3. Save and quit, then backup players.db
4. Load back in, everything should be fine. Now favorite the recipe for "Rip Clothing". Specifically the one for regular cloth, since it should have more entries.
5. Save and quit; I backed up players.db again.
  5.1. Note how players.db increased in file size from 16kb to 40kb just from this.
6. Load back in, and everything should be fine.

Test B: Just AuthenticZ
  1. Same as Test A, but now with the following mod list:
       ModManager,
       AuthenticZLite,
  2. It doesn't break either. players.db went from 16kb to 32kb this time.

Test C: Both!
  1. Mod list:
      ModManager,
      AlexDakis,
      WaifuDakis,
      ProfessionFramework,
      DakiAddonDDLC,
      DakiAddonTouhou,
      DucklingsDakis,
      AuthenticZLite,
  2. Now it should have broken. players.db hits 52kb. "None None" returns.

Here are all my test saves, for reproduction.

 

allTestSaves.7z

 

From this testing, it could still be a hard incompatibility between AuthenticZ and these Dakimura mods, but I assure you, it is not.

 

You see, I did some more digging. Mostly to see if I could manually fix the save file.

Ah, the rabbit-hole I jumped into.

I pulled an all-nighter decompiling, debugging the game, and reverse-engineering the player data format. It was fun.

 

Short story:

GameWindow$StringUTF saves and loads string lengths as signed shorts.

Mods can create situations where the game saves very long strings (>32727 bytes), overflowing the signed short, which wraps around to a negative number.

When loading the string, it reads a string value announcing negative length, exits early, and breaks the rest of the sequence for parsing character data.

 

Long story:

I'll skip over the reverse engineering process, and get to the point.

 

When favoriting a recipe, the game has to save that to your character data, so it saves a string in a binary format describing the recipe entry.

For some reason, this string is also including all the names for all the possible ingredients that can be used for the recipe.

I'm not sure if that's by design or not, and it doesn't really matter. It doesn't directly cause any problems, and B42 will certainly get rid of it with the crafting rework.

The real problem is the size of the string. This string is 34999 bytes long. This is a very large string.

 

I have found that a string in the save format has a 2-byte "string length" header - a "short" number.

 

In Java, number data types are always signed, meaning their first bit represents whether the number is negative or not.

An unsigned short can represent values between 0 and 65535, while a signed short can only represent values between -32768 and 32767... And I just said my string was 34999 bytes long.

Java's overflow behavior is to silently wrap around the number to a negative value, so nothing errors out while saving the game.

When loading the game, however...

 

When reading a string with an advertised length of -30537, the decoder (parser? reader? whatever.) treats it as an empty string and doesn't read anything more from the buffer.

The game continues sequentially reading the binary data, expecting a header byte describing the data type of the next thing it has to read.

Instead, it hits the start of the dropped string, which in my case, when interpreted as a number, is 99 - Causing the error stacktrace in my logs, all the way back there.

 

The program in these screenshots is ImHex - it was extremely useful to reverse engineer the binary data from the save format.

Here's an archive of the ImHex project and the binary data I was toying around with on these screenshots: ImHex workspace thing.7z

Edited November 14, 2023 by Trigg
Finished up the report with my own investigation

[Sir Warock]

Man.
Thank you.

I have lost my account data so many times now, and my server has been riddled with it, too. People losing their account data from stuff like this.
I hope this will be fixed.
Just lost another character today. But now I know why it happens.

[Trigg]

@Sir Warock

 

Oh, looks like I forgot to mention this after I wrote the bug report - I made a mod on the workshop to prevent the corruption: https://steamcommunity.com/sharedfiles/filedetails/?id=3085491446

Existing players will have their favorites reset, but it should be otherwise safe to add to a running save.

 

It doesn't restore already corrupted characters, though.

 

Doing that manually is possible - I've done it myself - it takes a lot of work, but ultimately boils down to finding the offending string, setting its "length" bytes to 00 00, and deleting its contents - not a byte too little, not a byte too much.

If you or anyone else wants to bother to try, though, take a look at the ImHex project I attached to the report, and see what you can do for yourself from there.