Search Unity

Obscure case for PlayerPrefs getting wiped on Windows 10

Discussion in 'Windows' started by skaarjslayer, Nov 18, 2016.

  1. skaarjslayer

    skaarjslayer

    Joined:
    Oct 10, 2013
    Posts:
    108
    Hey guys,

    Basically, I got an obscure case for PlayerPrefs getting wiped on Windows 10 builds. Basically, if you shut down a Windows 10 x86 device by pressing and holding the power button while the game is running, the PlayerPrefs for our game appears to be completely wiped.

    This does not occur if I shutdown the device through start menu.
    This does not occur if I restart the device through start menu.
    This does not occur if the device sleeps or shuts down on low battery.
    This does not occur if the device shuts down due to 0% battery.
    This only seems to occur if you press and hold the power button on the device.

    I wouldn't shut a computer down this way. But I can't say anyone who downloaded our game wouldn't, and if they did they'd lose their save. The only problem is, this doesn't seem to occur consistently. For instance, after the player prefs were wiped the first time, I attempted to create save data and force shut down again using the same build and the issue did not occur. To make matters worse, I couldn't reproduce the issue with a build attached to a debugger in Visual Studio, so there are no logs/errors I can provide that show anything.

    Easy questions to get out of the way:

    1) Yes, I've made sure that I am indeed saving to PlayerPrefs correctly before the forced shutdown occurs.
    2) This was made with a Windows 10 build made using Unity 5.4.3f1. It's been confirmed to happen on older Unity 5 builds, such as 5.4.2, and even as far back as 5.2.4. This does not occur on our Unity 4 projects.
    3) I tested this using a x86 Windows 10 laptop and an x86 Windows 10 tablet

    I don't even know if this is an "issue". It's hard to say if it's an issue with Unity reading the PlayerPrefs file (maybe it has been corrupted because of the forced shutdown?), or if the file location for the data is somehow affected by the forced shutdown (something that the OS does, maybe going back to a previous state or something?). I don't really know. Is this a problem anyone else can confirm?

    EDIT: Maybe Windows has processes for handling safe app shut down on low/zero battery which is why it doesn't get affected in that case? However, if this occurs on a hard shutdown from the power button, I imagine this problem has the potential to occur in cases where the device (such as a desktop PC) loses power completely (like during a power outage). If that's the case, that'd be a bigger problem but I don't really want to be plugging my computer in and out of the outlet to test that.
     
    Last edited: Nov 18, 2016
  2. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    SkaarjSlayer, thank you so much for this bug report! We experience similar obscure savegame wipes and had no clue where to start looking.

    We can confirm this behaviour for our game built with Unity 5.3.7f1 UWP
    1. Windows 8.1 Pro Desktop 6.3.9600 Build 9600 x64
    2. Windows 10 Home 10.0.14393 Build 14393
    3. Windows 10 Pro 1607 Build 14393.447
    using the hardware reset button. This does not occur if I the hardware power button is pressed as it shuts down cleanly ("Shutdown..." screen appears).

    Unity 5.3.7f1 UWP
    * Microsoft Lumia 950 XL, Windows 10 Mobile 10.0.14393.448
    when battery is removed

    Cannot reproduce for:
    Unity 5.3.4f1 Tizen
    * Samsung Tizen Z3, SM-Z300H, Tizen 2.4.03
    * Samsung Tizen Z2 SM-Z200F, Tizen 2.4.0.5

    Unity 5.3.4f1 Android
    * Samsung Galaxy S4 GT-I9515, Android 5.0.1

    when battery is removed.

    Have you already filed a bug report?

    Update2: Cant reproduce with Tizen and Android, so the OPs observation is right. Trying a minimal demo on Windows builds only.
     
    Last edited: Nov 25, 2016
  3. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    I did some in-depth testing on a small demo app. The demo app checks if a PlayerPrefs exist and if there is none, it generates a 6 char random string and saves it into PlayerPrefs, otherwise the old string is displayed. There a two buttons which set a new string and save to PlayerPrefs respectively.

    My test devices were:
    * 520 - Nokia Lumia 520, Windows Phone 8.1
    * 550 - Nokia Lumia 550, Windows Mobile 10
    * 950 - Nokia Lumia 950 XL, Windows Mobile 10
    * Desktop - Desktop PC, Windows 8.1 Pro Desktop x64 [1]
    * Tizen - Samsung Tizen Z3, Tizen 2.4
    * Android - Samsung Galaxy S4, Android 5.0

    When I just start the app and do absolutely nothing (no new strings, no saving):
    * 520: removed battery 12x, no corruption
    * 550: removed battery 8x, 1x corrupted
    * 950: removed battery 12x, no corruption

    I could more reliably reproduce the corruption when I press "save new string" + "save to PlayerPrefs" twice and then remove the battery immediately:
    * 520: removed battery 7x, 3x corruptions, 1x old state restored(?)
    * 550: removed battery 4x, 3x corruptions
    * 950: removed battery 7x, 6x corruptions
    * Desktop: pressed hardware reset 6x, 5x corruptions, 1x old state restored(?)
    * Tizen: removed battery 7x, 6x old state restored
    * Android: removed battery 7x, 2x old states restored

    The error message in debug console is usually "Data corruption detected while reading player prefs. CRC 0 != -1473662496".

    Update: tested on Unity 5.5.0f2 (the latest beta) with Desktop, pressed hardware reset button 5x, 5x corruptions.

    [1] I recommend against doing this to often as Windows displays all kind of unhealthy error messages on reboot.
     
    Last edited: Nov 28, 2016
    MrEsquire likes this.
  4. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    I just filed a bug report to Unity.
     
  5. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    If I open the file
    %USERPROFILE%\AppData\Local\Packages\UnityPlayerPrefsReset_pzq3xp76mxafg\LocalState\playerprefs.dat
    before restarting the app I get a file of the same size (180 bytes in
    this case) filled just with 0x00.

    Another interesting observation: If you start the app on Desktop and
    open the playerprefs.dat in parallel (in notepad for example) and press
    save after you changed the string, it correctly appears in the file.
    It's only after you press hardware reset that the file is corrupt. But
    the content was just correctly stored in the file.
     
  6. skaarjslayer

    skaarjslayer

    Joined:
    Oct 10, 2013
    Posts:
    108
    Hey Gerold_Meisinger, thanks for doing some more in-depth testing. Glad to know I'm not the only one with this issue. I don't have anything new to add, though. I'm hoping some of the Unity people may be able to confirm this issue and shed some light.
     
  7. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    We're discussing on the matter internally.

    @Gerold_Meisinger - what is the case number you filled?
     
  8. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    the case number is 855595

    I'll do some testing on general file writes now.
     
    Last edited: Nov 29, 2016
  9. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    Same thing happens if you use File.WriteAllBytes (Windows Store only!). I get a file of the same size with all zeros, even though it was successfully written before reset.
     
  10. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    We kinda got to the conclusion that the corruption part is indeed a bug on the player prefs side. I doubt we can give any guarantees for File.WriteAllBytes, though.

    The old data getting rolled back on Android is expected - if the data doesn't reach the storage drive in time before power gets cut off, there's nothing we can do to preserve new data.
     
  11. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    Can someone please create a simple native Windows Store app which writes a file three times with a different string and check if this is a native Windows error.
     
  12. ModernAlchemist

    ModernAlchemist

    Joined:
    Feb 17, 2014
    Posts:
    9
    Any update on this issue?
     
  13. skaarjslayer

    skaarjslayer

    Joined:
    Oct 10, 2013
    Posts:
    108
    Ticket is still open, last update was early December.
     
  14. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    We have a fix internally, and it's undergoing testing as we speak.
     
    skaarjslayer likes this.
  15. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
     
  16. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    I just tested it in 5.5.1p3 and it's even worse now. PlayerPrefs is already gone when save a string twice. Thus I cannot test if it's gone when rebooting because it's already gone when restarting. I also tested it in 5.5.0p3 and restarting works fine.
     
  17. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
  18. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    *bump*
    Is there an update on the status of this bug?
    5.5.2 is out which states it being fixed, but thats the old "fix", which made it worse, right?

    Thanks.
     
  19. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    The fix in 5.5.1p3 actually fixed it when targeting windows 10 SDK, but windows phone 8.1 used a little bit different path which made it worse indeed. I made another fix which landed to 5.5.2p1 (it's not released yet). Sorry about that.
     
  20. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Thanks for the update!
     
  21. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    @Tautvydas-Zilys: You guys should mention the Windows 8.1 bug in the "known issues" section as it will very likely corrupt all player prefs for developers using the 5.5.1 final(!). This is a critical bug. Btw I reproduced this bug on Windows 8.1 Desktop, not Phone.
     
  22. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    It doesn't actually corrupt them all the time, just when you hard kill the application from the task manager or the debugger. But sure, we'll add a known issue.
     
  23. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    The point is, if the change in 5.5.1.p3 is still integrated in 5.5.1p4 or 5.5.2f1 it actually does corrupt all the time (when saving twice) on Windows 8.1. I don't know if it is still in, but if it is, it's pretty critical.
     
  24. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    Unity 5.5.2p1 Universal 10 Master x64 running on Windows 10 Desktop x64
    tried 7 times
    player prefs: 2x restored old state, 5x worked
    file write: 5x content gone, 2x worked
    *sigh*

    Are you even testing this?

    Update: The rollbacks probably were data wipes, see below
     
    Last edited: Mar 14, 2017
  25. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    That sounds like it's not broken anymore?
     
  26. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    Restored old state means "data rollback to a state that existed before the last 5-10 writes". To consider data persistence to be working I would expect it to be, well, persistent. When the write is through, I expect it to be in the file, period. To me this sounds like a flawed implementation of some kind of journaling functionality.
    File.Write is still broken.

    Update: The rollbacks probably were data wipes, see below
     
    Last edited: Mar 14, 2017
  27. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    Unfortunately, we don't control the disk drive directly and have to rely on windows and filesystem drivers to flush the data. If it decides to do it a second later than you pull the power, the data will not be written. For saving player prefs we now write data to a temporary file, and then use ReplaceFileW function to atomically move it into place. I am not aware of any better methods to do it. At least it doesn't lose the data anymore.

    As for C# File class - that one is implemented by Microsoft and we have no control over it. I don't think it guarantees atomicity at all, though. You could implement your own if you need that functionality.
     
  28. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    The fix is already available for Unity 5.4 ?
     
  29. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    No and it is unlikely to get backported there. 5.4 is pretty much done.
     
  30. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    But this is a critical bug and it should be fixed in Unity 5.4.
    Me (and most likely many other users) can't upgrade my projects to Unity 5.5 as it break the compatibility with all the Unity versions older than 5.5
     
  31. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    I have to revise my test case. The data rollbacks were probably wipes as the program creates a new random string if the file was corrupted, which I forgot.

    Unity 5.5.2p1 Universal 10 Master x64 running on Windows 10 Desktop x64
    tried 7 times: player prefs: 2x wipes, 5x worked

    So the corruption is still there even on Windows 10 (Desktop).

    I retried 3 times saving only player prefs: 3x wipes
    Also tried 3 times using Unity debug export saving only player prefs: 3x wipes. I didn't get the corruption message though.

    > If it decides to do it a second later than you pull the power,
    > the data will not be written.

    The way I tested was: set string, save player prefs, write string down on paper(!), set string, save player prefs, write string down on paper(!), same for file write + write on paper, reboot. So we are talking several seconds here! But timing seems to be a factor as I got 3 out of 3 wipes saving to player prefs only + write down on paper.

    > As for C# File class - that one is implemented by Microsoft and we have no
    > control over it. I don't think it guarantees atomicity at all, though.

    Are you telling me, if I develop a non-Unity UWP application that does file writes I should also expect my files to be corrupted the same way? Seriously?
     
  32. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    Try it. As I said, we don't implement that API - you get the .NET default implementation (unless you use IL2CPP).
     
  33. Wadjey

    Wadjey

    Joined:
    Feb 4, 2015
    Posts:
    244
    And There is no chance to have this fix in Unity 5.4?
     
  34. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,674
    If what Gerold said is true, and the data still disappears, it means that it my fix in 5.5 failed and is no good. Which also means there's no fix to backport to 5.4. To be honest, I have no idea how to fix it at this point: it seems we're already doing everything we can to preserve the data, as mentioned here:

     
  35. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    I'LL - BE - DAMNED!

    non-Unity UWP application: save file -> save file -> reboot => file corrupted
    Notepad++: save file -> change file -> save file -> change file -> reboot => file corrupted
     

    Attached Files:

    Last edited: Mar 14, 2017
  36. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
  37. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    Last edited: Mar 15, 2017
  38. FuzzyQuills

    FuzzyQuills

    Joined:
    Jun 8, 2013
    Posts:
    2,871
  39. Gerold_Meisinger

    Gerold_Meisinger

    Joined:
    Sep 8, 2015
    Posts:
    93
    Are you sure data journaling is activated on your specific Linux installation? I read somewhere it's "unbearably slow in practice". While ext3 and ext4 support full data journaling it may not be (fully) activated on your system.
     
  40. FuzzyQuills

    FuzzyQuills

    Joined:
    Jun 8, 2013
    Posts:
    2,871
    I think my Debian Stretch system may have it active, I've had some odd crashes, and still have everything there. (Or at least, my system partition does)

    That didn't stop certain stability issues caused by a fragmented filesystem though. And from what I gather, journaling of that nature is on by default, unless the mount options are changed. (My data partition may not have it on, as I set it up to be speedy for my steam games)
     
  41. clown_hacke

    clown_hacke

    Joined:
    Jul 29, 2019
    Posts:
    11
    Any updates for this thread..
    I still face issues while switch off the pc directly. I'm building for webgl and when I close the application normally or switch off normally I don't face issues. But when I switch off pc directly my playerprefs data is not saving. The hard part is that, even I update or retry saving another value, it always gives only last selected values before switchoff. I also tried Playerprefs.DeleteAll() still it not deleting my values. Any solutions for that?