Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Ittermittent crashes on IOS due to Heap corruption

Discussion in 'iOS and tvOS' started by charlesJRG, Aug 7, 2017.

  1. charlesJRG

    charlesJRG

    Joined:
    Jul 18, 2016
    Posts:
    16
    Okay, so this one is causing us to pull our hair out, and causing a poor user experience for about 1/20 of our users. These are only happening on IOS

    Essentially we are seeing random intermittent crashes throughout our game. Most prominently right at start. We are getting a few different stack traces, but they all seem related.

    This is Unity 5.6.2.p2 and Unity 5.6.2.p4

    No pattern of device/OS versions.

    Between our previous release and this release we converted our internal CDN class to utilize UnityWebRequests to grab sprites from our CDN, added retry logic, and moved to add more aggressive Resources.UnloadUnusedAssets. (We aren't using asset bundles in this instance, we are downloading PNGS converting them to textures and applying them).

    Our running theory is that Resources.UnloadUnusedAssets is causing heap corruption, which is causing crashes eventually, but this is a tough one to confirm. I can generate crashes by setting up downloads and repeatedly calling UnloadUnusedAssets, but the stack trace is different, so I'm not sure if it's the exact issue. We can cause images in our app to glitch out with this technique too, which we think is related.

    Is there a known issue with unloadUnusedAssets and UnityWebRequest downloads on IOS? I can make a build demonstrating the problems if that helps.

    Here are the stack traces...

    Code (CSharp):
    1. Crashed: BackgroundWorker
    2. 0  libobjc.A.dylib                0x180250150 objc_msgSend + 16
    3. 1  Lucktastic                     0x1000c6290 WaitOnCondition(UnityWWWConnectionDelegate*) (WWWConnection.mm:127)
    4. 2  Lucktastic                     0x1000c602c UnitySendWWWConnection (WWWConnection.mm:359)
    5. 3  Lucktastic                     0x1010c6558 TransportiPhone::DoRequest(core::basic_string<char, core::StringStorageDefault<char> > const&, unsigned long, UnityWebRequestMethod, core::basic_string<char, core::StringStorageDefault<char> > const&, bool, bool, HeaderHelper*, UploadHandler*, DownloadHandler*, ResponseHelper*, bool) (TransportiPhone.cpp:126)
    6. 4  Lucktastic                     0x1010c66c4 TransportiPhone::DoRequest(core::basic_string<char, core::StringStorageDefault<char> > const&, unsigned long, UnityWebRequestMethod, core::basic_string<char, core::StringStorageDefault<char> > const&, bool, bool, HeaderHelper*, UploadHandler*, DownloadHandler*, ResponseHelper*) (TransportiPhone.cpp:134)
    7. 5  Lucktastic                     0x100d5bd58 UnityWebRequestProto<TransportiPhone, AtomicRefCounter, RedirectHelper, ResponseHelper, DownloadHandler, UploadHandler, HeaderHelper, AsyncOperation>::DoRequest() (WebRequestProto.h:421)
    8. 6  Lucktastic                     0x100d5b884 UnityWebRequestProto<TransportiPhone, AtomicRefCounter, RedirectHelper, ResponseHelper, DownloadHandler, UploadHandler, HeaderHelper, AsyncOperation>::Task_DoRequest(void*) (WebRequestProto.h:125)
    9. 7  Lucktastic                     0x100d5b910 UnityWebRequestProto<TransportiPhone, AtomicRefCounter, RedirectHelper, ResponseHelper, DownloadHandler, UploadHandler, HeaderHelper, AsyncOperation>::Job_ExecuteUnityWebRequest(UnityWebRequestProto<TransportiPhone, AtomicRefCounter, RedirectHelper, ResponseHelper, DownloadHandler, UploadHandler, HeaderHelper, AsyncOperation>*) (dynamic_array.h:369)
    10. 8  Lucktastic                     0x100e87900 JobQueue::Exec(JobInfo*, long long, int) (JobQueue.cpp:355)
    11. 9  Lucktastic                     0x100e8786c JobQueue::Steal(JobGroup*, JobInfo*, long long, int, bool) (JobQueue.cpp:561)
    12. 10 Lucktastic                     0x100e87c44 JobQueue::ExecuteJobFromQueue() (JobQueue.cpp:718)
    13. 11 Lucktastic                     0x100e87cec JobQueue::ProcessJobs(void*) (JobQueue.cpp:767)
    14. 12 Lucktastic                     0x100e872b4 JobQueue::WorkLoop(void*) (JobQueue.cpp:852)
    15. 13 Lucktastic                     0x100fe11c0 Thread::RunThreadWrapper(void*) (Thread.cpp:35)
    16. 14 libsystem_pthread.dylib        0x18089d68c _pthread_body + 240
    17. 15 libsystem_pthread.dylib        0x18089d59c _pthread_body + 282
    18. 16 libsystem_pthread.dylib        0x18089acb4 thread_start + 4
    And this one

    Code (CSharp):
    1.  
    2. Crashed: com.apple.main-thread
    3. 0  libsystem_kernel.dylib         0x189be5014 __pthread_kill + 8
    4. 1  libsystem_pthread.dylib        0x189caf264 pthread_kill + 112
    5. 2  libsystem_c.dylib              0x189b599c4 abort + 140
    6. 3  libsystem_malloc.dylib         0x189c2a9e8 nanozone_default_reader + 330
    7. 4  libsystem_malloc.dylib         0x189c2bc54 _nano_malloc_check_clear + 412
    8. 5  libsystem_malloc.dylib         0x189c2ac38 nano_malloc + 44
    9. 6  libsystem_malloc.dylib         0x189c19664 malloc_zone_malloc + 172
    10. 7  libsystem_malloc.dylib         0x189c1c56c malloc + 32
    11. 8  Lucktastic                     0x100bd2fd8 MemoryManager::LowLevelCAllocate(unsigned long, unsigned long) (MemoryManager.cpp:809)
    12. 9  Lucktastic                     0x100fd2598 bool UnityDefaultAllocator<LowLevelAllocator>::AllocationPage<(RequestType)0>(void const*) const (UnityDefaultAllocator.cpp:205)
    13. 10 Lucktastic                     0x100fd2054 UnityDefaultAllocator<LowLevelAllocator>::RegisterAllocation(void const*) (Mutex.h:30)
    14. 11 Lucktastic                     0x100fd1fc4 UnityDefaultAllocator<LowLevelAllocator>::Allocate(unsigned long, int) (UnityDefaultAllocator.cpp:30)
    15. 12 Lucktastic                     0x100bd203c MemoryManager::Allocate(unsigned long, unsigned long, MemLabelIdentifier, AllocateOptions, char const*, int) (MemoryManager.cpp:1153)
    16. 13 Lucktastic                     0x100f92a14 UnityWebRequestManager::CreateUnityWebRequest() (WebRequestManager.cpp:93)
    17. 14 Lucktastic                     0x100fb6900 UnityWebRequest_CUSTOM_InternalCreate(Il2CppObject*) (WebRequestBindings.gen.cpp:47)
    18. 15 Lucktastic                     0x1007c6a6c UnityWebRequest__ctor_m2378599348 (UnityEngine_UnityEngine_Networking_UnityWebRequest254341728.h:53)
    19. 16 Lucktastic                     0x1007c6cd8 UnityWebRequest_GetTexture_m333882849 (Bulk_UnityEngine_2.cpp:8223)
    20. 17 Lucktastic                     0x1002ff594 U3CExecuteGetTextureU3Ec__Iterator2_MoveNext_m3869864382 (AssemblyU2DCSharp_JumpRampGames_JRGWebRequest_U3CE4115363001.h:65)
    21. 18 Lucktastic                     0x1007d3dd4 SetupCoroutine_InvokeMoveNext_m2975616245 (Bulk_UnityEngine_3.cpp:2915)
    22. 19 Lucktastic                     0x100acebcc RuntimeInvoker_Void_t1841601450_Il2CppObject_IntPtr_t(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:4083)
    23. 20 Lucktastic                     0x1014c27a4 il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (Runtime.cpp:505)
    24. 21 Lucktastic                     0x100f00fec scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (ScriptingApi_Il2Cpp.cpp:216)
    25. 22 Lucktastic                     0x100efa03c ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (ScriptingInvocation.cpp:303)
    26. 23 Lucktastic                     0x100e23cfc Coroutine::InvokeMoveNext(ScriptingExceptionPtr*) (ScriptingInvocation.h:72)
    27. 24 Lucktastic                     0x100e23940 Coroutine::Run(bool*) (Coroutine.cpp:259)
    28. 25 Lucktastic                     0x100fda898 MonoBehaviour::TryCreateAndRunCoroutine(ScriptingObjectPtr, ScriptingMethodPtr, Coroutine**) (MonoBehaviour.cpp:727)
    29. 26 Lucktastic                     0x100fdabbc MonoBehaviour::StartCoroutineManaged2(ScriptingObjectPtr) (MonoBehaviour.cpp:745)
    30. 27 Lucktastic                     0x100fbfd5c MonoBehaviour_CUSTOM_StartCoroutine_Auto_Internal(Il2CppObject*, Il2CppObject*) (MonoBehaviourBindings.gen.cpp:129)
    31. 28 Lucktastic                     0x1002763ec CDN_Download_m1212489378 (Bulk_Assembly-CSharp_1.cpp:12162)
    32. 29 Lucktastic                     0x1002b1b9c DashboardView_ScrollChanged_m115476082 (Bulk_Assembly-CSharp_2.cpp:7979)
    33. 30 Lucktastic                     0x100390418 CreatedMessageSender_SendMessages_m618318556 (Bulk_Atesh.WeNeedCreatedMessage_0.cpp:782)
    34. 31 Lucktastic                     0x100ac9db4 RuntimeInvoker_Void_t1841601450(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:2459)
    35. 32 Lucktastic                     0x1014c27a4 il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (Runtime.cpp:505)
    36. 33 Lucktastic                     0x100f00fec scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (ScriptingApi_Il2Cpp.cpp:216)
    37. 34 Lucktastic                     0x100efa03c ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (ScriptingInvocation.cpp:303)
    38. 35 Lucktastic                     0x100fda578 MonoBehaviour::CallUpdateMethod(int) (MonoBehaviour.cpp:563)
    39. 36 Lucktastic                     0x100d08ba8 void BaseBehaviourManager::CommonUpdate<LateBehaviourManager>() (Behaviour.cpp:209)
    40. 37 Lucktastic                     0x100e0cf70 PlayerLoop() (PlayerLoop.cpp:190)
    41. 38 Lucktastic                     0x101045e98 UnityPlayerLoopImpl(bool) (LibEntryPoint.mm:223)
    42. 39 Lucktastic                     0x10003a5f4 UnityRepaint (UnityAppController+Rendering.mm:271)
    43. 40 Lucktastic                     0x10003a4e0 -[UnityAppController(Rendering) repaintDisplayLink] (UnityAppController+Rendering.mm:78)
    44. 41 QuartzCore                     0x18ddfc5dc CA::Display::DisplayLinkItem::dispatch(unsigned long long) + 44
    45. 42 QuartzCore                     0x18ddfc48c CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 436
    46. 43 IOKit                          0x18ae57b9c IODispatchCalloutFromCFMessage + 372
    47. 44 CoreFoundation                 0x18ab81960 __CFMachPortPerform + 180
    48. 45 CoreFoundation                 0x18ab99ae4 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 56
    49. 46 CoreFoundation                 0x18ab99284 __CFRunLoopDoSource1 + 436
    50. 47 CoreFoundation                 0x18ab96d98 __CFRunLoopRun + 1752
    51. 48 CoreFoundation                 0x18aac6da4 CFRunLoopRunSpecific + 424
    52. 49 GraphicsServices               0x18c530074 GSEventRunModal + 100
    53. 50 UIKit                          0x190d81058 UIApplicationMain + 208
    54. 51 Lucktastic                     0x1000340ac main (main.mm:35)
    55. 52 libdyld.dylib                  0x189ad559c start + 4
    56.  
    57.  
    Code (CSharp):
    1.  
    2. Crashed: com.apple.main-thread
    3. 0  Lucktastic                     0x127d35500 (Missing)
    4. 1  Lucktastic                     0x10032c5e0 U3CDownloadSpriteU3Ec__AnonStorey0_U3CU3Em__0_m4061181975 (GeneratedVirtualInvokers.h:36)
    5. 2  Lucktastic                     0x1003b40d8 U3CExecuteUnityWebRequestU3Ec__Iterator3_MoveNext_m1797967720 (Bulk_Assembly-CSharp_5.cpp:3525)
    6. 3  Lucktastic                     0x10088808c SetupCoroutine_InvokeMoveNext_m2975616245 (Bulk_UnityEngine_3.cpp:2915)
    7. 4  Lucktastic                     0x100b82e8c RuntimeInvoker_Void_t1841601450_Il2CppObject_IntPtr_t(MethodInfo const*, void*, void**) (Il2CppInvokerTable.cpp:4083)
    8. 5  Lucktastic                     0x1015775f4 il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (Runtime.cpp:505)
    9. 6  Lucktastic                     0x100fb5e5c scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (ScriptingApi_Il2Cpp.cpp:216)
    10. 7  Lucktastic                     0x100faeeac ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (ScriptingInvocation.cpp:303)
    11. 8  Lucktastic                     0x100ed8a80 Coroutine::InvokeMoveNext(ScriptingExceptionPtr*) (ScriptingInvocation.h:72)
    12. 9  Lucktastic                     0x100ed86c4 Coroutine::Run(bool*) (Coroutine.cpp:259)
    13. 10 Lucktastic                     0x100eb1e9c AsyncOperation::InvokeCoroutine() (AsyncOperation.cpp:19)
    14. 11 Lucktastic                     0x100d7fc14 UnityWebRequestProto<TransportiPhone, AtomicRefCounter, RedirectHelper, ResponseHelper, DownloadHandler, UploadHandler, HeaderHelper, AsyncOperation>::Job_InvokeCoroutine(AsyncOperation*) (WebRequestProto.h:109)
    15. 12 Lucktastic                     0x100eaa138 BackgroundJobQueue::ExecuteMainThreadJobs() (BackgroundJobQueue.cpp:85)
    16. 13 Lucktastic                     0x100ec1a3c PlayerLoop() (PlayerLoop.cpp:90)
    17. 14 Lucktastic                     0x1010fad68 UnityPlayerLoopImpl(bool) (LibEntryPoint.mm:223)
    18. 15 Lucktastic                     0x1000ee448 UnityRepaint (UnityAppController+Rendering.mm:271)
    19. 16 Lucktastic                     0x1000ee334 -[UnityAppController(Rendering) repaintDisplayLink] (UnityAppController+Rendering.mm:78)
    20. 17 QuartzCore                     0x19081d5dc CA::Display::DisplayLinkItem::dispatch(unsigned long long) + 44
    21. 18 QuartzCore                     0x19081d48c CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 436
    22. 19 IOKit                          0x18d877b9c IODispatchCalloutFromCFMessage + 372
    23. 20 CoreFoundation                 0x18d5a1960 __CFMachPortPerform + 180
    24. 21 CoreFoundation                 0x18d5b9ae4 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 56
    25. 22 CoreFoundation                 0x18d5b9284 __CFRunLoopDoSource1 + 436
    26. 23 CoreFoundation                 0x18d5b6d98 __CFRunLoopRun + 1752
    27. 24 CoreFoundation                 0x18d4e6da4 CFRunLoopRunSpecific + 424
    28. 25 GraphicsServices               0x18ef51074 GSEventRunModal + 100
    29. 26 UIKit                          0x1937a1c9c UIApplicationMain + 208
    30. 27 Lucktastic                     0x1000e80a0 main (main.mm:33)
    31. 28 libdyld.dylib                  0x18c4f559c start + 4
    32.  
    33.  
     
    Claytonious likes this.
  2. raphaelbaldi

    raphaelbaldi

    Joined:
    Mar 4, 2008
    Posts:
    23
    Have you been able to get more information on the error? We've been experiencing something similar, but are yet to find a repro case. Our stack trace reads the following:

    Code (CSharp):
    1. EXC_BAD_ACCESS · Attempted to dereference garbage pointer 0x41a9abeb8.
    2. 0    libobjc.A.dylib    objc_msgSend
    3. 1    CoreFoundation    CFBasicHashAddValue
    4. 2    CoreFoundation    CFRunLoopAddSource
    5. 3    CFNetwork    ClassicURLConnection::pinThisRunLoop(__CFRunLoop*, __CFString const*)
    6. 4    CFNetwork    ClassicURLConnection::scheduleWithRunLoop(__CFRunLoop*, __CFString const*)
    7. 5    CFNetwork    CFURLConnectionScheduleWithRunLoop
    8. 6    CFNetwork    -[NSURLConnectionInternalConnectionscheduleInRunLoop:forMode:]
    9. 7    pepe   WWWConnection.mm:353:9__UnitySendWWWConnection_block_invoke_2
    10. 8    Foundation ___NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__
    11. 9    Foundation    -[NSBlockOperation main]
    12. 10    Foundation    -[__NSOperationInternal _start:]
    13. 11    Foundation    ___NSOQSchedule_f
    14. 12    libdispatch.dylib    __dispatch_client_callout
    15. 13    libdispatch.dylib    __dispatch_queue_serial_drain
    16. 14    libdispatch.dylib    __dispatch_queue_invoke
    17. 15    libdispatch.dylib    __dispatch_root_queue_drain
    18. 16    libdispatch.dylib    __dispatch_worker_thread3
    19. 17    libsystem_pthread.dylib    __pthread_wqthread
     
  3. charlesJRG

    charlesJRG

    Joined:
    Jul 18, 2016
    Posts:
    16
    We seemed to have killed them off, but never found a solid repro.

    1. We rolled back to the old WWW class instead of using the UnityWebRequest
    2. I made sure to properly dispose of each request
    3. I added semaphores around Resources.ReleaseUnusedAssets and the actual request to ensure they couldn't happen at the same time.

    Seems to have done the trick in 99.99% of cases. I still see one error from time to time, but we've gone from about 90% crash free rate to a 99.9% crash free rate.
     
  4. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    900
    @charlesJRG did you ever submit a bug report for this? Since you found a way to make it crash by spamming UnloadUnusedAssets during downloads, could you file a bug report with a repro project that does that attached? We're having a similar problem - probably the same one - but these things don't get fixed without bug reports.

    Anyway thanks for the info you've shared!
     
  5. charlesJRG

    charlesJRG

    Joined:
    Jul 18, 2016
    Posts:
    16
    My plan is to submit one today. We literally just confirmed our findings over the weekend with a fix (which is different than our initial theories), and I didn't want to put in a bug report till I had something more solid confirming this. I'll link this thread once I put it in.
     
  6. raphaelbaldi

    raphaelbaldi

    Joined:
    Mar 4, 2008
    Posts:
    23
    Hey charleJRG, do you mind sharing your findings (and fix)?
     
  7. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    900