Search Unity

Too many method references; max is 65536

Discussion in 'Android' started by Qbit86, May 20, 2015.

  1. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Hi,

    After adding yet another fat Android plugin the build got broken:

    Code (csharp):
    1.     Error building Player: CommandInvokationFailure:
    2.         Unable to convert classes into dex format...
    3.     stderr[    
    4.     trouble writing output: Too many method references: 68573; max is 65536.
    I know about Dalvik limitations and have read related topics.

    How can I solve the problem?

    I need to build the game on continuous integration server, so manual exporting and editing Eclipse project does not suit me.

    There are suggestions over the internet to modify ProGuard build step but I haven't succeeded with this optimization.

    Probably I should strip large jars somehow, but I don't know how. The most fat are (with size in bytes):

    Code (csharp):
    1.     4599100  google-play-services.jar
    2.      995624  android-support-v4.jar
    3.      925432  everyplay.jar
    4.      852733  FlurryAds.jar
    5.      687508  vunglePub.jar
    6.      552394  facebooksdk.jar
    7.      504254  gamecirclesdk.jar
    8.      246509  login-with-amazon-sdk.jar
    9.      215040  AmazonInsights-android-sdk-2.1.26.jar
    10.      147242  unity-ads.jar
    Any concrete suggestions would be very much appreciated. Thanks!

    ----
    Unity 4.6.5p1
    Windows 8.1 Pro
     
  2. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Do you know if ProGuard is supported straight out of Unity ?

    If not, i think a Google Android project is the way to go. If you're building from a CI server, it should be fairly easy to get an APK from an Android project.
     
  3. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    > Do you know if ProGuard is supported straight out of Unity ?

    I don't know the details of build process neither in Unity, nor in Google Android project. I see a lot of `project.properties` files with contradictory lines:
    Code (csharp):
    1. # This file is automatically generated by Android Tools.
    2. # Do not modify this file -- YOUR CHANGES WILL BE ERASED!
    3. ...
    4. # To enable ProGuard to shrink and obfuscate your code, uncomment this (available properties: sdk.dir, user.home):
    5. #proguard.config=${sdk.dir}/tools/proguard/proguard-android.txt:proguard-project.txt
    Uncommenting the last line produced no effect (still 68573 method references).

    >it should be fairly easy to get an APK from an Android project.

    Well, I'm neither aware of build bot settings :) It has been already set up to build working directory with Unity; some custom steps were defined, though. But automatic on-commit exporting to Google Android project, applying custom changes (patches?) to `proguard.config` and compiling .apk — it seems to be far away harder to set up and maintain.
     
    Last edited: May 20, 2015
  4. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Exporting to Google Android Project also fails with the same error.
     
  5. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    Exporting as a Google Android Project should be the way to go. If export fails for some reason please file a bug report so we can fix the issue.
     
  6. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Does the Unity build process for Android generate an intermediate project structure and then proceed to build it automatically? (when building an APK)

    Or do the 2 build processes differ entirely?
     
  7. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    They don't differ entirely but it's works the other way around. We have our internal structure which we are using to build either the APK or the Android Project. It's just the last few steps that differ.
     
    liortal likes this.
  8. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    What i was getting at, is whether the "Google Project" build is guaranteed to succeed, if the APK build succeeds?

    Or, in this particular case - since the dex tool fails to complete, does it mean that the Google Project build is expected to complete successfully ? (since it does not invoke dex)
     
  9. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    yes, the "Google Project" is expected to complete successfully in this case
     
    liortal likes this.
  10. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Export fails for exactly the same reason: “Unable to convert classes into dex format... Too many method references: 68573; max is 65536.”

    I can try manual temporary removing some fat jar to prevent 65K limit overflow, exporting and adding it back to generated project. Where should I add this jar? To some libs/ folder? Is it enough or I need to edit some build scripts? How can I automate this actions for letting them be performed on build server?

    Is the way to go where? Should I try Android Studio or out-of-dated Eclipse plugin? What should I do next — edit ProGuard settings? How? To be honest, I even cannot figure out how to compile apk — “Build” menu in Android Studio does not help.
     
  11. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    When Unity builds an APK it runs tools from the Android toolchain to produce the APK for you.
    One of these tools (Dex) fails due to a limitation in the dex format (limit on the number of methods).

    In this case, it was suggested that "the way to go" (e.g: how you should solve this) is by exporting a "Google Android" project. In this process, Unity only builds a project template for you and you could then customize this before building an APK from it.

    This "customization" could allow you to run tools such as ProGuard that will strip off unused code, reducing the number of methods from your final APK, for example.

    I think you could use either Eclipse or Android Studio, it doesn't matter much.
    Android Studio is the newer one and is the suggested approach to use nowadays (even though i personally still keep Eclipse around as well).

    One you export an android project from Unity i could help you move on with ProGuard and getting an APK.
     
  12. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    It's strange that the same error appears when I try exporting, not building. I expect that all the toolchain should not be invoked, just prepared with generated intermediate project. But it looks like Unity actually runs Dex packing anyway.

    Thanks for clarification! I'm already using similar workflow for building WP8 edition of our game; and my colleagues do the same for iOS via intermediate XCode project. I'm just wondering if there are any ways to keep at least Android simple :)

    The main problem is that I don't have any experience in Android or Java programming (I'm .NET developer), so I'm facing with avalanche of beginner questions about toolchain and basic pipeline, not mentioning specific details like ProGuard settings.

    I'd prefer using the up-to-date tools; just wondering if Unity supports recommended approach.
     
  13. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    When I choose “Import project (Eclipse ADT, Gradle, etc.)” opening Android Studio, it gives the following options:
    Code (csharp):
    1. () Create project from existing source
    2. ( ) Import project from external model
    3.     () Gradle
    4.     ( ) Maven
    Which one should I prefer?

    (One more “thanks” for all participants of the discussion!)
     
  14. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    I think you should choose the first option.
    *If i remember correctly, Unity's project template is pretty dated (i am not using it, but i did try it in the past), so it's not using Gradle or any of the newer project structures that android uses.
     
  15. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Has anyone faced this kind of error error?
    Code (csharp):
    1. Error:ProGuard: [UnityOBBDownloader] java.io.IOException: Can't read [C:\Users\Viktor\.AndroidStudio1.2\system\compile-server\_temp_\proguard_input0.jar] (Can't process class [com/unity3d/plugin/downloader/BuildConfig.class] (Unsupported class version number [52.0] (maximum 51.0, Java 1.7)))
    Looks like incompatible versions of some tool (probably ProGuard itself) between SDK used by Unity and SDK installed with Android Studio. Can't figure out which settings are to be fixed.
     
  16. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    @Qbit86 can you please file bug reports on these two issues. The export problem plus the incompatible class file? Most likely whoever compiled the OBB plugin used Java 8 to do so.
     
  17. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Still not convinced that this is Unity fault, more likely something is wrong with my environment settings. But, of course I can report an issue.

    Plugin worked perfectly for a long time until we tried intermediate project exporting. This is official plugin, not something “home-made”: https://www.assetstore.unity3d.com/en/#!/content/3189
     
  18. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    I know, that's why I would like a bugreport on it ;-) Once you have filed the two bugs, please let me know what the bug numbers are. At least for the export issue it would be nice if you could attach your project so we have a solid way of reproducing the issue.
     
  19. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    I cannot attach our main project because of NDA, but I succeeded reproducing an issue in separate project, attached to the ticket: http://fogbugz.unity3d.com/default.asp?699275_sc6a9suigrjnc5v8
     
  20. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
  21. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    Thank you!
     
  22. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Please let me know if there are some new info on these issues, “confirmed” or “wontfix”.
     
  23. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    @Qbit86, I just got the exported project built in Android Studio after selecting this option (when I select the first one, Java 1.8, I get the same ProGuard version mismatch error). Or just make sure you select Project SDK when importing into Android Studio.

    Android Studio Options.png
     
  24. akasurreal

    akasurreal

    Joined:
    Jul 17, 2009
    Posts:
    442
    Did this never get fixed? I am running into the Too Many Method References, and assumed I could export to Google Android Project, but that still fails with the same error. :( Using 5.2.3f1. I am not sure what to do now, as I can't remove any of the jars from my project.

    Edit: I was able to use a script to remove some pieces from an older google play services jar to barely get this under the 65K limit for now, but there needs to be a better solution. Link: https://medium.com/@rotxed/dex-skys-the-limit-no-65k-methods-is-28e6cb40cf71#.dsya048xb
     
    Last edited: Nov 30, 2015
  25. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    @akasurreal we do have a fix for the issue (during export), it will land in the next 4.6 patch release. It is also planned for 5.2 and 5.3 patch releases.
     
    akasurreal and Qbit86 like this.
  26. mrm83

    mrm83

    Joined:
    Nov 29, 2014
    Posts:
    345
    Getting the same error.. I just imported everyplay and boom.. dexed.
     
  27. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    The dex fix should be available in 4.7 FINAL and 5.3.0p1.
     
  28. siberia

    siberia

    Joined:
    Apr 24, 2013
    Posts:
    3
    Hi, Yury, when can we download the 4.7final or the 5.3.0p1 ?
     
  29. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
  30. siberia

    siberia

    Joined:
    Apr 24, 2013
    Posts:
    3
  31. mrm83

    mrm83

    Joined:
    Nov 29, 2014
    Posts:
    345
    Yup.. it is fixed in the latest build.. however the build will not work with Amazon as they add more stuff to the apk file which will run into the dex problem.

    Mine just got rejected today because of that.
     
  32. mrm83

    mrm83

    Joined:
    Nov 29, 2014
    Posts:
    345
    I added a new library and the dex issue popped up again...
     
  33. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    @mrm83 - the fix was for exporting the project, not for building within the Editor. If you have issues with max methods when exporting the project on latest Unity versions - please let us know. Thanks!
     
  34. Beam-Ali

    Beam-Ali

    Joined:
    Aug 3, 2015
    Posts:
    1
    Hi guys,

    Also running into the dex issue while exporting -- we're still on Unity 5.2. Yury, when can we expect to see a patch release fix for this issue on 5.2?

    Thanks!
     
  35. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    Hi @Beamy!

    Looks like there are no more patch releases planned for 5.2 at the moment, and we missed the train with the dex fix :( Sorry for the bad news.
     
  36. infosekr

    infosekr

    Joined:
    Jul 12, 2013
    Posts:
    46
    Any plans to have Unity strip out unused class files? Have been fighting with Android Studio for days just to successfully build an APK without any luck. Using Unity 5.3.1f1, Android Studio 1.5.1, JDK 1.8.0_66 (also tried JDK 7).

    Here's my current errors if anyone has seen this before:

    "
    UNEXPECTED TOP-LEVEL EXCEPTION:
    Error:com.android.dx.cf.iface.ParseException: bad class file magic (cafebabe) or version (0034.0000)
    at com.android.dx.cf.direct.DirectClassFile.parse0(DirectClassFile.java:472)
    at com.android.dx.cf.direct.DirectClassFile.parse(DirectClassFile.java:406)
    at com.android.dx.cf.direct.DirectClassFile.parseToInterfacesIfNecessary(DirectClassFile.java:388)
    at com.android.dx.cf.direct.DirectClassFile.getMagic(DirectClassFile.java:251)
    at com.android.dx.command.dexer.Main.parseClass(Main.java:764)
    at com.android.dx.command.dexer.Main.access$1500(Main.java:85)
    at com.android.dx.command.dexer.Main$ClassParserTask.call(Main.java:1684)
    at com.android.dx.command.dexer.Main.processClass(Main.java:749)
    ... 12 more
    1 error; aborting
    Error:Execution failed for task ':brainItOn:transformClassesWithDexForDebug'.
    > com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command 'C:\Program Files\Java\jdk1.8.0_66\bin\java.exe'' finished with non-zero exit value 1
    "
     
    Last edited: Jan 5, 2016
  37. cocodrilo555

    cocodrilo555

    Joined:
    Feb 15, 2014
    Posts:
    2
    Hello Yuri, could you please detail your enviroment so we could get your results. I mean Android Studio version, JDK version, android sdk verion and if any of this are 32 bit or 64 bit, and also if you are using Mac or Windows. That could really help, because whatever i install i cannot see the "Project SDK" thing and i'm stuck in the "Unsupported class version number 52.0 maximum 51.0, Java 1.7" error. Thanks in advance.
     
  38. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    @cocodrilo555 : I'm on Windows with all latest release (non-dev) x64 JDK, SDK, NDK and Android Studio.

    I just select "Import eclipse/ADT project" in android studio, then select the main project folder (the one that is named after your app and has the main manifest and Unity libraries), and everything else is done automatically.

    The dialog I mentioned before does not appear for me now as well. But you may try to configure source and target compatibility to 1.7. Or could be some proguard config issues. What is the actual error you are getting?
     
  39. Troutmonkey

    Troutmonkey

    Joined:
    Jun 11, 2014
    Posts:
    7
    Running 5.3.3p3 and just ran into this issue after importing an extra ad provider. Is this supposed to be fixed now?
     
  40. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    Yes the "too many methods" while dexing error should be fixed. There was a number of issues with exporting, but I fixed most of them for the 5.3.4p6 release. Please let me know if you're hitting other errors while exporting. Thanks!
     
    Qbit86 likes this.
  41. bhatiarohan156

    bhatiarohan156

    Joined:
    Feb 23, 2015
    Posts:
    3
    (0034.0000) means your project should use java 7 but your android studio is using java 8 "Files\Java\jdk1.8.0_66\bin\java.exe". Download JDK 7 and set it as your java path.
    In Android Studio goto File->Project Structure->SDK Location(select from the menu on left)->JDK Location.

    This would solve it. It worked for me.
     
    Last edited: May 3, 2016
  42. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    That sounds like a bad idea. In Android Studio, you can set up the desired Java support level if you want.
     
  43. YondernautsGames

    YondernautsGames

    Joined:
    Nov 24, 2014
    Posts:
    353
    Hi there

    I just updated to 5.3.4p6 and I seem to be getting the same error:

    Is the issue only fixed when building a Google Android Project?

    Cheers
     
    Qbit86 likes this.
  44. Qbit86

    Qbit86

    Joined:
    Sep 2, 2013
    Posts:
    487
    Still getting the same error in 5.3.4p6.

    I've parsed Editor log to group by references count. It looks like many entries are duplicates of each other, but sum up separately:
    Code (csharp):
    1. 2029 com.google.android.gms.nearby
    2. 2025 android.support.v4
    3. 2025 android.support.v7.appcompat
    4. 2025 android.support.v7.cardview
    5. 2025 com.facebook.android
    6. 2025 net.prime31.plugins.googlecloudmessaging
    7. 2019 com.everyplay.Everyplay
    In some cases I can change `package` in third party `AndroidManifest`s to be the same, for example `com.unity3d.player`, as suggested here: https://goo.gl/vmDv8y
    But this is not solution in general.
     
  45. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    Dexing is not run only if you are exporting the project. If building in the Editor, you can get the "Too many methods" error. There's no current way to run ProGuard when building with the Editor.
     
  46. loko08

    loko08

    Joined:
    Oct 6, 2013
    Posts:
    57
    Are there any plans to fix this?? We are using several plugins and most of our Artists constantly create builds.

    Having to open an Android Studio project, compile, load and launch would really kills us right now.
     
  47. TalhaDX

    TalhaDX

    Joined:
    Feb 2, 2013
    Posts:
    94
    I am still facing this problem on Unity 4.6.9.

    As we have read above that people are facing it on even Unity 5.3.x as well. What is the solution?
     
  48. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167
    @TalhaDX : What problem are you facing?
     
  49. babaroga

    babaroga

    Joined:
    Nov 24, 2014
    Posts:
    5
    On 5.3.6 we are getting failure to compile an APK due to too many methods. The number of methods has increased (on it's own?) by migration from 5.3.3 to 5.3.6. We have spoken with middleware partners and they claim to have reduced number of methods, not increased them. Is this a known issue?
     
  50. Yury-Habets

    Yury-Habets

    Unity Technologies

    Joined:
    Nov 18, 2013
    Posts:
    1,167