Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

"Rendering to custom java surfaces. Exposed in Unity as additional displays."?

Discussion in 'Android' started by bjonreyes, Apr 13, 2015.

  1. bjonreyes

    bjonreyes

    Joined:
    Feb 1, 2013
    Posts:
    4
  2. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    UnityPlayer now expose a function "boolean displayChanged(int index, Surface surface)". Using this function you can attach your own surface to Unity. If you attach a surface using index 0 you will override the main unity surface. One thing you have to make sure of is to call this function anytime the Surface properties change (i.e. width/height/format). If you want to detach a surface you can call displayChanged with a null argument.

    The c# api for accessing that surface is promoted through the Display api.
    http://docs.unity3d.com/ScriptReference/Display.html
     
    SGA-issei-kuzumaki and bjonreyes like this.
  3. AlexSien

    AlexSien

    Joined:
    Jun 18, 2013
    Posts:
    3
    @bitter:

    Are you 100% sure both your C# API and the Unity 5 promoted feature are link? The C# api you pointed has something to do with the "Rendering to custom java surfaces". This promoted feature works on Android and the C# doc talk about about a feature working on IPhone.

    I am actively looking for some Documentation on this feature, but on the java side, I mean, when the Android native java side of the app.

    I am bumping this topic, because this new Unity 5 feature can be a miracle solution for making Android Live Wallpaper with the Unity Player. Unfortunately, I cannot use the function because I still has an exception trying to initiating the UnityPlayer with the Wallpaper Service,. The error is bellow:

    Process: com.sien.test.androidplayer5, PID: 6445

    java.lang.NullPointerException: Attempt to invoke virtual method 'boolean com.unity3d.player.UnityPlayer.displayChanged(int, android.view.Surface)' on a null object reference

    at com.sien.test.androidplayer5.GLWallpaperService$GLEngine.onSurfaceCreated(GLWallpaperService.java:103)

    at android.service.wallpaper.WallpaperService$Engine.updateSurface(WallpaperService.java:751)

    at android.service.wallpaper.WallpaperService$Engine.attach(WallpaperService.java:881)

    at android.service.wallpaper.WallpaperService$IWallpaperEngineWrapper.executeMessage(WallpaperService.java:1168)

    at com.android.internal.os.HandlerCaller$MyHandler.handleMessage(HandlerCaller.java:37)

    at android.os.Handler.dispatchMessage(Handler.java:102)

    at android.os.Looper.loop(Looper.java:135)

    at android.app.ActivityThread.main(ActivityThread.java:5343)

    at java.lang.reflect.Method.invoke(Native Method)

    at java.lang.reflect.Method.invoke(Method.java:372)

    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:905)

    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:700)
     
  4. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    If I were to guess you are trying to invoke displayChanged() on a null object.

    Code (csharp):
    1. UnityPlayer player = null;
    2. player.displayChanged(0, myawesomesurface);
     
  5. jlemos

    jlemos

    Joined:
    Nov 27, 2015
    Posts:
    8
    In this days I'm trying to attach surface from Unity to Java program, but I don't have any success.

    Below is link to my two projects:
    One project for Unity and other to Eclipse.

    https://mega.nz/#!iZxxhY7J!go4maln7SMHcgLnemzCWMIjBVEQ-oO2rTTC5x3J7pb8

    After that you build and deploy the project, run one time the application and later try to initialize Wallpaper no errors, but nothing is displayed.

    You have to run one time because I'm using the instance of activity to use later on Wallpaper.

    I used the method "displayChanged", but nothing is displayed.

    I've done several tests but got no success yet!

    If you make any changes in the code and the program will display any image of Unity, please share your changes.

    Best regards.
     
  6. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    @bitter
    Is there any chance for documentation on how and when to use UnityPlayer.onDisplayChanged? The Display api docs don't even mention Android.
    What I'm trying to do is to make a live wallpaper. Problem is, live wallpaper application generally has no Activity, just a WallpaperService, and UnityPlayer "UnityPlayer(ContextWrapper)" constructor crashes when the argument is anything other than Activity (i.e. "new UnityPlayer(this.getApplication())"). The error is below:
    Code (CSharp):
    1. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65] JNI DETECTED ERROR IN APPLICATION: java_object == null
    2. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]     in call to GetObjectClass
    3. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]     from void com.unity3d.player.UnityPlayer.nativeFile(java.lang.String)
    4. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65] "main" prio=5 tid=1 Runnable
    5. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]   | group="main" sCount=0 dsCount=0 obj=0x7415a000 self=0xb8cac7b8
    6. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]   | sysTid=29034 nice=0 cgrp=default sched=0/0 handle=0xb6fbabec
    7. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]   | state=R schedstat=( 0 0 0 ) utm=2 stm=9 core=1 HZ=100
    8. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]   | stack=0xbe7b0000-0xbe7b2000 stackSize=8MB
    9. 12-06 07:58:22.857: A/art(29034): art/runtime/check_jni.cc:65]   | held mutexes= "mutator lock"(shared held)
    10. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #00 pc 000047c0  /system/lib/libbacktrace_libc++.so (UnwindCurrent::Unwind(unsigned int, ucontext*)+23)
    11. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #01 pc 00002fc1  /system/lib/libbacktrace_libc++.so (Backtrace::Unwind(unsigned int, ucontext*)+8)
    12. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #02 pc 002464ed  /system/lib/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, int, char const*, art::mirror::ArtMethod*)+68)
    13. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #03 pc 0022aedd  /system/lib/libart.so (art::Thread::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) const+144)
    14. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #04 pc 000b1b2b  /system/lib/libart.so (art::JniAbort(char const*, char const*)+602)
    15. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #05 pc 000b2295  /system/lib/libart.so (art::JniAbortF(char const*, char const*, ...)+60)
    16. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #06 pc 001a8559  /system/lib/libart.so (art::JNI::GetObjectClass(_JNIEnv*, _jobject*)+556)
    17. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #07 pc 000b7011  /system/lib/libart.so (art::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)+48)
    18. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #08 pc 004f7f44  /data/app/com.zimm.est-1/lib/arm/libunity.so (NativeRuntimeException::IsErrorReporterActive()+208)
    19. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #09 pc 004f7358  /data/app/com.zimm.est-1/lib/arm/libunity.so (NativeRuntimeException::install_signal_handlers()+36)
    20. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #10 pc 004f1028  /data/app/com.zimm.est-1/lib/arm/libunity.so (NativeRuntimeException::GetExceptionState()+72)
    21. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #11 pc 004f23c4  /data/app/com.zimm.est-1/lib/arm/libunity.so (nativeFile(_JNIEnv*, _jobject*, _jstring*)+16)
    22. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   native: #12 pc 000076db  /data/dalvik-cache/arm/data@app@com.zimm.est-1@base.apk@classes.dex (Java_com_unity3d_player_UnityPlayer_nativeFile__Ljava_lang_String_2+102)
    23. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   at com.unity3d.player.UnityPlayer.nativeFile(Native method)
    24. 12-06 07:58:22.858: A/art(29034): art/runtime/check_jni.cc:65]   at com.unity3d.player.UnityPlayer.<init>(UnityPlayer.java:140)
    25.  

    This happens on all Unity 5.x versions, from 5.0 to 5.3. I can send a repro project if needed.

    If I use an actual Activity to instantiate the UnityPlayer, and then use UnityPlayer.onDisplayChanged in the live wallpaper service to redirect Unity output, then it kind of works, but extremely buggy, not to mention the UnityPlayer is tied with Activity, and live wallpapers don't even have a main Activity.

    TL;DR: Is there a way to use UnityPlayer for drawing on a custom surface without any Activity?
     
    Last edited: Dec 6, 2015
  7. bitter

    bitter

    Unity Technologies

    Joined:
    Jan 11, 2012
    Posts:
    530
    @ZimM that crash is a bug for sure. It should definitely be possible to run Unity without an activity. But with limited functionality of course. The limitations are not really Unity specific but the same as for any Android application without an Activity. For instance as of 'M' only Activities can be granted dangerous permissions iirc :-(

    Anyway, please file a bug on the crash. I know what the issue is and it should be easy to fix.
     
  8. jlemos

    jlemos

    Joined:
    Nov 27, 2015
    Posts:
    8
    In my example I'm using the Unity Activity in background to instantiate UnityPlayer, but nothing appears on the screen.

    Only black screen.

    Nothing happens, but no errors.

    Any advice or tips?
     
  9. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    @bitter
    I've filled the bug, case #751102.
    Limitations are expected, I'm totally fine with that. The only thing that bothers me is that input most probably won't work without an Activity. But it'd be pretty easy to program some basic input code, so it's not a big deal.
    Also, fun fact: being able to instantiate UnityPlayer without Activity was working fine until Unity 4.1 (or around that), so this is a 2+ years bug that I saw quite a lot of people complain about, and yet seemingly no one before has filed a bug report on it o_O
    So you are instantiating UnityPlayer in an Activity in background and then do... what? You have to force Unity to output to your own surface, and also prohibit Unity from pausing, since by default UnityPlayerActivity stops rendering when Activity is paused (which happens when Activity goes to background). But even if you manage to make that work (somewhat), it'd be hard to impossible to actually use that in a project, since that method is an extremely flawed hack, with no good way to make it work without actually not using the Activity. That's why the fix is required so badly.
     
  10. jlemos

    jlemos

    Joined:
    Nov 27, 2015
    Posts:
    8
    @ZimM
    Thanks for your reply.

    I'm making some testing and trying some possibilities!

    For now nothing works!
     
  11. bjonreyes

    bjonreyes

    Joined:
    Feb 1, 2013
    Posts:
    4
    Source:

    Code (Java):
    1. private SurfaceView findSurfaceView(View v)
    2.     {
    3.         if (v == null) return null;
    4.         else if (v instanceof SurfaceView) return (SurfaceView)v;
    5.         else if (v instanceof ViewGroup) {
    6.             int childCount = ((ViewGroup)v).getChildCount();
    7.             for (int i = 0; i < childCount; i++) {
    8.                 SurfaceView ret = findSurfaceView(((ViewGroup)v).getChildAt(i));
    9.                 if (ret != null) return ret;
    10.             }
    11.         }
    12.         return null;
    13.     }
    14.  
    I thought I could use displayChanged but I instead converted the above to C# so that my C# class with AndroidJavaObject instances can use it.
     
    Last edited: Jul 15, 2016
  12. ZimM

    ZimM

    Joined:
    Dec 24, 2012
    Posts:
    963
    @bitter
    Any update on this one (case #751102)? I've managed to work around this issue with a really ugly hack, but it'd be nice to avoid it, especially if it's an easy fix on your side :)
     
  13. psal1

    psal1

    Joined:
    Jun 21, 2016
    Posts:
    2
    If I Render Unity to my own view,
    would it be possible to Override the OnDraw() method and simulate an on-demand render compared to Unity's continuous render??