Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Released] AVPro Video - complete video playback solution

Discussion in 'Assets and Asset Store' started by AndrewRH, Feb 12, 2016.

  1. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi, yes you've identified a difference between our DirectShow and Media Foundation volume level calculations. We've fixed the issue now (thanks to you) and will have a new version available tomorrow (Wednesday).

    Thanks!
     
  2. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi, I'm not sure what you want to be able to do. We don't provide any low-level access to decoders. We simply allow videos to be decoded and play back the video frames and audio.
     
  3. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Glad you found the solution :) Thanks,
     
  4. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I'm afraid ambisonic audio is not supported. We're looking into it :) Thanks,
     
    GuyTidhar likes this.
  5. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I'm afraid ambisonic audio is not supported. We're looking into it :) Thanks,
     
  6. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi,
    I put an AMD GPU in our test machine, but AVPro Video worked just fine. Something I did notice from your log is that there seems to be 2 problems:

    1) The video "mhheart.mp4" appears to be loaded twice - I think this could be the source of your problems. Perhaps you're loading via the component, and also loading via scripting? I'm not sure.. You can see it trying to load the video twice in the log snippet below.

    2) There is an error saying that the mesh renderer is not set - you may have a rogue or incompletely setup ApplyToMesh or ApplyToMaterial script in your scene.

    Here is the snippet:

    Code (csharp):
    1.  
    2. [AVProVideo] Opening C:/Games/MangaGamer/Da Capo 3 R/DC3_Data/StreamingAssets\movies/mgheart.mp4 (offset 0)
    3.  
    4. (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
    5.  
    6. [AVProVideo] No mesh renderer set or found in gameobject
    7.  
    8. (Filename: C:/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
    9.  
    10. [AVProVideo] Opening C:/Games/MangaGamer/Da Capo 3 R/DC3_Data/StreamingAssets\movies/mgheart.mp4 (offset 0)
    11.  
    I couldn't replicate the crash though.. Perhaps you could tell me exactly what happens when you game loads..Is it instantiating prefabs to load the videos or loading scenes with videos in? If you would like to send me a copy of your game then I can test it on this PC here.

    Also - the machines with the crashes - what OS are the running? And is your game 32-bit or 64-bit? What format are your videos? Perhaps you could send me a copy of one for testing?

    Thanks,
     
  7. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi Dave, that is strange - I've always been able to get the display frame rate on Android... We have never tested on a Kindle Fire though... There isn't another way to do this. Quite battled as to why the function isn't working for you..
     
  8. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi All,

    AVPro Video version 1.5.20 has just been released!

    You can find the updated version on the Asset Store and the free trial version / demos on our website.

    Changes include:

    General
    • Added more notes to the Hap codec documentation
    • Added more notes to the Android OES documentation
    Android
    • Fixed fast OES rendering path in Unity 5.4.x and 5.5.x
    • Fixed various stereo rendering issues
    • Fixed OES GLSL shaders failing to compile due to inline keyword
    • Fixed OES path displaying debug GUI panel
    • Fixed OES shader compiling on Google Daydream platform
    • Fixed Unity 5.6beta GearVR stereo rendering issue
    • Disabled minor Daydream stereo eye shader optimisation as it was causing too many problems with older versions of Unity
    Windows
    • Fixed bug where changing playback rate when playback has finished would result in an error
    • Fixed the DirectShow audio volume level so it matches that of Media Foundation
    Thanks,
     
    Last edited: Jan 26, 2017
    TeamZero likes this.
  9. phileday

    phileday

    Joined:
    Feb 8, 2014
    Posts:
    120
    Hi
    Glad to see new updates coming out :)

    I'm currently optimising my project and I'm coming across a GUI.Repaint memory application that is adding to the garbage collection and it seems to be coming from AVPro. At least as soon as I turn it off it goes away. Is this something that can be fixed or do I have to live with it?

    All the best

    Phil

    Edit: I see that there is a way to disable the debug UI but it only works in build. Is that right?
     
    Last edited: Jan 25, 2017
  10. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Yes we have that global option to disable all of the debug IMGUI, but it is only done during builds. This was indeed done to help reduce GC a bit. This option requires a whole script recompile whenever it is changed so I figured it's probably fine to leave in the editor and only use when making a build. If you want to test the GC it's generally better to make a build anyway and attach the Untiy Profiler to it as the editor has a lot of extra overheads. Thanks,
     
  11. rossm

    rossm

    Joined:
    Nov 24, 2016
    Posts:
    3
    Thank you. it worked for me.
     
  12. xpansivevr

    xpansivevr

    Joined:
    Jun 22, 2016
    Posts:
    27
    I still cant get stereoscopic OES rendering to work... the normal android path works fine though.
    system:
    Unity 5.50f3 > AVPro 1.5.20


    Video encode:
    ID : 1
    Format : AVC
    Format/Info : Advanced Video Codec
    Format profile : Baseline@L4.2
    Format settings, CABAC : No
    Format settings, ReFrames : 1 frame
    Codec ID : avc1
    Codec ID/Info : Advanced Video Coding
    Duration : 1 min
    Bit rate : 81.9 Mb/s
    Width : 3 840 pixels
    Height : 2 160 pixels
    Display aspect ratio : 1.001
    Original display aspect ratio : 1.000
    Frame rate mode : Constant
    Frame rate : 30.000 FPS
    Color space : YUV
    Chroma subsampling : 4:2:0
    Bit depth : 8 bits
    Scan type : Progressive

    Any thoughts?
     
    Last edited: Jan 26, 2017
  13. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hmm....it should work. Did you switch the shader on the mesh to also be an OES shader? (there is a shader "AVProVideo/VR/InsideSphere Unlit (stereo) - Android OES ONLY").
     
  14. Balours

    Balours

    Joined:
    Nov 27, 2013
    Posts:
    59
    Just to be sure, I bought the Windows version but I'm actually testing the GearVR, is it normal that I got watermarks on my videos?
    Thanks!
     
  15. xpansivevr

    xpansivevr

    Joined:
    Jun 22, 2016
    Posts:
    27
    Just tried the demo scene and it worked OK. I was using the prefab before.
     
  16. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Yes that's normal. If you bought the Windows package then it will have watermark on non-Windows platforms.
    Thanks,
     
  17. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Ahh okay. Well that sounds like it is resolved. If there is still an issue just let me know the details. Thanks,
     
  18. xpansivevr

    xpansivevr

    Joined:
    Jun 22, 2016
    Posts:
    27
    Hi I'm trying to playback a 360 3840x3840 30fps T/B stereoscopic clip on Android via AVPro on a Samsung Galaxy S7 but just get sound and no image. Has anyone had any success at this resolution?
     
  19. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I believe that resolution is too high for the decoder. It can handle up to around 3840x2304 (8847360 pixels), though most sources recommend no higher than 3840x1920.
     
  20. ZealUnity

    ZealUnity

    Joined:
    Apr 13, 2014
    Posts:
    64
    Any examples for playing 180 degree stereo videos? They seem pretty common, but the only example in the project is for a sphere and cube...

    Are the UVs generated based on the positions of the vertices? So would it be a simple matter of cutting a sphere mesh in half (to make a dome)?
     
  21. rossm

    rossm

    Joined:
    Nov 24, 2016
    Posts:
    3
    Hi Andrew,

    I have a another question. is there a way to call OpenVideoFromFile in another thread to be able to preload the video without the little jerk on screen. From my understanding, OpenVideoFromFile is a little blocking.

    I tried to call OpenVideoFromFile from another thread but there is a lot of call that require the main thread. is there a way to expose only the loading of the video file in a non-main thread-friendly function without too much changes?

    thank you and have a nice day.
     
  22. alex_evp

    alex_evp

    Joined:
    Jan 29, 2017
    Posts:
    7
    Hi there,

    with AVPRO for windows is it possible to use a list of videos or just call *.mp4 and have a next video / last video control to trigger moving through a list of videos? or is it just for loading single videos 1 at a time and make a ui element for each video?

    thanks,

    Alex
     
  23. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Would love to create an example of 180 degree playback, but I'm not sure what format those videos are in? 360 videos are in the equirectangular format, but 180 degree videos are not.... Any ideas?

    The UV coordinates on the sphere mesh are already in the mesh, but yes cutting it in half (either directly on the mesh or in a shader using clip() function) would leave you with half of the video texture frame being drawing - but again this shader is for equirectangular videos so I'm not sure this would look correct..
     
  24. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I don't think it is possible to use another Thread, unless you used it for ALL communication with the plugin and didn't use any Unity functions - it would be tricky. In general the call to OpenVideoFromFile should be asynchronous, but yes in some cases there can be some slight blocking. If you're running on Windows and using the DirectShow video API then it will block until the video has loaded - this is the worst case. If you're using the Media Foundation video API for Windows then it should be async, just like on all other platforms. We have noticed though that there can be a slight cause when calling the function with more complexly encoded videos, so you may be able to remove this little "jerk" by encoding your videos more lightly (less reference frames, no CABAC, "fast start" atom ordering etc).
     
  25. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Yes of course you can use a list of videos, but this is something you would have to script yourself. For example you can use System.IO.Directory.GetFiles() method to get all mp4 files in a folder (mobile platforms such as Android may require a different method to access the file system) and then pass the path to the file to the MediaPlayer via the method OpenVideoFromFile(). Take a look at some of the sample scenes - they show loading of multiple videos using the same MediaPlayer and UI element.
     
  26. JoshWPro

    JoshWPro

    Joined:
    Jan 14, 2016
    Posts:
    2
    Hi everyone,

    I´m considering this product, but I have a question. How good is the performance while streaming atleast 4k videos, regarding an almost empty Unity scene? In other words, how much does the fps dip while streaming a video on a mesh?I used an other script which diped the performance from 100 fps (Unity stats) to 50 fps, which is not good for a VR Scene.is this script working better ?
     
  27. ZealUnity

    ZealUnity

    Joined:
    Apr 13, 2014
    Posts:
    64
    They are usually just two side by side 180 fish eyes, (resolutions vary from 2880x1440 to 3400x1800). So just projecting them onto a 180 dome should work?
     
  28. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi, you can download the free trial version of AVPro Video from our website if you would like to test it for yourself (it's very easy, you can just use the sample scenes and point them at your own 4K video files).

    For Windows 8 and above you should get really excellent performance with a 4K H.264 video, since it's all handled by the GPU decoder (suitable GPU required). Unity should be able to run at hundreds of FPS while displaying the 4K video. For mobile platforms 60 fps is possible.

    Thanks,
     
  29. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    It's not as simple as that. If you connect the two 180 images together it doesn't form the equirectangular mapping that the shader expects. Perhaps you could post a sample 180 degree image here - maybe someone will have a suggestion about the transform needed. I suspect we may have to convert from a fullscreen fisheye to spherical.
     
  30. alex_evp

    alex_evp

    Joined:
    Jan 29, 2017
    Posts:
    7
    Hi there,

    thanks for the answer, I am quite a beginner with c sharp, I saw the example and tried to modify it to my own needs but couldn't get that working. I saw the gui buttons trigger "video 1.mp4" & "video 2.mp4".. I just wanted those controls on keyboard press say 1 for video 1 and 2 for video 2 instead of gui buttons. I tried adding an if statement for keyboard input in the void Update section but it wouldn't work. I guess I need to learn more about programming really, I saw you posted a script here earlier making a random video play from a list, perhaps that could be modified? (page 5 of this thread)
    I will keep trying (painfully :)
     
  31. MESHED-space

    MESHED-space

    Joined:
    Nov 11, 2015
    Posts:
    22
    Hi, my problem is, that videos which are captured horizontal upside down, or in portrait mode, will not be rotated correctly by AVPro within an iOS device

    Is there an option to turn this feature on, or to check if the video is rotated ?

    Hardware: iPad Air 1
    iOS Version: 10.2
    AVPro Version: 1.5.20
    Unity: 5.4.4

    Thank you.
     
  32. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi,
    This is a feature we plan to add in the next update of the plugin - so very soon :)
    Thanks,
     
  33. MESHED-space

    MESHED-space

    Joined:
    Nov 11, 2015
    Posts:
    22
    Perfect, thanks a lot !
     
  34. xpansivevr

    xpansivevr

    Joined:
    Jun 22, 2016
    Posts:
    27
    Why would Samsung VR palyback a video and AVP not? I have a 3840x2304 T/B example that's fine on gearvr via Samsung VR but refuses to show an image via AVPro, I do get sound though. Balck screen via OES and White screen without. Mobile: Note 5.
     
  35. Balours

    Balours

    Joined:
    Nov 27, 2013
    Posts:
    59
    Hi,

    I've read in the documentation that it is possible to read HLS stream from Youtube using some obscure ways, I'm planning to use my own videos but I can't find a proper way to get the HLS stream /.mp4 access. And no, I don't want to use the source videos directly because the application will be too heavy. Thanks a lot!
     
  36. nbaris

    nbaris

    Joined:
    Jan 13, 2015
    Posts:
    27
    Hi,

    I sometimes see this warning when running on an iPhone:

    AVAudioSession.mm:1053:-[AVAudioSession setActive:withOptions:error:]: Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.

    This seems like it might be related to the issue I'd mentioned before (quoted below). Is it normal that I see this warning? Do you have any advice about getting rid of it?


    Thanks



     
  37. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Interesting! Yes I've heard reports of Samsungs VR software being able to play back high resolution videos. I suspect they're somehow tapping in to some lower-level capabilities of the hardware (they are the manufacturer after all). Does the video file play if you just use the normal built-in media player on Android?
     
  38. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    You would have to use a tool that extracts the HLS / MP4 from the contents at the YouTube URL. Eg an asset like this one has a GetVideo script which could be used. There are also several conserations online about how to extract this data. This isn't something I've tried though. Another option is to use the files on amazon S3 storage...
     
    Balours likes this.
  39. ZealUnity

    ZealUnity

    Joined:
    Apr 13, 2014
    Posts:
    64
    There are lots of sbs 180 videos on youtube



    All the VR video players (Virtual Desktop, MaxVR, Whirligig, etc...) usually offer some form of '180 dome' or 'spherical' projection. Not sure how the exact uv mapping is calculated (hence the question), but these VR players have figured out a universal way that works with most videos.

    *Remember also that the projection is one size fits all for all 180 videos, so the black borders don't really matter. If the video has borders, it will just show up as black around the edge of the dome.
     
    Last edited: Feb 3, 2017
  40. MESHED-space

    MESHED-space

    Joined:
    Nov 11, 2015
    Posts:
    22
    Hi,

    MESHED-space-GmbH said:
    Hi, my problem is, that videos which are captured horizontal upside down, or in portrait mode, will not be rotated correctly by AVPro within an iOS device

    Thank you.
    Hi,
    This is a feature we plan to add in the next update of the plugin - so very soon :)


    --> our customer is asking about this , can you estimate when this feature will be released ( flipped movie issue) ?

    thanks,
    Ulf Marsche
    MESHED space
     
  41. huxley

    huxley

    Joined:
    Apr 27, 2009
    Posts:
    334
    I have a 3D video that I shot with alpha to use as a billboard in VR. When I use your "AVProVideo/VR/InsideSphere Unlit (stereo)" shader on my quad, I only get one side of the stereo shown on the billboard. I can see the other side shown when move over the halfway point on the quad, but again it only displays one side of the video in both lenses.

    I assume that this stereo shader can only be used on a sphere, but are there any other settings that will allow me to isolate each side of the video for the proper lens in my HMD?
     
  42. Tom-Goethals

    Tom-Goethals

    Joined:
    Mar 6, 2015
    Posts:
    102
    Not behind a PC to check exactly where, but either the Mediaplayer or the material or the applyToMaterial Monobehaviour has a property where you have to define the camera. (drag it on), that should fix it.
     
  43. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I spoke to our iOS guy about this. He said we don't touch the audio session in our plugin and that that warning is caused by the audio session being changed while video playback is happening. In the past we've see that Unity's Microphone implementation seems to affect the AudioSession stuff in a bad way... Other than that I'm not sure.
     
  44. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Thanks for the example. Well our plugin is more of just a basic video player - any mappings we supply are just a bonus :) We'll have to investigate this in the near future to see if we can add a new sample that covers this. Maybe I can ask the Whirligig author to see if he can shed some light on it :)
     
  45. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi Ulf, we hope to have a new release of the plugin later this week and it should include support for this.

    Thanks,
     
  46. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Did you add the UpdateStereoMaterial component? This needs to be added and have the camera and meshrenderer set correctly so that i can update them for stereo rendering.
     
  47. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    I think you mean the UpdateStereoMaterial component :) Thanks!
     
    Tom-Goethals likes this.
  48. HJPL

    HJPL

    Joined:
    Dec 6, 2013
    Posts:
    12
    few quick questions: Can you...

    1. jump to a frame of a loaded movie via a script?
    2. play a loaded movie in reverse (and change it's reverse speed) via a script?

    Thanks :)
     
  49. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    Hi! Yes you can do both of those things, but the devil is in the details :)

    1) You can just to any point in time in the video, but the API function to do so jumps to a millisecond, not a frame. You could convert from frames to milliseconds though if you video has a constant frame rate and you know the frame-rate. Also the speed at which it can jump to any frame will depend on how the video is encoded. Videos without many key-frames will take longer to seek to a random frame.

    2) You can do this via 2 methods: change the playback rate to negative, or pause the video and seek through it backwards. The first option of changing the playback rate isn't supported on all platforms (such as Android) and the quality of the reverse playback greatly depends on how your video is encoded. The video that doesn't have many key-frames and uses B-frames will play backwards very badly. We have encoded our sample video BigBuckBunny720p30.mp4 with many key-frames as an example of how well it can work. Audio is also not guareteed to play backwards - again depending on platform and codec used. The second option is to manually scrub through the vidoe backwards. Again this relies on the video being encoded to suite this use case, but it will work on all platforms - but will not play any audio.

    I hope that answers your questions. I think you might find it easiest to download the trial version and test it for yourself. Thanks,
     
  50. huxley

    huxley

    Joined:
    Apr 27, 2009
    Posts:
    334
    the UpdateStereoMaterial component fixed the stereo issue I was having, thanks for the quick reply!