Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

MoviePlayer for Unity (plays MJPEG)

Discussion in 'Works In Progress - Archive' started by ShuuGames, Feb 19, 2014.

?

What codec support would you like to see next?

  1. VP9 video + Opus audio (more advanced, royalty free, but less common)

    10.3%
  2. h264 video + aac audio (less advanced, licensing and patent encumbered, but very common)

    64.1%
  3. Both

    23.1%
  4. Some other, please comment

    2.6%
  1. ThePayneX

    ThePayneX

    Joined:
    Jun 12, 2013
    Posts:
    7
    If i check the DrawToScreen option, the video is drawn over the UI. I want to use Unity 4.6 GUI and the AVI Player with the DrawToScreen option on. Is there a way to make the GUI Buttons to appear over the video?
     
  2. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    The DrawToScreen uses Graphics.DrawTexture. Draw order relative to other scene elements can't really be controlled with this. For more complex setup where you have total control over draw order you have a few options:
    • Create a material, assign it to MoviePlayer material property and then set up some geometry in a scene that uses this material. For full screen video that geometry could be a quad on a specific layer and it's drawn by a separate ortho camera that has a culling mask set so that it can only see that quad. Main camera culling mask excludes this specific layer.
    • Create RawImage UI element and write a script that will take the MoviePlayer framebuffer texture and assigns it to RawTexture after the movie has loaded (critical code line: rawImage.textue = moviePlayer.framebuffer)
    This gives me an idea that different draw order scenarios could make a good sample scene.
     
  3. Steva

    Steva

    Joined:
    Sep 9, 2010
    Posts:
    60
    Would it be possible to map a 360° video on the inside of a sphere, for an effect similar to google's street view?
     
  4. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    I'd make a setup with 6 MoviePlayers and assign video framebuffer textures to sky cubemap. Less UV issues than mapping onto a sphere, imho. Then I'd write a script that controls which clips are playing based on camera look direction so that CPU time is used only for videos that you can actually see. How exactly the setup should be done depends on source material and use case. One potential issue I see is that when playing, say, 4 videos at once, the video quality might need to be rather low due to mjpeg codec inefficiencies. Compared to other video playback solutions, with this plugin, keeping all those clips exactly in sync is much easier.
     
  5. sauceypan

    sauceypan

    Joined:
    Nov 21, 2012
    Posts:
    6
    What would be the simplest way to extract a png (or any other image format) from the video at a specific desired frame?
     
  6. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Depends where and how you want to do it. In this plugin you could do any time.
    byte[] pngData = moviePlayer.framebuffer.EncodeToPNG();

    In VLC you can take a snapshot with Shift+S.
     
  7. sauceypan

    sauceypan

    Joined:
    Nov 21, 2012
    Posts:
    6
    Hi thanks for your last reply.
    I'm using a Quad mesh to project the video on. I've noticed that the image colors are slightly different from the original video. The colors appears desaturated when compared to the original video. When "drawing to screen" the problem is resolved. It only appears when drawing onto a material it seems. I've tried numerous shader and material settings to no avail. Would you have any insight on this?
     
  8. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Try a simple picture on the material you're using together with MoviePlayer. If it's unlit shader and there are no post effects on camera (maybe something HDR related) then the pixel values should be exactly as they are on the original texture. The draw to screen uses a just simple unlit shader.
     
  9. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    Thank you for a great asset!!! There are two things I would suggest. First is a sample file using Unity's new GUI system. Second is PlayMaker support. My project uses both of these so I will be doing quite a bit to integrate them.

    Thanks again
    ~V~
     
  10. ZealUnity

    ZealUnity

    Joined:
    Apr 13, 2014
    Posts:
    64
    Exactly how many video formats does this plugin currently support? I need VLC 'play anything' style functionality. Would it be possible to extend this plugin to work with more codecs/formats?

    And how is the performance? Does it utilize any threading?
     
  11. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    How did I miss forum notifications... >_<

    Playmaker action and new GUI integration will be worked on this weekend. If I say so, I motivate myself :D

    It supports only avi container with up to one video and audio stream (mono or stereo). From a script a containerless video or audio stream can be used too. A video stream can be MJPEG, MPNG, RGB555, RGB24 or ARGB32. Audio stream can be 16 pcm le, ulaw or alaw encoded mono or stereo. This is a complete list for now and adding anything more adavanced to it will either break Unity 4 non-pro compatibility (native code plugins) or require a lot of coding effort (re-implementig video codecs in c#).

    It's single threaded and that limits max practical video resolution to 720p on average desktop, it's less on mobile. Again it's a sacrifice made to get maximum portability and reliability.

    I've probably said it before, but I've been wanting to add VP9 video and Opus audio support, that would also be cross platform including webgl. It's a major effor though (native code plugins for multple build target platforms) and would set hard requirement that it's Unity5 only. I'm afraid that the sales of this plugin won't cover the cost (time) that goes into this major coding leap and support work that I'm going to get because there are some old, obscure, unpached or buggy Androids out there. With current non-native code implementation most of the lower level issues and workarounds are dealt with inside Unity engine itself. I see two options:
    * extend this plugin, drop Unity4 support, increase its price and hope it'll still sell and cover all the costs
    * create a separate plugin only for Unity5 and over time phase out current plugin. Might be feasable, but do people want to "pay twice". Can't decide yet, but it feels like a better choice for now
     
  12. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188

    Working on Playmaker support. It's hard to keep it simple yet expose most of the features.
     
  13. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    This is looking awesome!! Any timeframe for an update?
     
  14. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    I am trying to figure out how to get this to work. I have found this code and have tried to get it to work. biggest challenge seems to be getting the framebuffer. I also tried using playMaker to get the framebuffer property from the MoviePlayer script but it appears that I have no way to store it since none of the variable options seem to work.

    Ultimately I need to be able to animate a RawImage GUI element with a movie playing on it.

    Any guidance would be most appreciated.

    Thanks
     
  15. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    "create a separate plugin only for Unity5 and over time phase out current plugin. Might be feasable, but do people want to "pay twice". "
    I would pay twice even for this plugin. And surely, and I would pay twice for another even more performant asset. ;)
     
    Vern_Shurtz likes this.
  16. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    Ditto!!! :)
     
  17. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Thanks for the thougths and support :) The Playmaker stuff is working, but I'm not totally happy with it yet. There is a bit too much complexity and tweakable parameters exposed. The most trivial use case should be doable with one Playmaker action alone. At this point first you load the clip and set up where it will be played, then optionally control the playback (play/stop, seeking, etc) and finally there's a Playmaker action to clean it all up and release resources taken.

    Here's full code that binds framebuffer to RawImage automatically whenever a new clip is loaded.
    Code (csharp):
    1. using UnityEngine;
    2. public class BindFramebufferToRawImage : MonoBehaviour {
    3.    public MoviePlayer moviePlayer;
    4.    public UnityEngine.UI.RawImage rawImage;
    5.  
    6.    void OnEnable () { moviePlayer.OnPlay += HandleOnPlay; }
    7.    void OnDisable() { moviePlayer.OnPlay -= HandleOnPlay; }
    8.  
    9.    void HandleOnPlay (MoviePlayerBase caller) {
    10.      rawImage.texture = moviePlayer.framebuffer;
    11.    }
    12. }
     
  18. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    Thank you very much for the code and working on the playMaker actions. I am not the best coder out there so I use visual scripting quite a bit.

    Thanks again
    ~V~
     
  19. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    Hello, My apologies for so many questions and I greatly thank you for your help with this wonderful asset.

    I have a number of videos to playback and have set up a number of buttons with PlayMakers "Set Property" for MoviePlayer with Source as the property and a different avi for each button in the Source input field. When you click the button it assigns the selected avi file then loads/plays the video. Works great in the editor. On the iPad if I have all of the buttons in the scene with each button assigned a different source it crashes the app with an out of memory error. Apparently it is trying to load all of the avi.byte files assigned in each buttons source block. Probably not the best way to do this. :)

    So I started looking into loading the file from a path and assign it as the source. I would like to keep this within playmaker as much as possible. I noticed in the sample scene image above you are doing exactly what I need to do in PlayMaker. Any chance I could help test your playMaker actions or is there a way I could do this from within playMaker as is? I have a major project to get out the door in less then a week.

    Any help here would be greatly appreciated.

    ~V~

    Edit: I hacked your PlayFromPath script previously posted so I can assign a different path and the call a LoadAvi method to load it. Seems to work. Need to redo all of my buttons and see what happens on the iPad. Bellow is my modified code.

    Code (CSharp):
    1. using UnityEngine;
    2. using System.Collections;
    3.  
    4. public class LoadFromPath : UnityEngine.MonoBehaviour {
    5.     public string path = "";
    6.     void LoadAvi () {
    7.         var mp = GetComponent<MoviePlayer> ();
    8.         mp.Load (path);
    9.     }
    10. }
     
    Last edited: May 7, 2015
  20. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    That's a nice and simple hack.

    Playmaker support is not yet ready, but it's usable. There's no documentation, a few TODOs and some unsolved cases. The action parameter names may change before final release. Maybe some actions will be renamed too. Relying on them now may cause rework later. Upgrading from current version to the latest fresh-from-oven package should work without problems, but it's always good idea to make a backup from the whole project first, just in case. If that didn't scare you away, then I've got a unitypackage to share. Sent you a PM :)
     
  21. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    Thank you very much. :) Will give it a go and share some feedback.
     
  22. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    264
    Here is the code I am using to load movies in the streaming assets folder on iOS.

    Code (CSharp):
    1. using UnityEngine;
    2. using System.Collections;
    3.  
    4. public class LoadFromPath : UnityEngine.MonoBehaviour {
    5.  
    6.     public string MovieSelected = "";
    7.     void LoadAvi () {
    8.         var mp = GetComponent<MoviePlayer> ();
    9.         mp.Load (System.IO.Path.Combine(Application.streamingAssetsPath,MovieSelected));
    10.     }
    11. }
    12.  
     
  23. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    In Playmaker action loading from a file, binary TextAsset, StreamingAssets, Resources folder or over WWW will be all built in. I hope it covers most use cases.
     
  24. Testerr

    Testerr

    Joined:
    Aug 6, 2014
    Posts:
    1
    My team is planning to build a RTMP client for streaming video. Do you think we can improve this plugin to fix our project easily? Is it possible?
    Or can you improve it support RTMP?
    Regards.
    :)
     
  25. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    I don't think adding RTMP to this plugin will be easy. Improving streaming support is not a first priority right now, because better performance and better video codec support is a more pressing topic in Unity 5 (in Unity 4 it was technically not possible). Even before that Playmaker and basic VR/AR support is on its way. Sorry.
     
  26. _W-Man_

    _W-Man_

    Joined:
    Sep 26, 2013
    Posts:
    1
    This looks good and all... But it cost money. Could you make a free version?
     
  27. vzheng

    vzheng

    Joined:
    Dec 4, 2012
    Posts:
    45
    Code (CSharp):
    1.     void Start () {
    2.         path = Application.streamingAssetsPath + "/abc.bytes";
    3.         GetComponent<MoviePlayer> ().Load (path);
    4.         GetComponent<MoviePlayer> ().play = true;
    5.     }
    this my code,and it works fine on iOS,but not on Android...
    is there something wrong?
     
  28. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Sorry, for now a free version is not planned.

    It looks all good at first glance. Can you copy/paste any error messages here (from Android Debug Console maybe)?
     
  29. xenoJiang

    xenoJiang

    Joined:
    Jun 5, 2015
    Posts:
    1
    Can you add a example that work with the new gui system ?
    thank you.

    oh i see that
    "
    • Create RawImage UI element and write a script that will take the MoviePlayer framebuffer texture and assigns it to RawTexture after the movie has loaded (critical code line: rawImage.textue = moviePlayer.framebuffer)
    "

    i wiil try it first , thank you.

    after 2 mintues, it worked fine. thank you.
     
    Last edited: Jun 5, 2015
  30. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    In a next version there'll be some example showing how to use it, but it's really that simple as you found out.
     
  31. Ben-BearFish

    Ben-BearFish

    Joined:
    Sep 6, 2011
    Posts:
    1,204
    @ShuuGames Are we able to use the WWW class to download a .avi file from a server and then play it with your MoviePlayer component with seeking functionality? Will the downloading with WWW from the server work for the WebPlayer and Standalone?

    Also, based on what I read 720p is the max out video can be for performance issues? Because I have a somewhat custom setup where my video resolution is 4096x154. Is this possible with your plugin?

    Finally, although this may be obvious, I wanted to check is scrubbing possible with the seeking in your plugin? Thank you.
     
    Last edited: Jun 16, 2015
  32. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Yes, works in both. Use moviePlayer.Load(www.bytes)

    What really matters is pixel count on one frame. "Use 720p" is an easy way to say "don't go over 1Mpixel". 4096x154 is about 0.6Mpixels so it's fine. There is a tiny file size penalty when width or height are not dividable by 16, but unless you're dealing with very small videos like 17x17 pixels, it's negligible.

    Absolutely, and it works like charm.
     
    Ben-BearFish likes this.
  33. adam1988

    adam1988

    Joined:
    Feb 6, 2015
    Posts:
    3
    Hi! We purchased your product and it's working fine. My question is how can I import the MoviePlayer actions into PlayMaker? I can't see them.
     
  34. Ben-BearFish

    Ben-BearFish

    Joined:
    Sep 6, 2011
    Posts:
    1,204
    Thanks for answering all my questions. I've been trying to grab a .avi file I made and import it into Unity, but everytime I do that I get an "Error while reading movie". Do you know what I need to prevent this error, or is there another workaround for the AVIs required by your plugin?
     
  35. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Currently the version in Asset Store doesn't have Playmaker actions. You could write your own wrappers, but if you can wait a bit, then official Playmaker support will go to Asset Store review later today and then after a few days it'll be available to everybody.

    Are the videos encoded properly with mjpeg and pcm codec? Do they play locally? The error logged into Unity console should be more descriptive and it states the exact reason why it failed to load, maybe you can find that?
     
  36. adam1988

    adam1988

    Joined:
    Feb 6, 2015
    Posts:
    3
    Thanks, we'll wait for it.

    My other question: we tried to bulid the feature demo scene on various devices and a tablet with Intel processor (Lenovo Yoga 2) failed to play the video. Do you have any idea why? All the other devices run it perfectly.

    Thanks in advance.
     
  37. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    I uploaded the version with Playmaker actions to Asset Store yesterday. It takes up to 5 working days to go public. After long thought about how to continue supporting this asset I decided to set it's price to 49 USD. I hope it's fair. It'll change once the new version goes live. Everyone who have purchased it with lower price should get the new upgrade for free.

    Great to hear it works on almost all the devices you tested it on. I don't have a good idea why it doesn't work on Yoga 2, can you get a more detailed error or exception message? Everything that goes wrong in this plugin should be logged into Unity console log.
     
  38. sauceypan

    sauceypan

    Joined:
    Nov 21, 2012
    Posts:
    6
    Hi,

    Is there an easy way to clear the framebuffer?
    I am in a situation where I am constantly seeking at many areas inside the clip and for some reason that eludes me, sometimes upon seeking from one part of the clip to another, I get a very quick flash of the most recently displayed frame.
     
  39. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Interesting. There shouldn't be any glitches if used properly. For the most precise control over arbitrary frame order video playback you should set MoviePlayer play=false and set MoviePlayer script execution order after your playback/seeking script that controls videoFrame (or videoTime). If you set it up as described then after you've set videoFrame, MoviePlayer in its Update method will decode that and upload it into the framebuffer texture. Then that will be either available in a material you assigned it to or it'll be displayed on screen inside OnGUI. But if you keep play=true, then playhead may advance inside MoviePlayer Update as a first thing even if you're driving videoFrame, because videoTime is "stronger". If you need to have play=true, then you should drive videoTime instead. I hope I didn't explain it too confusingly, but you can always look into MoviePlayer.Update() to learn about what and exactly when happens. There are also more aspects to it like audio syncing and frame caching, which may or may not be relevant for you. And finally, for absolute control you can call methods directly on moviePlayer.movie.videoDecoder too :)
     
  40. metalquan

    metalquan

    Joined:
    May 4, 2013
    Posts:
    3
    hello:
    Prompt : this tool depends on ffmpeg. please in instll it and add it to your PATH

    FFMPEG how to install?
    Where is FFMPEG PATH?

    1.png
     
  41. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
  42. EdwinPC

    EdwinPC

    Joined:
    Nov 25, 2014
    Posts:
    6
    I bought your package and it is totally confusing on the installation of the ffmpeg for windows, really confusing and irritating. I will def put 1 start on your package. What do you freaking mean with PATH ? what path? what installer? damn wasted time. I follow all the instruction to set the variables in windows and all that stuff for nothing your thing aint working.
     
  43. Ben-BearFish

    Ben-BearFish

    Joined:
    Sep 6, 2011
    Posts:
    1,204
    I had a question regarding your plugin. I have a project requirement where I need to have a video that scrubs on a mesh wall, but also that the user can draw on like a whiteboard.

    Currently, for the drawing I'm using code to raycast and draw directly on the mesh's texture using texture coordinats as mentioned here:
    http://docs.unity3d.com/ScriptReference/RaycastHit-textureCoord.html

    I was wondering with your plugin, am I still able to get the texture coordinates of the video that's playing on a mesh?
     
  44. MarkD

    MarkD

    Joined:
    Sep 18, 2013
    Posts:
    193
    Hi, I recently bought your Asset.

    First of all, great work, this player is amazing.

    Now to the question. I have some problems with javascript (Unityscript).

    I try the let the player start manually through script, but I keep getting "play does not exist" error.

    script:

    Code (JavaScript):
    1. var loadWaitTime:int;
    2. var level:String;
    3. var LeftCube:Transform;
    4. var RightCube:Transform;
    5.  
    6. function Start () {
    7. yield WaitForSeconds(3);
    8. Application.LoadLevelAdditiveAsync(level);
    9.  
    10. yield(Application.isLoadingLevel);
    11.  
    12.  
    13. if(LeftCube.gameObject.active && RightCube.gameObject.active){
    14. Destroy(transform.gameObject);
    15. // Get a MoviePlayer component and stop the playback
    16.           var audioPlayer = LeftCube.gameObject.GetComponent(AudioSource);
    17.         var moviePlayerL = LeftCube.gameObject.GetComponent("MoviePlayer");
    18.         var moviePlayerR = RightCube.gameObject.GetComponent("MoviePlayer");
    19.      
    20.         if (moviePlayerL != null && moviePlayerR != null) {
    21.                 moviePlayerL.play = true;
    22.                     moviePlayerR.play = true;
    23.                         audioPlayer.Play();
    24.             }
    25.  
    26.  
    27.  
    28. }
    29.  
    30.  
    error:
    BCE0019: 'play' is not a member of 'UnityEngine.Component'.
    BCE0019: 'play' is not a member of 'UnityEngine.Component'.

    Also when using
    Code (JavaScript):
    1. import UnityEngine;
    2. import System.Collections;
    3. import MP;
    4. import MP.AVI;
    5. import System;
    6. import System.IO;
    7. import System.ComponentModel;// for simple threading
    I get error:
    Namespace 'MP' not found, maybe you forgot to add an assembly reference?
    Namespace 'MP.AVI' not found, maybe you forgot to add an assembly reference?

    Am I doing something wrong?

    thx
    Mark
     
    Last edited: Jun 25, 2015
  45. MarkD

    MarkD

    Joined:
    Sep 18, 2013
    Posts:
    193
    I know I am not the author, but can't you use two seperate meshes? one for drawing and one for playing the movie ?
     
  46. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    I'm sorry to hear that it didn't work out for you. Maybe you didn't restart Unity after setting PATH environment variable? Sometimes Windows log out/log in is required too. Ffmpeg is not a hard requirement for this plugin, any tool that can encode videos into mjpeg encoding can be used, any respectable video editing/encoding software can do it. But if you're really unhappy and want refund, then you can contact Unity support, and they can process it (all Asset Store refunds go through Unity support).
     
  47. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Yes, it works exactly like with regular Texture2D.

    Thank you for suggestions. It can be two meshes or it can be one mesh with a material with that combines two textures together, one is video framebuffer, another is whiteboard.
     
    Last edited: Jun 25, 2015
  48. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    The root of the problems you're having is script compilation order related (http://docs.unity3d.com/Manual/ScriptCompileOrderFolders.html). By default Unity compiles Unityscript first, then C#. Therefore you can't see classes, namespaces, etc written in C# in Unityscript files. I discovered this issue after the plugin was released for a while and now it's a bit risky to fix it, because people upgrading it will have to do manual file moving.

    I know 3 options:
    * Don't use "#pragma strict" in javascript that accesses MoviePlayer. This way types are not resolved at compile time, but it happens during run time, when all scripts are already compiled.
    * Move MoviePlayer folder into Standard Assets or Plugins folder. Then it's compiled before any Javascript. This option has downside that whenever an update for this plugin comes, you need to copy the folder over again.
    * Lastly, you could not use Javascript, but this is likely not an option :)
     
  49. Aelcyx

    Aelcyx

    Joined:
    Jun 13, 2014
    Posts:
    12
    Awesome product. I noticed the Readme says that 30 FPS is the optimum quality. Is there any way to get the FPS higher than that?
     
  50. ShuuGames

    ShuuGames

    Joined:
    Jun 25, 2013
    Posts:
    188
    Higher video framerates work just as well as lower framerates. It really doesn't matter much if the video is 10fps, 30fps, 60fps or even 200fps. What really matters is that this plugin must be able to decode one video frame in less time than your monitor vsync interval (usually 1/60th of a second) plus all the stuff your game and Unity will do per render frame. Therefore high video resolution and high monitor refresh rate can be issues, but not video framerate. Due to no-delta-frame encoding, 60fps video files will be 2x larger than 30fps ones.
     
    Aelcyx likes this.