MoviePlayer for Unity (plays MJPEG)

@ShuuGames Ok, that sounds good. Would you like me to PM you through the Unity forums or email you directly with the changes I made?

 

Either way is fine, shoot away

 

Hi ShuuGames, just a question: is it possible to update the playlist once the standalone build has been created? (For instance, adding videos or links in an external folder linked to the project).

That would allow my user to update the videos autonomously and according to his needs.

Thanks in advance!

 

Yes, if the videos are in a playable format.

 

@ShuuGames Do you have plans to support more AVI formats as time goes on?

 

Cross platform VP9 video and Opus audio, maybe in MKV container are something that I'd want to add, but it's more work than I've got time these days. Keeping it cross platform is the hardest thing to.

 

@ShuuGames Do you know is there a way to get the video name that is being played when we call Load()? Before I was doing it through accessing the source variable, but now that seems to be null.

 

Since there are many Load() functions that takes different source types, file names and such are not stored. The only exception is when the source is TextAsset. When loading other source types it is easiest to track them yourself. Maybe before calling Load() you could store the source or it's name into a variable to be later used.

 

Ok, yeah I forgot that to call load in the first place I have to know the name of the video. Thanks.

 

I would like to play of multiple AVI in multiple Quad.
Is it possible?

 

Can do. Just be reasonable with video resolution or you'll run into performance issues. Maybe 6 videos, each 512x512, any fps, can play smoothly on a regular desktop. Also 100 videos each 128x128 would probably play well too.

 

@ShuuGames Do you know what's the best way to use your movieplayer with the WWW and WWWForm classes to upload a video?

 

Ooh, so you got some video capture working? If you've already got a file or a memory buffer containing a capture, then uploading it somewhere is a task for Untiy API, this plugin doesn't deal with that. The first thing I'd try to do is use WWWForm like in the scripting reference example http://docs.unity3d.com/ScriptReference/WWWForm.html. For this to work you need server side code too. If WWWForm turns out to be simple or poor api the I'd look into System.Net.Http namespace, which is not so cross platform as WWW.

 

Anyone using 0.11, please update to 0.12 that was just accepted in Asset Store. There was a critical bug that caused many videos to be erroneously treated as interlaced while they were not.

 

@ShuuGames By uploading a video I meant sending a video to the server, but you're right the Unity WWW & WWWForm api was they key.

I had a question for you, when I call Movie.Load() on an external .avi to load, and then play, the texture turns white with a red question mark then plays the video. Do you know what might cause this?

EDIT:
It looks like I accidentally had MoviePlayer CurrentVideoFrame set to an incorrect value in the Unity Inspector. I'll make sure it's always 0 to begin with from now on.

 

It's always a safe practice to not seek beyond the end, but it shouldn't cause red question mark. That question mark basically tells that there IS a frame to decode, but it couldnt decode it.

 

Hi @ShuuGames! I bought the AVI Player plugin, and noticed that when running the FeatureDemo MovieStreamer on the Laptop it works great with very little lag, but on a Android mobile it has a 2 second lag (mjpeg - 720p 25fps). When I play the same stream with chrome browser on the device there is also no lag. Is there a way to reduce the lag with your HttpMjpegStreamer on mobile? cheers!

 

Thank you for purchase. Is the lag when connecting only or when it's playing back? Mjpeg at 720p may be a bit too much for Android, it'll play, but can bring down rendering framerate in general.

 

Hi! Thanks for the quick reply. The lag is when playing back. There is no lag when playing the same stream with Chrome browser.

 

Stream playback in this plugin has no buffering. When the last byte of next video frame is received it will instantly be decoded and pushed into a texture. This is by the way a reason why smooth high frame rate stream playback is not possible, network packets arrive as they please. The only explanation that I can give are that Android OS is doing it's own buffering and delaying network traffic that goes to Unity. It may be due to high CPU load that Unity app is causing. What you could try to do is reduce video resolution and see what happens.

 

ok, thanks. I tried with a 360p stream but it's the same ~2s lag. When I write a java app using libimageproc it works fine. iOS has lag of ~1s, and again no lag in the browser with javascript. So it must be something unity does. Very sad, I understand it's outside of your control. I guess I will have to use xamarin for my app and can't use unity with your plugin for the project. Thanks for the support!

 

If you've not given up yet, then there are a few advanced things that you can try.

* HttpMjpegStreamer.cs line 79: try to increase data receiving thread priority right after thread.Start (url); like this thread.Priority = ThreadPriority.AboveNormal;

* HttpMjpegStreamer.cs line 258: where new BufferedStream is created, add a second constructor parameter that sets buffer size to something smaller than default. Something like this var reader = new BinaryReader (new BufferedStream (stream, 512), new System.Text.ASCIIEncoding ());

HttpMjpegStreamer.cs is the guts of streaming this format, if the lag can be reduced, then that is the place to do it.

 

@ShuuGames I'm trying to play a fullscreen splash screen intro and so far everything seems to be fine with me using a RawImage and loading the video into that. The issue I'm having is that after the video is done playing I'm trying to fade the rawimage to transparent to show the scene, but for some reason the fading goes immediately transparent. Do you have any example where you fade the video to nothing?

 

Did you try changing RawImage color alpha manually in Inspector while the video is playing? That's what should be animated.

 

That did the trick,

I was making a build of my application to Unity's WebPlayer and I noticed in the Windows Task Manager when I'm running Firefox or Internet Explorer when my AVI video loads from the avi.bytes file in the scene the memory consumption on my computer is way higher than it should be for my AVI video.

Specifically, I took a 500 MB video and compressed it with your MJPG to 120MB. Then I put it in my scene. When I look in the Windows Task Manager for how much the WebPlayer tab takes up in my browser, the memory amount says 750MB after the video loads. My scene is super simple memory-wise except for the video I load, so I thought the memory issue might be related to how the WebPlayer handles the video loading.

Might you have any suggestions for what I should look at? Or have there been any issues with WebPlayer with your plugin?

 

Webplayer should work fine and without excessive memory consumption. Can you see how much memory this takes? http://shuugames.com/preview/MoviePlayer.html

How are you downloading and starting playback? Maybe you got "preload video" or "preload audio" load option set which you probably don't want?

 

how do i get a video to play within the confines of a meshs' UV's? Say a Movie/stage screen? Right now it seems stretched out, and whenever i adjust the tiling, and offset, it crashes on me. how do I fix this?

EDIT* never mind lol completely derped it. didn't realize I had to rescale my UV to full 0,1 space lol because i was using this image with 3 separate panels in it lol and then scrolling it. so um yeah..consider this solved. oi vey, long day. great software bub love it.

 

got another issue. my game window sticks to around 30 fps, is there a reason for this? I am using the oculus Dk2 and when I turn off the video on the mesh, it goes back to around 120+ fps. is this tied to your plug in? anyway i can get past this issue so the framerate is not capped at around 30fps?

 

Neither video or rendering framerate is artificially capped. I assume your video resolution is over 720p so it may be a performance issue. I don't recommend using over 720p videos, but some have reported that 1080p works well too, depends on hardware. While common video resolutions are fine, it's best to use power-of-two dimensions like 1024x1024.

 

My team is trying to load IP Camera footage in Unity Web Player. We faced Cross domain problem when using WWW to fetch the IP Camera footage. I was wondering whether we will encounter the same issue if we use your plugin? Another question is the number of footage we can display at the same time if we use your plugin?

 

In web player you can't escape the security sandbox, these policies are there to prevent cross domain attacks. You need access to that remote domain and set up cross domain policy file as described here:
http://docs.unity3d.com/Manual/SecuritySandbox.html

There's no hard limit to how many streams can be played simultaneously. If network bandwidth is not enough, then you'll get lag, disconnects or some streams may just stop. CPU shouldn't become a bottleneck, because these streams usually have low framerate and moderate resolution. I know one instance of this plugin being used in a web player streaming and it played 6 videos simultaneously, setting up cross domain policies was the trickiest part.

To be safe, I let you know that this plugin only plays mjpeg over http. For streaming, only low framerates are feasable and don't go crazy with resolution, 720p max should be fine, 480p is totally safe. I haven't tested, but going over 10 simultaneous streams may start to have various problems.

If you get cross domain policies right, then even the web player feature demo that I have should be able to access and play your stream http://shuugames.com/preview/MoviePlayer.html

 

This ended up being a bug somewhere else. Your example was fine. I had code accidentally loading the video multiple times. Thanks for the suggestion.

 

I had an implementation question for you. How would you recommend doing visual frame scrubbing with your plugin (like YouTube)? Here's an example of what I'm talking about:

My thoughts were to store low quality textures of the preceding and upcoming frames, but I wasn't sure if this would be the best implementation.

 

You're thinking in right direction. You need separate textures for video frames to show them like this. I'd think about some automatic way of creating and destroying them on the fly. I'd call it thumbnail cache. It's not a trivial thing to do, but may be worth it.

If the videos are short and known at build time, then there could be a pre-generated texture atlas containing a selection of low resolution video frames. I'd probably create a simple Unity editor extension that takes video as an input, uses this plugin to decode some frames and then uses Unity builtin PackTextures function that reduces resolution and builds one texture atlas. This thumbnail generation may take minutes for longer video clips, but that would all happen in Editor, later it would be super fast.

The hackiest way is to set up multiple MoviePlayers that all have different videoTime. It may even work. Then care must be taken when to update videoTime for each one of these, because you'll get horrible jitter if all those movie players start decoding next frame at the same time.

 

I do the operations like this,but there still have this mistake,what should I do to solve the problem?Windows7,64Bit.

 

I wrote you an email.

 

Hi,
I have one problem that I don't know how to manage:
I'm doing lipsync by wathching a recorded movie in editor (and using animation by animating parameters). I want to be able to scrub AVI player movie in animation timeline. Now I'm doing it by keyframing first and last animation keyframe, and this works just fine. But a main problem is that this works fine until I'm focused in inspector on a object that holds Movie Player component. If I switch to another object, it stops updating frames. Is there a way to force texture updates if a script isn't in focus?
Also, I've noticed that some parameters gets automatically added in animation timeline (this isn't such a big problem but it would be nice if this doesn't happen).
Thanks!

 

Cool, this is the first time I've heared someone to use it like this.

I'm guessing that the parameter that gets added is videoTime when you're animating videoFrame or vice verse. Can't do anything about it except you can later delete all the animation keyframes that are not needed.

About updating the video while movie player GO is not selected... maybe you can open another Inspector tab, dock it so that it's always visible and then lock it? Haven't tested it, but it might just work for your setup.

 

hi publisher.
i have a big problem.

i do not have any trouble with my converted video colors. but when i add video to your script file , i become dark. why.
i wasted lots of time. but i could not solve this

do you have any idea.

 

Are you using Linear color space (Player Settings -> Other Settings -> Color Space = Linear) and trying to play full screen video? Maybe this is happening in final build only and it's fine in Editor? If so then you'll need to do reverse gamma correction manually. Why this happens is a bit longer topic about how Unity handles color spaces. I can try to elaborate if you're interested.

For this specific case there is "Unlit/Texture ManualGamma" shader included in this package. To use it create a material, choose this shader, set gamma value to 0.455 (that's reverse of the most common gamma value 2.2) and make sure MoviePlayer comonent uses this material. It may happen that video brightness is fine in final build, but in Editor it's over exposed. If that's the case, then shader replacement should be used (different shader for build and editor).

With Color Space = Gamma, everything works automagically.

 

hi admin. Thank you for your help.
but the situation is became so complicated.

I will share pictures.

the picture is below is orginal video . my video have to be like that. (gamma.JPG)




when I change color space to "gamma", everything is good. My video's orginal colors.

but When I change to "linear", video turn dark. the picture name is (linear.JPG)


.
BUT I have to work linear colors. in unity's other scenes. my other shaders need it.

bytheway I couldn't found the shader "Texture ManualGamma" , you'are talking about it.
I will share my unlit shader list picture below.

 

As expected. I've seen this exact same darkening issue before and this is why a gamma correction shader was created.

Earlier versions of this plugin didn't have the shader included. It should be here Assets\MoviePlayer\Shaders\Unlit_Texture_ManualGamma.shader. You could update the plugin (just in case make project backup first) or it may be safer to just copy the shader into your project http://shuugames.com/download/shaders/Unlit_Texture_ManualGamma.shader

After that you can use this shader for video as described before (with 0.455 value) and keep using linear color space. It should work fine. If there are noticable bightness differences between final build and what you see in Editor then please tell your Unity version and what rendering path you're using.

 

thank you sooo much Admin.

god bless you.
my unity version is 5.0.2f1

rendering mod : forward
color space : linear.
beacuse I use Alloy shaders.-----

dear admin. After you shared Shader.my video became better then before..
if you have any question I will answer.

 

Soooo 5.3 with "Async Texture Upload" is out. Have you tested it to see if it has any effect on the performance? I tested the plugin and it works, but don't know about the performance. Although it may not work with your plugin:
http://docs.unity3d.com/Manual/AsyncTextureUpload.html

 

@ShuuGames So I'm using the WWW class to download an AVI as byte array from the server, which works correctly. When I try to load the byte array into the MoviePlayer though I get an error. Might you be able to suggest what would be causing this?

Here's my code:

Code (csharp):

1. MoviePlayer moviePlayer = new MoviePlayer();
2. LoadOptions lo = new LoadOptions();
3. lo.enableExceptionThrow = false;
4. bool loadedCorrectly = moviePlayer.Load(byteArray, lo);
5. if (!loadedCorrectly)
6. {
7.      Debug.LogError("ERROR: Attempted to load  video into MoviePlayer failed!");
8. }

Here's my error:

EDIT:
It looks like the issue is I was trying to create a MoviePlayer with a ctor. I rewrote the code to be like this:

Code (csharp):

1.  
2. public void LoadByteArrayIntoMoviePlayer(MoviePlayer moviePlayer, byte[] byteArray)
3. {
4.       if (moviePlayer != null)
5.       {
6.            LoadOptions lo = new LoadOptions();
7.            lo.enableExceptionThrow = false;
8.            bool loadedCorrectly = moviePlayer.Load(byteArray, lo);
9.            if (!loadedCorrectly)
10.            {
11.                 Debug.LogError("ERROR: Attempted to load incorrect video format into MoviePlayer!");
12.            }
13.        }
14. }
15.  

 

Thanks for pointing this out, but this can't be used here. Uploading to GPU is fortunately only a fraction of the time it takes to decode a video frame, so the benefit would be small even if this could be used, maybe 20% max. (Replying late, because missed the notification, sorry).

 

Yup, Component can't be created with new keyword. MoviePlayer inherits from it, so the same case here. Just replacing the "new" part with this should make it work

Code (csharp):

1. MoviePlayer moviePlayer = AddComponent<MoviePlayer>();

 

Hi SHUU, I am using this and looking for a good way to "skin" the player controls. I looked into the DemoUI, but was unable to find controls for the look of it.

Any tips would be appreciated!

Works great, by the way, thanks!

 

PS, would that be handled using playmaker?

 

DemoUI uses OnGUI to build UI procedurally. Video playback itself is UI-less so in a project where UI controls are needed the buttons and sliders should be hooked to MoviePlayer component booleans, floats and methods. For example Pressing a Play button should set moviePlayer.play=true;. Please see this video using new uGUI and scripting. It doesn't use Playmaker, but the idea is roughly the same when you're using Playmaker.