[Released] AVPro Video - complete video playback solution

Hi Andrew,

I've got a feature request that would take a whole lot of text and images to explain, so i made a slide deck.
it's about optimizing texture space in a 360 stereo image/movie so there's more resolution and stereo in the front and less pixels are wasted on stereo and pixels in the back. (where non of the action is)

https://docs.google.com/presentation/d/1bpMV6jnWfForXV1ZrsakY272TCbod9H8JBkiAzmtO-g/edit?usp=sharing

The actual question/request is on slide 13 and reads:
Would it be possible to have a shader and script combination where you can select the UV channel used per Eye in VR.
I would be able to provide the geometry/UV maps and test images if needed. And understand this probably falls outside of the scope of normal support.

 

Hi Tom,
Well this is the first time I've had a presentation sent for a feature request! Very cool
Yes I'm happy to help out with this idea. I'll quickly make a rough version of the shader and email it to you. Happy to continue this on email thread.
Thanks,

 

Hi,

I'm trying to get the AVPro Video plugin working on mac, currently testing with the provided uGUI demo scene. On windows, it works perfectly. On mac, however, the video won't play. It shows only the first frame, and unity freezes.
AVPro v. 1.5.6
Unity v. 5.4.0
OS X 10.8.5

Attached the Player.log

 

Hi,

I'm having an issue with this plugin on iOS/OSX. The FinishedPlaying event never seems to fire, but it works fine on Windows/Android. I've added some logging to the OSXMediaPlayer and it seems like AVPPlayerGetStatus never returns Finished, it jumps from Playing to ReadyToPlay when a video ends.

Tested with:
Unity 5.4.2f2
AVPro 1.5.6

 

Hi,

I'm trying out the latest AVPro trial package on Unity 5.4.2p1 on MacOS. However, very frequently the videos fail to load both on my project, as well as in your demos. It is really inconsistent and no errors are thrown. Do you have any idea what could be the issue?

 

Hi Jelle, Thanks for the log file and versions etc.
The reason it's failing to load is that AVPro Video doesn't support OS X 10.8.5 and currently requires 10.10 as a minimum. You should be able to upgrade to a newer OS X version, but we would like to fix this issue so it works for 10.8.5. I have emailed you about this, so perhaps we can send you a new build to test?
Thanks,

 

Hi,
Yes we're aware of this and are currently working on a new version to fix this. The fix should be out early next week.
Thanks,

 

Hi, I think this is related to an issue that we know about and that we're currently working on fixing. But could you tell me:
1) AVPro Video plugin version
2) OS X version

Thanks,

 

@AndrewRH Thanks!
1) 1.5.4t (scripts v1.5.6)
2) 10.11.6 El Capitan

 

@AndrewRH

Hey Andrew, does RenderHeads offer any form of educational license for non-commercial products? I'm a student, and I've been thoroughly impressed by the AVPro Video free-trial. Unfortunately, the $450 price-tag seems marketed to industry professionals and is cost-prohibitive for our purposes. My friends and I think we could build some seriously cool software experiences using the plugin. Please let me know if you can help us out, or if you might be able to direct us to someone who we could talk to regarding educational/academic licensing.

Thanks so much.

 

Hi @AndrewRH , im wondering if its possible to play videos backwards ?!

 

Hi @AndrewRH, I'm been trying out AVPro with the Google Daydream with no success. Have you or your team given it a try? Thanks!

 

Hi Andrew,

Would it be possible to provide a bit more of a fallback system for the OES path. If a device does not support it, fallback to the reqular shader/rendering path or something.

We're in the process of submitting the app to the Oculus store and they're getting back with these screens. (See attachment)

The issue where OES will fail on some devices is the cause i think.
Thanks for looking in to it already!

 

It may be possible.. Unfortunately the Unity Asset Store doesn't yet have this feature but perhaps we could work something out. Please email unitysupport@renderheads.com so we can discuss it.
Thanks,

 

Yes and No Most platforms won't really support this directly, but there are usually two methods you can use:

1) Set playback rate to -1. This method isn't very well supported at all. You you are only targeting one platform then it may be worth exploring, especially if it's Windows. This method also requires the use of a codec that doesn't use B-frames compression.
2) Pause the video and scrub it manually using Seek(). This method is totally manual and one issue is that you won't get audio playback. Again the way the video is compressed affects the performance a lot. Ideally the video should contain no B-frames, and no P-frames.

A codec like Hap works well for both of the above cases. It's best to just try the trial version and see how it works. Let me know if you have any other related questions.

Thanks,

 

Yes we are working on Daydream support. Daydream support in Unity is still in beta. Perhaps you could tell us more detail about the type of problem you're having with it? We've had it working but just with a low frame rate.
Thanks,

 

Hi Tom,
I'm not sure if this helps, but when we submitted to Oculus we were building with Unity 5.3.6.
The OES shaders do have a fallback path defined at the bottom of the shader file. It should fall back to the standard non-OES shader... No?
I haven't seen OES fail on a device before...that is strange!

 

The shader does indeed has the fallback, but the checkbox for android... don't know, there probably is no way for you to see if it's failing and then turn of the checkbox? Maybe it's something else, i'm guessing i need to find someone with an S7. i'll try building with 5.3.6 first. Thanks!

 

hi Andrew,

I am having a small problem in iOS build. In the build the 360 video and the cardboard scene, the video is flipped upside down, is there any solution for this problem.
i am using AVPro version 1.5.2

 

Hi, is there a way to change the layer of the video so that certain objects will not draw in front of it? I'm using UGUI

 

Hello,
bought "AVRPro Video" and I need to make able to play stereo 180 video.
How can I do it?

 

I see what you mean...which Android model + OS does it fail on? S7 should be fine I would have thought!

 

I'm sure AVPro Video plays back the video, but you're looking for a shader that will display it correctly?
AVPro Video doesn't include any shaders for stereo 180 degree videos, but if you send us your sample video perhaps we can add support for this. Please email unitysupport@renderheads.com or PM me here.
Thanks,

 

This is a Unity specific matter and outside of the scope of AVPro Video. We include a uGUI component that is no different to any other uGUI components (text, image, button etc). I'm not sure whether what you're asking for is possible with uGUI. uGUI lets you specify the order in which things render via hierarchy and Z depth, so you may be able to configure it with these. Alternatively you could use the uGUI mask component perhaps. You could also try assigning the UI to another camera and set the Depth value of that camera so that it renders last. Or if you switch to the IMGUI component you can always make sure your video is on top because IMGUI is always the last thing rendered. I hope these suggestions give you some ideas.
Thanks,

 

We're looking into this. Which version of cardboard and Unity are you using?
Thanks,

 

Hi All,

AVPro Video version 1.5.7 has just been released!

You can find the updated version on the Asset Store and the free trial version / demos on our website.

Changes include:

Android

• Minor optimisation for non-OES rendering path by eliminating blending during blit
• Added frame timestamp support via TextureProducer.GetTextureTimeStamp()

macOS, iOS and tvOS

• Added support for OS X 10.8
• Added frame timestamp support via TextureProducer.GetTextureTimeStamp()
• Fixed issue when Seek and Play were called before another command had completed
• Fixed issue of events not firing
• Added support for custom URL schemes
• Improved support for device simulator
• Various small bug fixes

Thanks,

 

I tried adding AVPro to a sample DayDream scene. When I run in it in editor, it works fine. However, when I try to build and run it, the load screen (where it says "Made in Unity") lasts a good 10 seconds before loading into the scene and then no video is renderered on the GameObject.

 

googleVR verison 1.0 and unity version 5.4.0

 

Do you have the component SphereDemo included in your scene? This component could be interfering with the rotation and it should be removed.
I loaded the GVR "HeadSet" demo, put in the "360SphereVideo" prefab, set the video filename, and it all played perfectly.
Thanks,

 

i have the sphere demo script only in the 360 video scene not the cardboard scene.
it works fine in the editor and android the issue happen on iOS when you load video from path (gallery)

 

Is the video in the gallery the same video or a different one? Perhaps you could send us a copy of the video? Was the video captured with the phone? If it was then it may have rotation/orientation metadata which isn't currently read.
Thanks,

 

yes the video captured using the iPhone 6s, we tried the same build with iPhone 7 it worked fine.
how can i fix this issue (metadata).

 

Hi,

While testing out the free trial version, UWP apps (needed for HoloLens) crash both on HoloLens and on the Windows 10 machine. On the PC, it shows a white screen for a few seconds, then crashes (tried multiple scenes). Using Unity 5.5b11, latest non-beta W10 build (Anniversary Update, 1607). Everything is cool in the editor and when creating a standalone .exe.

Event log doesn't help much:
Faulting application name: VideoTest.exe, version: 0.0.0.0, time stamp: 0x56c6997e
Faulting module name: UnityPlayer.dll, version: 5.5.0.0, time stamp: 0x5819b92e
Exception code: 0xc0000005
Fault offset: 0x003e4a55
Faulting process id: 0x2c760
Faulting application start time: 0x01d2419740e7e816
Faulting application path: D:\Work\Projects\360World\VideoTest\App\VideoTest\bin\x86\Master\AppX\VideoTest.exe
Faulting module path: D:\Work\Projects\360World\VideoTest\App\VideoTest\bin\x86\Master\AppX\UnityPlayer.dll
Report Id: 0c214dfb-63b6-4172-9143-5db33c835b41
Faulting package full name: VideoTest_1.0.0.0_x86__pzq3xp76mxafg
Faulting package-relative application ID: App

Attaching a debugger doesn't help either.

The situation is the same even if the scene included in the build doesn't contain anything AVProVideo related (was running fine before I performed an Import Package for AVProVideo). Please help.

 

Hi, i've read about some people having problems with daydream/googlevr.
I have a project i made for gearvr and i would like to convert it to the daydream. I have a problem with the video, like it is "flickering". So i've tried just the demo of the plugin itself, and keep having this flickering problem on the daydream.
I'm using Unity 5.4.2f2-GVR12, and with the Demoscenes i still having the flickering (on a PixelXl phone with daydream view headset)

 

HI :
We are investigating to see if we can live stream hd video on android.
I can use the Http play video. But RTSP can't. This address RTSP://.../test1.sdp or RTSP://.../test1 (...is number ip)
help. What should I use

 

Hi alaa-mohamed,
AVPro Video doesn't currently support reading the orientation field from videos. It may be that iPhone 7 alters the actual video instead of changing the orientation field metadata, and this is why you see the different behaviour. We do plan to add support for orientated videos at some point soon. For now you may have to just have an option to manually rotate the video during display.
Thanks,

 

Interesting! Obviously Hololens support in Unity is still in beta so it could be a bug in Unity. Did you try disabling hardware decoding (in MediaPlayer > PlatformSpecific > UWP)?
We'll investigate this further and get back to you.

 

Yes, we've had reports about this. There is definitely still an issue with Daydream and our plugin....We're trying to get in contact with Unity/Google to try to resolve this. Hopefully the issue will be resolved soon. Thanks,

 

Perhaps try with lower-case "rtsp://". If it still doesn't work, perhaps you can try sending us your URL so we can test it? Thanks,

 

Hello,
I'm facing an issue that app crashes when I'm running build.
AVPro Video version 1.5.5t (scripts v1.5.4b)

Here is an output error:

Spoiler: error.log

Unity Player [version: Unity 5.4.1f1_649f48bbbf0f]

nvwgf2um.dll caused an Access Violation (0xc0000005)
in module nvwgf2um.dll at 0023:6a432637.

Error occurred at 2016-11-18_164312.
E:\Projekty\Expo\Expo\Build\expo.exe, run by danny.
69% memory in use.
0 MB physical memory [2469 MB free].
0 MB paging file [235 MB free].
0 MB user address space [3312 MB free].
Read from location 0000000c caused an access violation.

Context:
EDI: 0x12277398 ESI: 0x00000000 EAX: 0x072c9500
EBX: 0x00000000 ECX: 0x12277398 EDX: 0x072c9500
EIP: 0x6a432637 EBP: 0x06f9fad0 SegCs: 0x00000023
EFlags: 0x00010206 ESP: 0x06f9fac8 SegSs: 0x0000002b

Bytes at CS:EIP:
8b 46 0c 83 78 54 00 0f 95 c1 32 4f 10 80 e1 01


Stack:
0x06f9fac8: 072c9500 12277398 06f9faf8 6a4329c1 ..,..s'......)Cj
0x06f9fad8: 00000000 072c9500 072c9500 12277398 ......,...,..s'.
0x06f9fae8: 00000000 00a90000 12277398 00000028 .........s'.(...
0x06f9faf8: 06f9fba8 6a60195d 072c9500 00000000 ....].`j..,.....
0x06f9fb08: 1350e698 00000000 00000000 0ebc4b08 ..P..........K..
0x06f9fb18: 1208ee10 072c9500 12023348 00000000 ......,.H3......
0x06f9fb28: 1226f420 1350e698 0eddf0b0 0b77e1f8 .&...P.......w.
0x06f9fb38: 1207dfcc 00000048 00a97ff8 00000f00 ....H...........
0x06f9fb48: 0000002b f9060491 000000d5 00000000 +...............
0x06f9fb58: 0eb09750 072cea14 00000000 0ec6ed3c P.....,.....<...
0x06f9fb68: 00000000 00000005 00000000 00000000 ................
0x06f9fb78: 072c8a68 2e04f529 00000000 06f9fbfc h.,.)...........
0x06f9fb88: 6a4b0fb7 0ec6ed3c 00000001 0eb09750 ..Kj<.......P...
0x06f9fb98: 072c9500 2e04f515 6a4b144e 00000000 ..,.....N.Kj....
0x06f9fba8: 06f9fbd8 69ef47b0 072c9500 1226f420 .....G.i..,. .&.
0x06f9fbb8: 00000000 67a94374 1350e698 00000000 ....tC.g..P.....
0x06f9fbc8: 072c9500 072c8a68 0737e498 1226f420 ..,.h.,...7. .&.
0x06f9fbd8: 06f9fbe4 69eeb74d 06f9fc20 06f9fc08 ....M..i .......
0x06f9fbe8: 69ef1879 072c9500 0737e480 0737e970 y..i..,...7.p.7.
0x06f9fbf8: 072c8cc8 072c8a68 072c8a68 072c9500 ..,.h.,.h.,...,.
0x06f9fc08: 06f9fc28 69ef17e6 06f9fc1c 06f9fc20 (......i.... ...
0x06f9fc18: 072c8a68 0737e480 0737e4a0 00000005 h.,...7...7.....
0x06f9fc28: 06f9fc44 69ef1748 6a79bea9 072c8db0 D...H..i..yj..,.
0x06f9fc38: 0ec6ed60 0ec6ed60 00000000 06f9fc50 `...`.......P...
0x06f9fc48: 6a43f8e5 0b39c228 06f9fc88 6a79bdfd ..Cj(.9.......yj
0x06f9fc58: 072c8db0 2e04f285 6a79bea9 0b39c228 ..,.......yj(.9.
0x06f9fc68: 0737e970 c0000005 06f9fc5c 06f9f4fc p.7.....\.......
0x06f9fc78: 06f9fce0 6a698cb0 426f24e5 00000000 ......ij.$oB....
0x06f9fc88: 06f9fc94 6a79bf25 6a79bea9 06f9fca8 ....%.yj..yj....
0x06f9fc98: 758262c4 0737e970 758262a0 da97cbb6 .b.up.7..b.u....
0x06f9fca8: 06f9fcf0 778e0719 0737e970 afa526a9 .......wp.7..&..
0x06f9fcb8: 00000000 00000000 0737e970 c0000005 ........p.7.....
0x06f9fcc8: 00000000 06f9f4dc 76c0d150 afa526a9 ........P..v.&..
0x06f9fcd8: 06f9fcb4 06f9f4dc 06f9fcf8 778f5ef0 .............^.w
0x06f9fce8: decba7f1 00000000 06f9fd00 778e06e4 ...............w
0x06f9fcf8: ffffffff 77902636 00000000 00000000 ....6&.w........
0x06f9fd08: 6a79bea9 0737e970 00000000 00000000 ..yjp.7.........
0x06f9fd18: 00000000 00000000 00000000 00000000 ................
0x06f9fd28: 00000000 00000000 00000000 00000000 ................
0x06f9fd38: 00000000 00000000 00000000 00000000 ................
0x06f9fd48: 00000000 00000000 00000000 00000000 ................
0x06f9fd58: 00000000 00000000 00000000 00000000 ................
0x06f9fd68: 00000000 00000000 00000000 00000000 ................
0x06f9fd78: 00000000 00000000 00000000 00000000 ................
0x06f9fd88: 00000000 00000000 00000000 00000000 ................
0x06f9fd98: 00000000 00000000 00000000 00000000 ................
0x06f9fda8: 00000000 00000000 00000000 00000000 ................
0x06f9fdb8: 00000000 00000000 00000000 00000000 ................
0x06f9fdc8: 00000000 00000000 00000000 00000000 ................
0x06f9fdd8: 00000000 00000000 00000000 00000000 ................
0x06f9fde8: 00000000 00000000 00000000 00000000 ................
0x06f9fdf8: 00000000 00000000 00000000 00000000 ................
0x06f9fe08: 00000000 00000000 00000000 00000000 ................
0x06f9fe18: 00000000 00000000 00000000 00000000 ................
0x06f9fe28: 00000000 00000000 00000000 00000000 ................
0x06f9fe38: 00000000 00000000 00000000 00000000 ................
0x06f9fe48: 00000000 00000000 00000000 00000000 ................
0x06f9fe58: 00000000 00000000 00000000 00000000 ................
0x06f9fe68: 00000000 00000000 00000000 00000000 ................
0x06f9fe78: 00000000 00000000 00000000 00000000 ................
0x06f9fe88: 00000000 00000000 00000000 00000000 ................
0x06f9fe98: 00000000 00000000 00000000 00000000 ................
0x06f9fea8: 00000000 00000000 00000000 00000000 ................
0x06f9feb8: 00000000 00000000 00000000 00000000 ................
0x06f9fec8: 00000000 00000000 00000000 00000000 ................
0x06f9fed8: 00000000 00000000 00000000 00000000 ................
0x06f9fee8: 00000000 00000000 00000000 00000000 ................
0x06f9fef8: 00000000 00000000 00000000 00000000 ................
0x06f9ff08: 00000000 00000000 00000000 00000000 ................
0x06f9ff18: 00000000 00000000 00000000 00000000 ................
0x06f9ff28: 00000000 00000000 00000000 00000000 ................
0x06f9ff38: 00000000 00000000 00000000 00000000 ................
0x06f9ff48: 00000000 00000000 00000000 00000000 ................
0x06f9ff58: 00000000 00000000 00000000 00000000 ................
0x06f9ff68: 00000000 00000000 00000000 00000000 ................
0x06f9ff78: 00000000 00000000 00000000 00000000 ................
0x06f9ff88: 00000000 00000000 00000000 00000000 ................
0x06f9ff98: 00000000 00000000 00000000 00000000 ................
0x06f9ffa8: 00000000 00000000 00000000 00000000 ................
0x06f9ffb8: 00000000 00000000 00000000 00000000 ................
0x06f9ffc8: 00000000 00000000 00000000 00000000 ................
0x06f9ffd8: 00000000 00000000 00000000 00000000 ................
0x06f9ffe8: 00000000 00000000 00000000 00000000 ................
0x06f9fff8: 00000000 00000000 ........

Module 1
E:\Projekty\Expo\Expo\Build\expo.exe
Image Base: 0x00fe0000 Image Size: 0x011b8000
File Size: 17729536 File Time: 2016-09-01_113248
Version:
Company:
Product:
FileDesc:
FileVer: 5.4.1.40776
ProdVer: 5.4.1.40776

Module 2
E:\Projekty\Expo\Expo\Build\expo_Data\Plugins\AVProVideo.dll
Image Base: 0x03e00000 Image Size: 0x000cb000
File Size: 796672 File Time: 2016-10-31_211026
Version:
Company:
Product:
FileDesc:
FileVer: 0.0.0.0
ProdVer: 0.0.0.0

Module 3
C:\WINDOWS\system32\nvspcap.dll
Image Base: 0x10000000 Image Size: 0x0015c000
File Size: 1423304 File Time: 2015-10-12_040502
Version:
Company: NVIDIA Corporation
Product: NVIDIA GeForce Experience
FileDesc: NVIDIA Capture Server Proxy
FileVer: 2.5.15.54
ProdVer: 2.5.15.54

Module 4 ....

// I HAVE REMOVED REST ~100 MODULES LISTING

I have recent Nvidia drivers. When I'm disabling my Nvidia GPU and runs app on Intel's integrated GPU it works without crashes.

When my build crashes and I'm trying to get into Playmode in Unity -Unity also crashes. But then error stands that it is caused by nvwgf2umx.dll Acces Violation
Any ideas ?

 

Yes, we have fixed that bug in 1.5.6, I recommend you upgrade to the latest version of the plugin.

Thanks,

 

Hi

I'm using the Audio Out option with AVpro for Windows. I'm using this to get the audio to the Oculus headphones rather than position the audio source so I'm not using 3d positioning. The problem I'm having is that changing the volume doesn't work in either the AVpro plugin or on the Audio source. The other issue I have which I've found a workaround for, is that the audio source requires an audio file to be in it to get the audio to pass from AVpro to the audio source. Without it I don't get any audio. I've worked around this by putting a WAV with no audio in it in the audio source.

Thanks for a great plugin.

Phil

 

Hi,

The same happens when I try to create a UWP app with AVPro, without any HoloLens magic. Just build a UWP app that has AVPro package imported (not even used), and the resulting UWP app crashes even on a desktop.

Any help would be appreciated, I'm heading fast for a deadline...

 

We're looking into this now. It's odd - it seems to run if you do a Debug build from Visual Studio...But not Release or Master. Hopefully we'll find a solution soon.
Thanks,

 

Hey Phil,
I think in the editor that volume may be getting overwritten. It should work in a standalone build? That is, you should be able to do mp.Control.SetVolume().

A workaround is to set both:
mp.m_Volume = x;
mp.Control.SetVolume(x)


Hmm, the AudioSource shouldn't need an AudioClip assigned to it. It does need the AudioOutput component to be before (above) it though.

Hope these two suggestions help. Let me know if they don't.

Thanks,

 

Hello again

Another question from me.
I need to rewind my video when it's finished. I'm using this code:

Code (CSharp):

1. mediaPlayer.Control.Rewind();
2. mediaPlayer.Control.Pause();

But sometimes it is rewinded to very begining (like 0.0 ms) and sometimes to first frame. My videos are ~1 sec long.
The issue is when it is rewinded to 0.0ms MediaPlayer returns black frame instead of my videos' first frame. Pics related:

Good Rewind:


Bad Rewind:


Same code, same movie Why it happens?

 

Hi,
Are you still using 1.5.5? I believe this is a related bug and was also fixed in 1.5.6.
Thanks,

 

Hi we've tested here but it's working as expected. I tested with just building UWP apps in Unity 5.3.6 and 5.4.2 and also building Hololens using 5.4.0-HololensVersion.

Did you try with any of these versions? I don't think using the beta of 5.5 is the safest option...

Are you trying to run from within Unity (build and run) or via Visual Studio? If Visual Studio then have you tried selecting different build profiles (x64, x86, debug, release etc)?

Thanks,

 

Thank! address: rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov
log:
[AVProVideo] Failed to open rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov
UnityEngine.Debug:LogError(Object, Object)
RenderHeads.Media.AVProVideo.MediaPlayer:OpenVideoFromFile() (at Assets/AVProVideo/Scripts/Components/MediaPlayer.cs:380)
RenderHeads.Media.AVProVideo.MediaPlayer:Start() (at Assets/AVProVideo/Scripts/Components/MediaPlayer.cs:319)

 

Hi,
I think the problem is that Android doesn't handle MOV files. If you used MP4 then it should work.
Thanks,