[RELEASED] VR Panorama - render 360 stereo videos

Sent you the invoice number this afternoon. Oliver Pavacevic Oliver Graphics, correct?

 

Hi there,

I downloaded this asset, looks amazing and i hope to get some great results out of my scenes.

It seems that the VR 360 camera is not working well, i get strange stereo images, can't be right i think.

Any idea what is happening ?

 

It can be that you have forgotten your oculus attached to your computer and turned on. It happens when oculus tracking takes controll of unity cameras. Turn it off, and everything should work fine.

 

Based on your demo video this looks like a cool way to do 3d videos for YouTube that can be viewed without vr gear. Is that the case or is other sofware involved as well ?

 

I am happy with this product. Fast rendering time and does everything what it says. However I am having some problem with the scale of the rendered scene. It stretches upwards and can't seem to find the correct camera setting. Tried to play with the setting even scaling the room but this didn't help. Am I doing something wrong ?

 

Could you send me a screenshot or video example?
Most important thing, for correct scale perception, is IPD size (distance between eyes). It is in meters, so it should be 0.064 - 0.067. Check this value if your scene is scaled correctly (1 unit = 1 meter). Does this happens only in stereo mode, or in mono? Contact me here on in private and we will find a solution

 

For now, this plugin doesn't manage audio from unity (as this is a difficult task, because rendering isn't done realtime. But I'm working on a solution). For now it should be added later in video editing software. Other than that, when rendered, you have to use Youtube tool for 360 videos. This tool injects metadata that is needed by youtube before uploading. It takes seconds to inject this data.
BTW, I'm doing some researches on what data should be injected into a video stream, so you can expect that VR Panorama will do this for you in future versions.

 

Hi
I also suspect the IPD configuration. Where can I reconfigure the IPD size ? I am using the very latest version of your product. But I couldn't find where to adjust the IPD setting. Thanks again for the great support !!

 

I know this is just render your game into a stereo video (which imo you can do for free using the oculus sdk + fraps) ...

but, what about something for playing spherical videos on iOS or Android?

 

Not exactly Stereo panorama video are quite different and impossible to capture with screen capture software. Well, you could capture a video from oculus, but you would loose rotational tracking. It would be just a normal video, not a panoramic video.

There are many solutions for playing spherical videos on Ios and android. Youtube app lets you play them. There are also Google cardboard app or Kolor Eyes player. Or dedicated apps for Gear VR.
As for integrating 360 videos inside Unity app, this task is a little bit more complicated, because of Unity MovieTexture limits. Unity Movie Texture can't play well Hi Res 2k or 4k videos so you would end up with a bad performance and low quality.

 

Exactly - wishlist item: consistent plugin for spherical video for unity movie texture on ios/android.

 

New version is available on assetstore.
It features new improoved interface and many new options.

 

cool which new options?

 

This looks pretty cool but I cannot actually work out exactly what it does, or why we need it? Can someone explain in layman terms what this asset is for? Im involved in VR and very interested but I'm struggling to work out the purpose, mind you I been coding for 16 hrs.

 

Works Very nice! I just test the older version, and works fine for me.... Just not happy enough with the final quality, but it`s no the plugin fault...it`s the pixel limit of h264. Can`t wait to an improve of h264 or the replace by h265. BUT VERY HAPPY WITH THE ASSET!!! It will be nice to have a permanent camera visualization of the final render. Y use the game view, but when i make some change y edit view i lost the camera reference.

 

I'm working on a realtime (in editor) lo-resolution preview, and also on a rendering time preview in full res with some onscreen rendering informations. This is one of the things that is on TODO list, and will be featured in VR Panorama 1.0.

It supports also h265 but a feature is not exposed yet, will be added in next release.

 

A whole list is in a first post of this thread

 

It can be used to make stereo 360 panorama videos for Gear VR, Google Cardboard, cutscenes for your VR Applications, 360 videos for streaming services like youtube or VRideo. Stereo panorama videos are like normal videos, so there is no interactivity. But they support rotational tracking and can be viewed on devices that aren't capable to render complex 3d realtime scenes. With this tool you can make a presentaions of your product (game, assets) or make 360 stereoscopic movies (a new form of media). Also, you can use it to export screen captures from unity.

 

It can be used to make this.

Interactive Pano

 

Hi olix4242! Will UI elements captured by VR panorama camera, too? I have VR scenes with WordSpace Canvas, including images, text, video on image object etc. Really would like to know if these elements will be in the rendering, too (all worldspace canvas of course)?? Do you have examples which show UI elements in 360 video?

 

Yes, you can render them, but they have to be in World Space. Also, there are some procedures to follow - VR camera and Canvas should be parented to an object that holds animation (if you want to animate your VR Camera movements).

 

Ah sorry, I forgot to mention this: VR panorama at this state doesn't support Layered cameras. It supports layers, but not multiple cameras (and I don't know when or if it will support them - mainly for performance reasons and Unity limitations).So, you wil have to workout somehow z fighting problems with interface. It has to be rendered together with all other elements. I'm doing a rearch on how it can be done, but for now, all solutions are too costly in terms of performace and VRAM usage.

 

Thanks olix4242!
I would not need layered cameras, just the main camera with some world space canvases.
Could you do me a favour and test if something like this can be captured with VR panorama 360 (one canvas parented to camera, others similar to static 3d objects in scene)? Maybe you could send me 2 frames out of the camera animation (different positions) captured in 360, just to see if it works?! Thank you!

Another question, is it possible to render the png's with alpha (if camera background is set to solid with transparent color)? This way more layers / cameras could captured seperatly and put together in post production...

 

I made a whole animation (took 30 sec. of rendering time for a 10 second clip).
https://www.dropbox.com/s/kixt0a270jgacs5/VR_UI.mp4?dl=0

Not yet, but I'm working on this. Probably for a next release.

 

Thanks olix4242! Purchased!
Your asset works great, super simple to use, fast rendering!

 

Hi there, I just bought this hoping I'd see more performance gains vs the free offerings, but unfortunately so far it's pretty much the same.

I see that VR Panorama/Unity Editor is only using about 14% of my CPU according to Task Manager while it renders away. Disk usage is at 0-1%.

Is there anything I can do to enable any sort of multithreading or something?
I'm on a 4.5gHz i7 with an Asus Strix 970

Edit: I should note that the H.264 encoding happily uses 100% of my CPU, but of course that is after the fact...

 

Also, there is a bit of a bug with the "Encode H.264 video after rendering" option: It only works if the files were exported as jpg.

Line 217 of VRPanoramaEditor.cs
String formatString has ".jpg\"" hardcoded. It should probably be reading from the VRP._prefix

 

CPU usage shouldn't be great, and this is normal as the rendering is done on GPU and CPU is idle while GPU renders. It should depend mostly from what you are rendering. On I5 I get something like 50% for a quite heavy scene.. Be sure that your editor is set to "Run in background" as otherwise it will pause rendering when not in focus. Go to Player settings and set Run In Bakground to true. Also, turn off a Vsync in your quality settings.

From my benchmark tests (on a same STRIX as yours), you should gain at least 5X - 10X faster rendering with VR Panorama for a 2048x2048 stereo. All this with overall better AA quality in deffered mode and with a complete support for all image effects.

 

Yes, this is known bug and a correction is in a next patch. Nevertheless, Thanks!
BTW, I'm suggesting to use JPG sequences, if you don't have to do any heavy post color correction. It should give you good quality (JPG is coded at max quality), takes less space, and saves/loads faster resulting in a faster workflow.

 

Ahh, okay the GPU offloading makes sense. I'll keep playing and see how it goes. Thanks for the reply

 

If you have other questions, feel free to ask

 

Seems that I do. I've just finished a render of a current project with the following settings. The video plays back nicely and looks great, but it's way slower than it should be and is much longer than anticipated. My settings + outcome settings are attached... any ideas?

 

Ok, you found new bug. Surry about that, a documentation for library that I used for compression was misleading. But a solution is easy: you have to change all lines in VRCapture.cs that start with:

Code (CSharp):

1. System.Diagnostics.Process.Start( ffmpegPath...

(there are two of them)
with :

Code (CSharp):

1. System.Diagnostics.Process.Start( ffmpegPath + "ffmpeg" , " -f image2" + " -framerate " + FPS + " -i \"" + fullPath + _prefix + "%05d" + formatString + " -vcodec libx264 -y -pix_fmt yuv420p -b:v " + Mp4Bitrate + "k \"" +fullPath + Folder + ".mp4\"");

and for VRPanoramaEditor.cs to:

Code (CSharp):

1. System.Diagnostics.Process.Start( ffmpegPath + "ffmpeg" , " -f image2" + " -framerate " + VRP.FPS +  " -i \"" + fullPath + VRP._prefix + "%05d" + formatString + " -metadata Spherical=\"true\"  -metadata Stitched=\"true\" -metadata StitchingSoftware=\"Spherical Metadata Tool\" -metadata ProjectionType=\"equirectangular\" " +  " -vcodec libx264 -y -pix_fmt yuv420p -b:v " + VRP.Mp4Bitrate + "k \"" +fullPath + VRP.Folder + ".mp4\"");

Sorry about this, it will be corrected in a next version.
On the other hand, If you are rendering for online services like Youtube, I'm suggesting to keep your framerate at 25 or 30. It should be enough - and it will give you better encoding quality for a same file size, and, mostly important: a better performance on a client side.

 

Yep, got it thanks. Don't forget there's one in VRPanoramaEditor.cs as well that needs to change to:


System.Diagnostics.Process.Start(ffmpegPath + "ffmpeg", " -f image2" + " -framerate " + VRP.FPS + " -i \"" + fullPath + VRP._prefix + "%05d" + formatString + " -vcodec libx264 -y -pix_fmt yuv420p -b:v " + VRP.Mp4Bitrate + "k \"" + fullPath + VRP.Folder + ".mp4\"");

 

Next update will come with some small bug fixes, but mosty with optimized VRAM memory usage, and even higher speed rendering (it should give from 30% to 70% faster rendering). You shoud notice some greater performance improvements, mostly for 4k stereo renderings. Also, an audio capture support is underway.

 

Important notice:
Unity 5.2.1f1 introduced a bug on a VR device detection. If you aren't using a HMD device and get an error:

Code (CSharp):

1. Unable to render VR Panorama, please, TURN OFF your VR HMD Device UnityEngine.Debug:LogError(Object) VRPanorama.VRCapture:LateUpdate() (at Assets/VRPanorama/VRCapture.cs:719)

Please go to the Project Settings/Player and activate "Virtual Reality Supported".
Looks like VRDevice.isPresent in Unity 5.2.1f1 gives a true result if Virtual Reality Supported is off (but it should give false result).

 

Audio capture support is done, thanks to all of you who voted for it. A new version will hit the store as soon as possible, after some testing.
More on audio capture:
-For now it's a little bit less intuitive than I would like it to be. There are some problems that are hard to solve. One of them is syncro with soundtracks that have "Play On Awake" enabled. In Unity Awake fires before a level loads, so audio would star before capture, leading to chopped or longer audio track. For this type of sounds, there will be a script that has to be added to a sound, a simple script that starts audio on level load, and not on awake. Fro other sound that fire with scripts, or a loopable ambient sounds, this isn't a problem, and they will sound correctly.
Workflow will be this: You will have two buttons, one for audio capture, and second for video capture. If you want audio capture, it should be done separately and before Video capture or Video Encoding. You should click a button for audio capture, and a scene will play and capture audio to a desired folder (same as image sequnce folder). It will finish playing after a length givven in a sequence parameters. Afterwards, you should click on a button for Render Panorama, and if a checkbox "Encode after rendering" is selected, it will look if there is an audio file in a folder, and it will encode it together with video.

 

I'm Unity 5.2 user.

I replace the code as under to work.

'''
//#if !UNITY_5_0
#if !UNITY_5
'''

 

For proper audio capture, I'm assuming that the game/whatever has to be able to play back at full speed in realtime, correct?

 

Yes and no Audio capture is separated from video capture, and audio capture will start a scene at normal speed without capturing panorama. Thus should be enough to fire all audios at correct time and to record an audio track. Of course, this can give you some problems if there are scripts that use random audio generation, but there's not much I can do on this side. You should assure that you use controlled repeatable random scripts (with random seeds).
Also, in some cases you will (as I do) want to use best quality settings, ton of image effects, hi polycount meshes and so on, as you dont have to worry about framerate. But, in this case, it's enough to turn off your rendering camera component as audio capture doesn't need it. It needs only audiolistener.

 

I was having trouble getting exported files to play back at 60fps on the gearVR, I dug into the FFmpeg call your making and I think I found the bug.

ffmpeg requires the "-framerate" flag as the first arg to override the default framerate of 25, then you have to move the "-r" to after the "-vcodec" for it to count as setting the output file. Yes ffmpeg is funny that way.

with this version my video comes out as 90 seconds and 60 FPS and not 3 min 36 seconds at 25 fps.

Code (csharp):

1.  
2. ffmpeg.exe -framerate 60 -f image2 -i img%05d.jpg  -vcodec libx264 -r 60 -y -pix_fmt yuv420p -b:v 14000k VR_Pano_cmd.mp4
3.  

 

Yes, thanks, this is now known bug, I've posted a solution one week ago here on forum topic. Sorry about that, but information on ffmpeg wasn't quite clear. It is corrected in a new version of VR Panorama, and will be submitted to assetstore in day or two, with other improvements. If you want it before it gets aprooved, feel free to contact me.

 

Just scrolled up and saw that, Glad the fix is coming out.

 

This might be useful to others. http://support.middlevr.com/hc/en-u...VR-compliant-post-processing-effects-in-Unity

 

Yes and no. Article speaks about effects for cave systems - but a VR Panorama improoves a lot upon this. For example, all bloom, glow, DOF, Ambient occlusion and obscurance, Camera motion blur effects will work just fine. One thing that article mentions as compliant is a Tonemapping, but it works only in static modes - and can't be used in dynamic adaptive mode.

BTW, new VR Panroama version was submitted this weekend, with some speed and VRAM improovements and an audio capture tool, and will be online as soon as assetstore team aprooves it.
If someone needs it before, just drop me a PM with your Invoice number and I will send it to you.

 

New version is available in asset store!

 

Plans for future releases:
-Flexible support for layered cameras (initial support is ready, if you need it before feel free to contact me).
-Side by Side rendering.

 

Hi, I am new for Unity. I am just wondering that if VRPanorama could render well on Mac?
Because I once tried use another plugin to render panorama image, but it required PC with DirectX 11.
I was so frustrated.
I'm still seeking the possibility of rendering VR from Unity in Mac OS.
Thank you.

 

Yes, it is fully compatible with OSX and doesn't require DX11

 

Hi,
Using your project we wanted to convert our linear unity gearVR experience into 360 video for MilkVR video.
It looks like it is perfectly capable of doing so but when we include it in our project and run it, although the scene view is accurate, the output video is in a different position and does not render much of our assets, just text, gui, some trees, and skybox weirdly enough. Can you shed any light on this?
It seems like the VR camera is being set to weird culling mask but because of the rendering dialogue box when we press play we can't pause to figure out what's going on.


Thanks,
Will