Jove 2.0 - Replacing the rendering pipeline

Say you declare "float4 _MyShaderVariable[2]" you can then fill it from the CPU using Material.SetFloat("_MyShaderVariable0", 0) and Material.SetFloat("_MyShaderVariable1", 0). Unity supports shader arrays, it's just a little convoluted to fill them from the CPU

DX11 is a lot of freedom. I mean, I've already done a lot of stuff I would not be able to do otherwise and I still have not implemented even a quarter of all the DX11 features I want to do.
On a completely unrelated note, I'm very happy with how the glass shader turned out

 

It's a strange way to pass shaders ^^
I know dx11 is a lot a fredom because with my wip Unity indie graphics effects, dx11 is 1.5~2 times quicker than dx9.
With dx9, a lot of things (like dynamic branching) is much slower.

 

Aieth I was looking at this new asset which is pretty cool, would I be able to use something like this in Jove...

http://forum.unity3d.com/threads/horizon-on-make-your-horizons-look-real.259733/

 

Depending on how that is implemented, it might even work right away.

DX11 does branching a lot better like you say, and it has compute shaders. Both of those are real game changers.

 

Fascinating. The introduction post is very well written and the visual result is very immersive, the atmospheric scattering is very realistic and futuristic for grapics, it creates mind warping atmospheres

I would be very interested to read the documentation to learn about rendering technology, to figure out what kind of shaders Jove uses.

 

I appreciate your kind words
And I would also be very interested in such a documentation. If only it would create itself I am going to create such a documentation at some point, but the initial documentation is going to be more aimed at instructing how to use Jove than how it was made

 

I see you've reworked Unitys shadows which is a welcome change. I'm not familiar with the shadow techniques you used, but can I ask if their anything like PCSS shadows. Where the penumbra increases with distance or is it a constant blur? Whatever it is they look nice!

 

Unfortunately they are not like PCSS shadows. I really wanted to do that and was all set for it, but shadow techniques with a variable penumbra are just too expensive to use for anything but the highest quality settings.
The technique I am using is known as exponential shadow maps, it is comparable to standard PCF shadows (which Unity use) but they have various benefits, it is easier to combat aliasing and you do not have the same issues with banding/shadow acne. There are also a few other technical benefits, they integrate well into other algorithms that need shadow map access (such as volumetric fog and stuff).
I've also been able to make available a lot more customization than default Unity does, so as a developer you have a lot of control over the looks as well as everything that affects performance. For example, you get to specify how many frames should pass before updating (re rendering) a shadow cascade. On my system I've been able to get fully cascades shadows at really good quality down to 2ms with settings tweaking, but I've also made it take 20ms Basically, more freedom to the user (as in, the developer, not the developers users... that would be bad)!

 

So many options

 

OK so this begs the question... will we have presets (lights, shadows etc) and will we be able to share presets between users, like a repository of sorts...

 

Interesting question, I had not thought about that possibility. I suppose that would be very useful, both to store quality settings and to share with others. It won't be in the first version but I like the idea

 

Okay....now this is getting interesting...finally every PBL renderer have their own distinctive feature which is very awesome and i really support it. Great work Aieth...especially for the custom shadow

 

I've just got a simple question — your works looks great, but aren't there other already existing solutions that can really effect the appearance of a game? Won't unity 5 also introduce similar graphics upgrades to the ones you are currently implementing? I'm sorry if I have completely missed the purpose of your project.

 

Also I see you haven't gotten to DoF and motion blur yet. I think the most natural implementation of both these things that I've seen is the icenhancer mod. Here's a good showcase of it.

it has like a autofocus thing going on. What do you think? great work btw

 

Thanks for your support The shadow mapping system has probably seen the most revisions since its initial birth six months ago, so it better be good now

Yes, there are other solutions to various parts of a graphics pipeline. However there is a major difference between five random systems designed around their own thing and a system of components designed with a common goal.
For example, if you were to go and purchase the 5 best graphical assets you can think of and try to use them all in the same project you would likely run in to a host of compatibility and performance issues. You can compare it to a car, it is no big deal to change one part for a custom part, but if you want to do start combining multiple different parts you really need to know what you are doing or its gonna get funky.

That is probably the best way I can explain it without going too technical If anyone wants a technical explanation, say so and I'll write one as well, it's a fair bit more involved.

As for Unity 5, again it is hard to explain without being too technical. Unity is a multi purpose engine designed for a wide variety of use cases. This has meant, means and will always mean that Unity suffers from the jack of all trades, master of none syndrome. At least to me, Unitys strength has always been the flexibility of the framework and not specialized features. And that flexibility is what has allowed me to toss multi platform compatibility out of the window and aim solely for DX11 level features. Not much is yet known about graphics in Unity 5 except that it will support physically based shading, reflection probes and Geomerics Enlighten. But I mean, my first version of Jove, 1.0, did the first two and it is hardly a masterpiece in its current condition. And if Enlighten is what it seems to be, there's no reason I can't support it in Jove Jove 2.0 does so much more than Jove 1.0, and there are a lot more things to a graphical pipeline than just shading. Post processing, scattering effects and volumetrics play a huge part, as does the lighting system itself. Jove 2.0 is very close to a releasable alpha status, but I have far more planned and I can confidently say that Jove is going to be superior to Unity 5 graphically, simply because I do not have the restraints that the Unity team do. I've barely begun implementing post processing and already they transform a scene from meh to wow.

You can actually do that in standard Unity already You simply ray cast every frame and set the focus distance to the length of that ray cast. I myself am glancing at Crytek and their technology, I love their recent move towards physically based everything.




EDIT: Here is a comparison with the few image effects I have on and off. Basically it is just bloom, atmospheric scattering and vignetting. Already makes a huge difference, and there is still a lot to do. What I am saying is that there is a lot more to graphics than simply shading

 

Wow, that thing looks incredible. I assume it looks best with HDR enabled?

One thing I'm curious about is how to handle anti-aliasing in the situation where you have UI elements sandwiched in between 3D elements. Since Unity doesn't support hardware AA in HDR mode, it means you have to use post-processing AA. But when you have a mix of UI overlays and 3D meshes (not to mention a detailed skybox), post AA can look bad. It can blur your crisp skybox images and UI elements.

Ideally there'd either be a way to AA only the edges with a post AA effect (if that's even possible), or have a way to look decent without enabling HDR. Given how differently Jove works, I'm just curious if there's any clever way to solve that issue. I realize that may be wishful thinking!

 

In Jove you don't get a choice, you get HDR anyway This really wouldn't work in LDR, if you force it to LDR the whole image is basically white.

Post process AA it is I'm afraid I do however plan on implementing a temporal AA solution (basically that is a fancy way of saying blending between multiple frames), but before I can do that I need to add motion blur as believe it or not it depends on that. Temporal AA is pretty much like free near free MSAA, but it has a host of prerequirements before it becomes feasible and I *almost* have all of those in Jove now. For now you are unfortunately stuck with the default post process anti aliasing scripts

You could disable AA from running on GUI elements and/or a skybox by using a stencil mask, that would be trivial with most post process AA solutions. Perhaps I am overlooking something though and stencil masking is not an option for you? It seems like the obvious choice

 

Ah, good to know about HDR and Jove.

Hehe, I'm sure it's obvious to you, but you're like 17 levels above me. I'm a "weekend and evenings" hobbyist who tries to spend most of my time on gameplay-related things, so I rely quite a lot on existing assets for art and shaders, etc.

I really had no idea about the stencil mask thing - assuming that solves the UI and skybox blurring issue, that opens up a lot of possibilities for assets that require HDR, like Jove. So would this stencil mask thing require that I write or extend custom shaders? I vaguely remember doing stencil buffer shadows in XNA, but that's been a while. I'll Google it. Thanks for the reply!

 

I apologize if I sounded elitist, it is hard when you don't know everyones levels of knowledge
In Jove I tag all my deferred objects in the stencil buffer so that the expensive lighting calculations only run on pixels where it actually matters.
You can use something like this to tag the 8th bit. Add it inside the subshader brackets on the shaders you want left alone

Code (csharp):

1. Stencil
2. {
3. Ref 128
4. WriteMask 128
5. Comp Always
6. Pass Replace
7. }
8.  

Then add this to an AA shader and you should be set

Code (csharp):

1.  
2. Stencil
3. {
4. Ref 128
5. ReadMask 128
6. WriteMask 128
7. Comp NotEqual
8. }
9.  

If this sounds complicated, you could set up a second camera as well. That would be a lot easier, at least if you end up using Jove. Just make sure its draw order is after your main camera and just render UI and stuff on top.

 

Very interesting, but why wouldn't you just develop using crytek or unreal if you're aiming towards a certain platform? Aren't there pipelines more focused towards high end computers and dx11? Is the system you're making going to be better than what these two engines have implemented? Again, I'm just curious, it seems like you have spent a ridiculous amount of time on the project already!

 

This looks great. Add a little film grain and it could fool me for a cgi movie

 

True and a valid point. Thing is though, Unity is ridiculously easy to work with. Unity has completely nailed the workflow, and Unity is an awesome tool. So the choice is really between ultimate control (UE4 source access) and awesome high end features vs usability and dated features. What if Unity had awesome graphical features as well? I mean, people tend to overlook this, but there is a lot more to an engine than just graphics and Unity has nailed many areas, and not just for mobile development. I can't compare them straight up performance wise since I simply just don't know, but graphically I am pushing towards that quality. Not there yet, but gonna get there.

You're gonna see a lot of fancy effects as soon as I get to it I really want to do that though, can't wait.

 

Ok one more thing and I promise I'll be out of you hair I know you plan on doing clouds after everything but I just wanted to throw this out there. http://reset-game.net/?p=284
Do you think this is feasible?

 

Hehe, no worries. It didn't come across as elitist - you clearly know what you're doing, so I have no doubt that something like that IS obvious.

For the game I would consider using to Jove with, I already have a stack of five cameras (10 for Oculus Rift!), two of which contain my UI elements. It's a first-person space game where some UI elements are outside of the cockpit and some are inside. So with Jove I could alter the draw order to draw the cockpit meshes and the 3D world objects, apply AA, and then drop in the skybox and UI stuff? If so, that would be too perfect. Sorry if I'm not understanding it right.

 

I've already got the same sky algorithm that they are using, but the rest of what makes up a sky (clouds/volumetrics) is gonna get some love in one of the first patches.

The sky is rendered as part of the main rendering, but otherwise yeah. You'd just use a second camera to render in UI afterwards like you are currently doing.

 

Sorry to be dense, but you're saying that with the proper draw order settings using Jove I could draw the UI after the normal geometry and the UI could still be drawn "behind it" when necessary?

 

Out of the box that would not work, but it would be very easy to modify so that the depth buffer from the main rendering can be used to cull UI that is supposed to be hidden behind objects in the game. I could help you set it up, would not take long

 

That would be great!

 

So what part of the roadmap would you say your on now in terms of features and a release?

 

This looks awesome! I'm glad someone is focusing on high end hardware. Was wondering if you were planning on supporting any of the following features

1. Volumetric light scattering
2. Depth fade particles (Like in UE4, where particles fade as they approach a surface to stop hard edges)
3. Particles which interact with light

Keep up the good work!

 

This is already supported in Unity for long time ago...

 

Do you mean soft particles?

 

isn't,or it just maybe i missunderstood???

 

I just took a look at them again and you're right - I was missing the soft particles factor setting

 

This looks very promising. I was always wary of using third-party rendering-related assets with Unity as vast majority of them usually tackled some isolated effect (only cubemaps, or only IBL, or only fog, or only shadows), which promised a world of pain, hugely complicated workflows and problems with performance whenever you would try to merge those solutions into one project.

This is the first project in the field that looks like an all-encompassing solution that covers a lot of use cases. I'm very much looking forward to it being released. All I have read about it so far makes it infinitely more attractive than using separate narrowly specialized assets.

 

I'm on the less glamorous pre release fixup part Meaning I'm user proofing stuff, coding UI and generally cleaning up. I've also spent a few days working in light cookies for all light types, which normally wouldn't be very hard but it is a complete major pain in the ass when doing clustered shading. Normally you would have two shaders, one with and one without cookies and then just run the appropriate one while iterating over your light list. Since I am processing everything (from lighting, to scattering and fog) in a single pass I can't do that It is finally working, so now I am moving on to the last UI script I need to write.
After that I'm writing documentation, and after that... well, I'm thinking volumetric scattering (fog) We'll see if I get it done before Sunday which is launch day (uploading it then, gonna take a few days before Unity accepts it on the store).

1. Definitely, hopefully in time for release but otherwise it is in the first patch
2/3. To begin with I'm just porting Unitys particle shaders to work with Joves depth buffers. I'm probably also doing a vertex lit particle shader to begin with, but later on I want to do my own particle system which will allow me to support more advanced features.

You are spot on. This was exactly my reasoning when I began working on Jove. You can't split graphics into parts, develop them independently of each other and except to end up with a great result. So much stuff can be optimized by integrating it into other stuff. The entire Jove pipeline is literally this:
1. Render GBuffer
2. Evaluate lighting, scattering etc per pixel in a single pass (all lights, all shadows, all cookies, all scattering etc...)
3. Add in bloom and post processing then tonemap to displayable LDR

It is going to increase with more image effects, but it is still way less passes than what would have been the case if you bought 10 products for different parts and used it in the same pipeline.


It is not the fanciest of pictures, and it is probably the most retarded point light cookie you've ever seen, but this caused me so much headache that I need to post a picture of it Light cookies! Yay!

 

It's great to be proud of hard work!

I was also wondering you said you wanted to do a temporal AA solution, had you seen this presentation by Guerilla Games?
http://www.guerrilla-games.com/presentations/GDC2014_Valient_Killzone_Graphics.pdf

They talk about temporal AA but also some other great stuff like temporal resolution and frame reprojection.

 

Awesome! I'm really excited by this. Don't know how I missed it all these months. I'm the kind of guy who loves the Unity workflow, but love the graphics UE4 provides. This looks like it will bring the best of both worlds together I'm glad you're looking into better AA methods too - SMAA is the best available currently for Unity, TXAA looks promising, and DICE developed an MSAA solution for deferred rendering which might be worth looking in to. Keep it up!

 

Any ideas on price?
Until Unity 5 arrives, I could really use a fat G buffer and better PBR integration...

 

Actually, I've got more questions...

1.) Is it possible to have lit particles with this system? Would be pretty badass to have my dust particles or whatever using the shadowmap for lighting.
2.) Will it be possible to "share" shadowmaps between cameras? Like, have my weapon camera reuse the shadowmap generated by the main camera in order to have my first person geometry properly receive shadows cast by the environment?
3.) Can we manually tweak the distance of each individual cascade in the shadowmap? GTA 5 does this AFAIK, by having the highest resolution shadow map be fairly tight around the character, but the other cascades extend quite a distance away from the character.

 

Sorry to go on about hybrid raytracing again but I have found an approach that shows the technique running on a 750ti at 30fps with real time reflections,global iillumination,soft shadows etc

 

I saw that video as well. Even though it's pretty much unusable. If a 750ti goes down to 30fps just for the lighting in a small scene I don't think it's feasible. I could be wrong though!

 

Agreed. Hybrid raytracing IS the future.... of dedicated hardware. Which doesn't really exist yet.

 

I thought I had all the Killzone Shadowfall presentations but I must have missed this one, thanks!

I appreciate your kind words The Dice MSAA solution is too implementation specific for me to do it inside Unity so that's out I'm afraid. I'm going to start with a temporal solution and then work from there.

1. At this point no. I'm about to start porting Unitys particle shaders over to Jove though, and I'm gonna add a vertex lit version with full integration into point/spot/directional lights. At some point I'm doing my own particle system, which is going to be a lot more fancy but it won't be any time too soon (hopefully this year though).

2. I've had that request several times by now. It doesn't work currently but adding it would not take long, I'm putting it on the list of small stuff to add. I can't promise it will make it to release though, but should be there soon afterwards

3. Yes you can. You don't get exact control of the cascade split, although I could add that as well, but you do get to choose how it interpolates between a linear and exponential split, meaning you can have one shadow map that is 5 meters and one that is 500.

It looks great but there's a lot of things they're not telling you. For example, I see no mention of whether or not acceleration structures were used (which leads me to believe they were used), meaning that scene breaks as soon as theres a single dynamic object. Also, 30fps with no post processing is horribly slow. Jove runs at 300fps on my 560ti in a moderately complex scene with no post processing.

 

Also, I got an early Christmas present a few days ago. Here's a peak at a few fancy textures from
[Modular] PBR SciFi Pack by Jakob R. (http://forum.unity3d.com/threads/modular-pbr-scifi-pack.257239/) rendered in Jove. Finally actual textures and not just single color spheres

High resolution version: https://dl.dropboxusercontent.com/u/233290703/Spheres_Highres.png

 

Beautiful!

 

Man that is dy-no-mite!!!!

 

I forgot to mention that the dev has said that there is some bug that's making performance bad. This is still obviously not usable in games yet,but in another 5 years hopefully

 

Also the dev says here: http://www.gamedev.net/topic/655166-approximate-hybrid-raytracing-work-in-progress/ that it supports fully dynamic scenes

 

I shall wait patiently until release :3