Jove 2.0 - Replacing the rendering pipeline

Nope it is still broken and I have the latest drivers. There is definitely a glitch in your shaders as I also tried it on my surface tablet and it completely crashed the intel graphics driver on it and had to be recovered on windows 8.1. Perhaps it's an issue that only shows up at high frame rates with the nvidia driver. On my computer it looks like something is going wrong with the lighting about every second to start and then gets worse over time, thats why I though it was the frame buffer being force cleared by the driver. It might actually be something going wrong with your lighting once a second as that would also make it go black I guess.

It crashes in about a millisecond on the intel driver. It crashes almost instantly and then windows 8 restarts the graphics driver and refuses to execute the web player due to a fatal graphics error.

 

Crashes for me, too.

 

It's not surprising that it crashes on intel graphics. The demo runs a pretty heavy compute shader first thing that happens to generate a look up table. I have no idea why it crashes your main computer though, a gtx 680 should have no issue with it. I've tested it on several computers and I can't reproduce the crashes : / Has anyone succesfully run the demo? Oh, and carking1996, what hardware are you on?
Also, saying this again, the demo requires DX11, I have no idea what happens if you try to run it on something that doesn't support DX11.

 

I have a windows 8 PC, so I'm pretty sure it is DX11. I have an AMD A8 processor. And an AMD Radean 7640G card.

 

Unfortunately your card might be too slow to generate the lookup texture. In the final version Im going to include a precalculated texture instead of calculating on load. The problem is that if the calculations take too long time Windows thinks the driver has crashed and reboots it.

 

What graphics cards have you tested it with Aieth it's strange that you don't see it. Maybe you should try changing the demo to instead of moving the light automatically have it where you can type in values to fix it's position. It doesn't crash the nvidia driver which might be why you are confused. The only reason I can think why you wouldn't see it and I would be if mine computer is generating more frames per second than yours and so drawing the light at smaller intervals or you have a very low refresh on your monitor. Perhaps your computer just isn't generating the frames where the position of the light messes it up. That's why I wanted to try it on the intel graphics. I assumed it would run as it supports directx 11 just at a much lower frame rate. Perhaps it's a precision error ? Are you using a sin or tan function that is producing an infinite value or something to mess up the lighting calculation at certain angles ?

 

I've tested it on three separate GTX 560 ti's Does the error look like what boomsma1995 posted? I'm pretty sure the error has nothing to do with the code. I mean, it works on three different operating systems (Windows 8, 7 and vista) on three different computers, all with the same graphics card though.
The only reason I can think of is either some kind of driver bug. That picture almost looks like overheating artifacts, but that makes no sense on a GTX 680.
It also seems that for some reason some systems are more sensitive to the small hickup when precalculating the scattering, as in the webplayer crashes. I wondered if there was a need for storing the lookup tables offline instead of generating them at runtime, I guess this proves there is a need for that

EDIT: I think I might have tracked down the issue. Working on fixing it now, I'll upload a new webplayer later today

 

No the error on my card doesn't corrupt the frame buffer like that it just constantly flickers black like it is force clearing it to prevent a buffer overflow.

 

Alright new version is up and running! This time the lookup textures are precomputed offline, so it should at least run on all systems Please do tell me if it doesn't work. Requires DX11
https://dl.dropboxusercontent.com/u/233290703/ScatteringDemo.html

 

It works and looks great now

 

It works fine for me to as well. Are you planning to make it so that you can combine that dynamic lighting with a skybox cube map ? I would love to be able to render a realistic sky with clouds in vue extreme and then combine it with your light scattering so that you could use one cube map but with dynamic lighting. Also how are your transparent material shaders going ? That dynamic light scattering would look impressive alongside that glass dragon that a lot of demo's seem to use.

One suggestion would be to look at applying filtering to your dynamic shadows to remove the flicker though if possible or perhaps blend two frames of shadows together to remove the flicker as they move. I would have thought that better dynamic shadows would be a compelling reason to use Jove's lighting over Unity's.

 

Yeah there's no clouds at the moment, it's planned though. I'm planning to allow a night skybox. I'm not sure I understand what you're saying about skyboxes? You mean only using the scattering part but use a skybox for the actual sky? Or do you mean multiplying the dynamic sky by a skybox?

The filtering is actually turned off for this demo, I broke it while fixing a bug. It's gonna be fixed, don't worry

 

What I mean is have some way to be able to combine your dynamic lighting as you are using there but still be able to include a skybox to help with the image based lighting. Multiplying would be one way to do it I suppose or you could create a shader that expects a cloud mask so that that the clouds affect the scatter as in real life.

In a nutshell what I am getting at is in all your previous demo's you used a skybox with all your demos and so would have reflections of clouds or other objects interacting with the pbr shaders.

Most games are still going to require baked sky boxes for the backgrounds. It's image based lighting based off these skyboxes alongside pbr shaders that is responsible for most of the graphics improvements on next gen consoles.

For example suppose I had a game with a mountain in the background covered in tree's. Those mountain and tree's would need to baked into a skybox rather than all be polygons. If someone wanted to do a dynamic sunset over that mountain do you have something planed where your dynamic lighting and scattering would blend with it. For example at night you probably wouldn't be able to see the mountain or it should be pretty much black.

 

I disagree with most games requiring baked sky boxes. In fact, most games released recently with great graphics use dynamic solutions. A cloud mask would be easy to implement though (clouds are going to be supported).

If I understand correctly, would you want an option for a skybox that blends (say based on the alpha channel) between what is in the skybox and the dynamic sky? So that you could have far away mountains affected by the scattering and then at an angle it switches to the dynamic sky? That could work, but it would be a little more taxing (3 texture lookups instead of 2 and a lerp), but not by a lot.

 

Do you have anything showing Jove with interior scenes?

 

Not yet unfortunately. I've had my programmer art of spheres and boxes to test with We are in the process of building a better test environment though, so with a little luck I'll have interior screenshots soon.

 

Aieth, I've been hugely interested in Realtime lights, area lights, IBL and GI. And really, I'm hoping the best for this system, I think you have made a very wise decision by not working in the Unity Lighting Box.

With that said, I have seen through the last six months some promising realtime lighting solutions come through the asset store, they where expensive and released in very rough BETA with the promise of, with time, complete functionality... only to fade into the abyss of non-response or sorry I must make a living and can no longer support this product.

I do believe that many have hit insurmountable obstacles within the Unity framework and have gone too far down the road to turn back.

Aieth you don't owe me or anyone any explanations, but what kind of assurances do we have that this won't end up in the abyss and why this time it will be different.

I say this will all respect...

Thanks,

A cautious potential customer

 

As an added question... How will decal systems work with this since you are making your own shaders...

 

I completely understand your reservations. There have been plenty of projects promising gold and either delivering buggy unusable betas or just fading away and the creator disappearing. I'm not sure exactly what I can do to assure you, except tell you that I have no plans on stopping development

No existing system is going to work without a hitch, that's almost a guarantee. It would require either modification of said systems or you could just use the screenspace decal system I'm planning That's a bit further down the road though.

 

What about using dual parabaloid reflections instead of cubemaps? It means if you want dynamic reflections you only have to update 2 textures instead of 6. Method:http://graphicsrunner.blogspot.ie/2008/07/dual-paraboloid-reflections.html?m=1

 

If you were to render a dynamic cubemap every frame and use that for reflections, that would be an excellent choice. I'm pulling the system in a direction where that won't be needed. Even if you "just" have to render two faces instead of six, it's still three times the drawcalls. PC as a platform, and Unity in particular, is no good at high amounts of drawcalls. A console game can afford thousands due to a tighter more efficient pipeline. Thanks for the link though, and if you find anything else interesting please don't hesitate to post about it I really appreciate it!

 

I realise its not capable of being updated every frame but it could be used to solve your differences in time of day. Also according to nvidias blog this is the technique used by watch dogs for its real time reflections. So you never know,maybe if you only update the nearest 1-2 cubemaps it could work

 

The issue isn't the rendering though. That's pretty cheap and can be spread out. The issue is the filtering that has to be done, with IBL you usually use the mipmaps of the cubemap to store "blurred" versions. That's the expensive part and that can't really be solved, so the "solution" as such is to scale and modulate the cubemap according to time of day instead of creating new versions.

 

I saw this : http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter10.html
It's realtime cubamp convolution using SH.
If it can help.

 

I can't believe I forgot about convolooution but as a previous poster said there are she techniques for that but only work for diffuse really. I have recently stumbled upon another new interesting technique. Its called hybrid raytracing. I did my best to follow the description but correct me if I make a mistake, basically what they do is take the g-buffer that is created as usual and use that as the input for the raytracing. Through this you can raytrace soft shadows,global illumination, reflections etc all in real time on mid end hardware. Here's a link:http://gamasutra.com/blogs/Alexandr...tical_techniques_for_ray_tracing_in_games.php

 

I've been using that technique, but it unfortunately only works for a lambertian diffuse (N * L). For a microfacet BRDF the math isn't as simple anymore. Sorry for the late answer, I've been busy this week.

I've seen hybrid raytracing before and it is a really exciting subject. I totally believe in it being the future, but I find it hard to see it being used today. Especially when it comes to stuff like GI and reflections the runtime cost increase exponentially... I'm probably going to be one of the first people to jump on that bandwagon but not before I see proof it works in an actual game environment

 

Yeah its really cool. I was thinking about implementing something like it myself(not in unity only have free). I have also seen a rival of yours who is using hybrid raytracing called spectra GI by livenda the makers of candela SSR. Can't wait for this technique to become the norm as I'm so sick of having to fake everything with cubemaps and lightmaps etc.

 

is spectra gi more than just a big dark hole?
i have not seen anything but just 3 sec of a more or less static scene, so i would never rely on that before it has proven that it is production ready.
jove 2.0 however looks very promising. i just wished that it would not be designed around dx11.

lars

 

Perfect worlds and all that, Aieth said from the outset it was aimed at very strong computers with no real compromise - im guessing he meant 'best everything he can get together' and dx9 is not going to cut it whatsoever when anyone with a machine like this development was intended for probably hasnt been anywhere near a sub-dx11 card in god knows how long (i know i havent). I'd rather have more concern that Unity 5 or/and this produce some well performing choices for both directx and opengl targets so the macs arent arbitrarily excluded. I don't see any way round the dx11 thing though

Agreed on the spectra thing btw, im not even sure there was any movement in the scene at all? the tactic of making a fancy promo vid, include 3 seconds of what could be from *insert max renderer of choice* is to me pretty dodgy but hey. Jove seems to be pretty honest and its development publically visible and uses stuff that can be directly corrolated and understood once having read literature on it a few times, which to me is a bunch more valuable than a black box till launch day, especially if they try to play people like it seems candela was being milked

 

Yeah that is what is making me wary of that product. I hope they give us a video of it in motion,and also tell us what graphics card it runs on. I don't want to find out after purchasing that it requires a gtx 780 ti/titan.

 

I gave into curiosity and bought Candela and after a bit of experimentation to patch up the techniques' shortfalls (quite blurred box projected environment maps did the trick in an acceptable way) it's really quite nice so i know they can make some nice things but everything has such an air of snake oil about it until its proven because of the way it's promoted that it seems like a con job. Put bluntly, advertising like that is cheap and tacky, practically valueless as the product itself is visible for 3 seconds in a shot that could have come out of anywhere and slightly insulting, like a userbase is too dim to see through a few minutes of aftereffects and be informed or excited by it on a qualitative basis.

I already have Jove, i'm hoping it was a great investment cause it would be pretty great to have such a solid and ambitious option open and freely informing possible customers of what is being made and its progress is very agreeable. That's what i consider decent promotion as its software being sold, not diamante jewellry

 

I agree. We're moving away from faking and heading towards the real deal, but we're still many years away from a fully functional real time path tracer. I can't wait for the day when cubemaps are not necessary anymore

I don't want to badmouth anyone but I just don't see path tracing being done on todays hardware. Sure, I could be mistaken, I wouldn't mind a real time lighting revolution. The reason for going for DX11 is that it gives such freedom. I mean, it isn't even funny to compare DX9 and DX11. Had I limited myself to DX9 (Unity OpenGL) it would've been ridiculously hard if not impossible, no wait, actually impossible, to deliver this kind of graphical quality. That said I do want to supply a very limited DX9 version, so that games developed with Jove can still be run on older computers. That version won't have many of the fancy features though, and will perform a lot worse, so it's not a perfect solution. Compute shaders are so useful it's insane.

Something like best everything I can get together, with the added criterion that each individual part can't take more ms rendering time than what is motivated. I could easily go crazy with the shadow mapping and make the best god damn shadows you've ever seen. But then that would be all you get And at 30 fps with standard Unity shaders and no post process. As I said above when replying to @larsbertram1, I do want to include a DX9 compatible version as well just so games can actually run on DX9. It won't be near as pretty but at least it is gonna run.

I hope you will find it to be a great investment I'm hoping to release an alpha in July, but I can't make any promises. I'm not going to release a half assed product that can't be used. It is one thing to lack features but still have a workable foundation and one thing to have a foundation with cracks everywhere that you can fall through if you don't have great knowledge about the product. Which nobody but me and my team has at this point
For example, take the dynamic sky, it's going to be in the alpha release but clouds are unlikely to make it. You can of course use a skybox instead but if you want the dynamic sky it is gonna work, but without fancy features like clouds and weather effects.

 

Haven't heard anything in a while... so how goes it?

 

It goes well, thank you for asking. I've been working an internship at a media company playing around with OpenCL all of June. Working fulltime has definitely slowed the development progress of Jove I'm afraid... But the good news is, today is my last day and I now have four completely free weeks ahead of me. I'm going to make a push for a public alpha release during these weeks (expect it towards the first days of August).

In what time I've had, I've been working on getting clustered shading up and running. Even with unoptimized, unthreaded (and generally pretty crappy) culling, 500 point lights on the screen at once and I still run at over 100fps with full postprocess on. I tested how many point lights I could use with a standard mesh approach (like all DX9 engines do in deferred mode, including Unity) and it choked to a halting 10fps with 50 lights. So the future is looking bright, pun intended

I've been thinking on how I'm gonna prioritize features, as I won't be able to get everything in before the alpha release (hence why it is an alpha). I'm thinking the priority list is going to be as follows

1. Clustered shading
2. Bloom
3. Localized IBL (currently doing global cubemap)
4. SSR
5. Local fog (in addition to atmospheric scattering, local fog needs more precision and volumetric lighting than far away stuff does)
6. Virtual camera* (bloom really goes here as well, but as it is a more prevalent effect I bumped it up).

Stuff that's definitely not gonna make the first release is skin shading, hair shading, terrains, point light/spot light shadows, clouds and a few other secrets that I've yet to unveil but are in the works. Now, how far I'm gonna get on that 6 item list is anyones guess. I have to say that it's largely down to chance, with a little luck each item could take just a single day each, or they could take two weeks (thinking specifically of point 4 & 6).


*Virtual camera is what I am calling basically simulating a cameras imperfections, such as DoF, bloom, vignetting, chromatic aberration, motion blur etc.

 

Sounds exciting, I work primarily in 3ds Max and have grown accustom to lighting in that environment, i.e no light probes etc.

Will this be comparable to how a 3d package like Max, Maya, Blender works where any light lights any mesh dynamic or static?

Also would you be able to enable/disable meshes from certain lights to keep from casting multiple shadows?

Also can't remember... did you discuss light emitting shaders?

 

I have never used any modelling packages for anything but modelling (aka, no lighting), but I assume they work by placing lights and then numerically integrating them? (As in ray tracing a bunch of samples and then dividing by the amount of samples). If so, then yes, kind of, Jove is a step towards that direction.

All lights are indeed going to affect all meshes and static prebaked lighting is a goner.

As for shadows, pointlight/spotlight shadows are not in as of yet but the way they are going to function is that you can per light specify a layer mask to cull meshes from being rendered.

As for light emitting shaders, I'm afraid that is still in the realm of "only possible on offline renderers". You can of course have glowing materials however, but they don't light up their surroundings the same way a light source does.

Here's 500 point lights and full post processing running at over 200fps (less than 4ms per frame). All point lights are fully dynamic, nothing is prebaked, and all point lights have full specular reflections. Please also note that because lighting is decoupled from mesh rendering, the lighting would not become more expensive even if this was a scene filled with meshes. So if you aim for 60fps you still have 12ms to fill with *just* rendering meshes. This is on a GTX 560ti, so hardly a high end card in todays market.




And just because there's been a long time since I posted any beauty shots demonstrating what stuff can actually look like (instead of just technical visualizations like above), here's another picture of the Cerberus gun (I promise I'll get other stuff to show off soon )

 

The performance of that clustered rendering sounds great. You should post a web demo lighting up that gun with those lights and animate the lights so you show how well they work with the pbr shaders on that gun. Any idea when you will be making the first release of it ?

 

I'm gonna push an alpha version out before August 11th.

Here's a webdemo, it's pretty heavy at 50mb due to the Cerberus textures (the gun), and for some reason the controls work horrible in a webplayer (but fine in standalone) so be careful with the camera You control it like you control a standard Unity scene camera.
Requires DX11 and a capable graphics card

https://dl.dropboxusercontent.com/u/233290703/ClusteredDemo.html

 

Aieth is there any specific non-programer art you could do with to help show things off? I've got a bunch of hard surface kitbash items and want an excuse to make something out of them (in fact a game itself by a pal could do with models so 2 birds, one stone) and it's very hard to see the relevance of your numbers when I see very little game-related content to test them. Like, 200 lights is nice but how many things do they illuminate, what's the complexity of these things and so on. Not promising anything but yes if you can specify what you might need in this regard i might be able to offer some better context to all this

 

If you are saying that you have models I could load up and take pictures off, sure I have access to other stuff for testing but nothing I can show off as of yet.

As for the complexity, the system as it is now allows up to 500 point lights on the screen at once (not in a level, but on screen) and 32 spot lights. As it is deferred, the amount of lighting calculations correlate directly with the amount of lights and not scene geometry. This basically means that lighting does not become more expensive no matter how many meshes you have.

 

Well I don't have models, more that I can make them, if you can think of specific things that would be relevant to go by for measuring the performance and visual quality. I've recently purchased substance painter and quixel suite, neither of which has had a proper shout either, both of which are aimed at physically based wotnot, so it's not really a matter of creating useful models in terms of poly counts, and more shading itself.

 

I've kept an eye on both of those products and they seem incredible, unfortunately I have yet to have time to test them. If you feel like making models, I'd much appreciate it As for specific things, everything metallic tends to shine in PBR so if you feel like modelling some kind of vehicle, robot or weapon of sorts, that would be awesome.

 

Can you give a summary of the lighting models that might be accounted for at the moment? Am I just looking at something similar to U4's default lit? Or for previewing cases, which shader product you'd find best to work with? I suppose I could use Jove 1.0 if that would be most convenient but i'm familiar with Lux and Skyshop too. I suppose finally i'm interested in how animation is accounted for, but yeah it's nice to have some motivation - i'll start up Max and finally get to learn Quixel via some kitbashed environmental stuff. Hopefully my concentration will keep up and i'll make something worthwhile

 

Lux does a Cook-Torrance right? Should be fairly similar in lighting model to what I am doing, although the roughness is almost surely handled differently (we've been tweaking it a lot, I'm working with a company I can't name at this point). UE4's default lit should be similar at well.

I'm only using a single lighting model for specularity, and that is the same lighting model that UE4 uses. For diffuse, we've got a standard lambert and a very simplified oren-nayar.

 

That all sounds mighty. I'll go for Lux, then anything I do can be used for Lux bumping too. Lux uses a roughness value I think based on work by sebastien lagarde, i'm sure it uses GGX for the microfacet model anyways, I was digging around to get an ON diffuse model in there for both forward and deferred some months ago. I'm not sure if you're going about roughness/glossiness the same way but I guess it shouldn't matter - i'll consider some decent rough diffuse surfaces for ON

 

The performance of that clustered rendering runs great. It even runs fine of my surface pro tablet. Would this be suitable for attaching to a particle system or is it very expensive to add or remove lights with clustered rendering. I'm making a game which has particle splines I was wondering whether this would be suitable to emit lights from the spline ? Also is the performance the same with clustered rendering whether the mesh that is being lit static or dynamic ?

 

To be honest I'm not quite sure myself, I was not the one who made it I'll prod him and see if he can explain it in a little more detail.

Cool It is super cheap to add or remove lights, everything is completely dynamic. It makes no difference if the mesh is static or dynamic (of course, it still makes a difference while rendering if Unity can batch drawcalls, but for the lighting part it does not matter at all).

I'd say yes, it is suitable to emit lights. Of course, you'd want to pool it and reuse them to avoid garbage collection. It depends a little on the size of the lights and the amount. If you want a lot of very small lights you could probably even bump up the built in hard cap from 500 to a few thousand instead.

 

Amazing stuff, just tried the clusterd shading web demo, looks really great!!
I'm looking forward to see more!

 

Thanks


Bloom went a lot faster than I had expected, so I've already checked off 2/6 things on my to do list (posted a few posts up). Here's three comparison images, the first one if with bloom at max, second with a more moderate bloom and the third one has bloom turned off. Note that this isn't your typical game bloom, it is based more off of a bunch of optics math than it is based off traditional bloom algorithms. It is supposed to model imperfections in a camera lens (or dust accumulated on it). Thinking of adding a dirt texture or something to it as well.

 

That's actually very nice and yep that kind of bloom would be consistent with a mucky lens or imperfections in/lack of lens coatings - maybe even the lens needing a hood in those lighting conditions. It looks like 'proper' bloom, not just blowing out highlights with some additional texture treatment (Not that i'm an authority on post effects).

If possible, and as it seems you're going that way anyways, would it be possible to simulate the different conditions bloom can occur, including the lack of lens hood? (Which is probably the chief reason it does occur, sunlight falling directly onto the lens) Being able to provide a texture that affected both the cumulative colour as the light passed through lens dirt/imperfections and supplied a blurriness value too (I guess your average RGBA texture, maybe encoding colours into R and G (i have an article) so B and A can be used for alpha and blur) would be great, as someone who dabbled with cameras a bunch over the years it would be really nice to not only represent a mucky lens but to simulate some classic photography effects like smearing vaseline on the front element - I'm getting well into my VR stuff but although that implies no lens, i cant be drawn away from the allure of a decently realised lens as integral aspect of a renderer