Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Light Probes Questions/Feedback

Discussion in 'Developer Preview Archive' started by AcidArrow, Dec 22, 2011.

  1. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    659
    ok I got it, I need to add

    Code (csharp):
    1. float3 worldNormal;
    2. INTERNAL_DATA
    to the Input struct.

    It now samples it very nicely.

    Can anyone tell me what the 4th value that gets passed into ShaderSH9 is for? Setting it to zero seems to make it darker but it's still getting the lightprobe light.
     
  2. Kuba

    Kuba

    Moderator

    Joined:
    Jan 13, 2009
    Posts:
    416

    INTERNAL_DATA is a define that contains the tangent to world space matrix (three half3 vectors actually), so it is necessary when WorldNormalVector or WorldReflectionVector is used.
     
    Last edited: Jan 2, 2012
  3. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    The light probe coefficients are being packed in a clever way before they are set as shader uniforms. The packing allows for nice vectorization of the calculations that need to be done to get lighting for the direction of the normal you pass in. The "1" at the end has no interpretation, it's just there to make the result work and in our shaders the normal usually already has a "1" at the fourth position. So passing anything else there will give you incorrect results.
     
  4. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    659
    This is all good.

    Is there any way to decode specular highlights from probes? Or is there a decent approximation method I can use?

    Edit: I found this in a paper on the internets

    Do we have a view-reflection vector lying around?
     
    Last edited: Jan 5, 2012
  5. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    There's no built-in way, there's also no "correct" way to do that, but there are various hacks you can try.

    To calculate specular you usually need the direction of the incoming light and it's intensity. When there's a single direct light shining on an object, the shader has that information. But in a scene with GI where lighting is hitting the object from practically all directions, it's not feasible to take all of them into account. So instead you can cheat a little bit and analyze the interpolated probe to figure out which direction is the brightest and limit specular to only that single direction. Since it's a hack, you'll have to handle what happens when a different direction suddenly becomes the brightest one, to avoid the specular popping between those directions.

    Some engines (wink, wink) in the case of indoor lighting, where there's no directional light influencing the object, try to extract the "fake" directional light from the probe, remove this lights influence from the probe (that's just calculating the SH coeffs for the light and subtracting them from the probe) and then render that light for the object as if it was a normal directional light, including using it for casting shadows. Of course it causes every character in the room to cast shadow in a different direction and the shadows rotate or flip around them as they move, but then it's something that you try to hide with art and a different placement of the original baked lights.

    Or you can choose one global direction for the entire level, calculate specular for that and then modulate the result with the light from the probes - so in darker areas the probes will dim the spec highlights. That's even hackier, but easier to do.
     
  6. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    659
    I'm now using something like this, I dunno how much of a hack this is, but it gives what I think is nice looking shiney specularity.

    Code (csharp):
    1.  
    2. fixed3 worldN = WorldNormalVector(IN, o.Normal);
    3. fixed3 worldR = normalize(WorldReflectionVector(IN, o.Normal));
    4. o.Emission = o.Albedo*ShadeSH9(float4(worldN,1.0)) + o.Gloss*pow(ShadeSH9(float4(worldR,1.0)),2.5);
    The 2.5 is a hard coded shininess value, the higher I make it the smaller the specular highlights get.
     
  7. Chickenlord

    Chickenlord

    Joined:
    May 13, 2011
    Posts:
    381
    I'm just wondering, do the coefficients simply get applied per object or do you have some other "magic" way so that it works properly? And if it's per object, how does it handle really large objects (so that they usually should use multiple probes) and batching?
     
  8. DanTreble

    DanTreble

    Joined:
    Aug 31, 2010
    Posts:
    590
    Coefficients get applied per vertex. You can find the code in one of the cginc files.

    The average probe gets calculated per object though, this scheme isn't designed to help with big objects.

    If you have big static objects and lots of lights, use lightmapping. If you have big dynamic objects and lots of lights use deffered.
     
  9. Chickenlord

    Chickenlord

    Joined:
    May 13, 2011
    Posts:
    381
    Yeah i know, but thats the calculation stuff inside the shader. I was wondering how they get passed from the cpu.
     
  10. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    The interpolated probe is calculated per object, so on big objects the lighting will of course be under-sampled, but that's the only way to make it really fast.

    Still, if you need to to handle really big objects, you can either split the object into smaller ones (in some special cases seams between parts using different probes just won't be visible, if the object is naturally broken up like that anyway) or you can write a script that gets a couple of interpolated probes in some regular pattern for a given big object and then decode and interpolate between them yourself in the shader.
     
  11. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    How about Dynamic object like trees in terrain? when using lightprobe it's losing dynamic light from direct light (or any light source).
     
  12. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    If you use dual lightmaps, direct contribution from Auto lights isn't baked into the light probes and instead done real-time.

    Otherwise (when the light is baked only or if you're in single or directional lightmaps mode) full contribution of the light will be baked into the light probes, so the light won't be additionally calculated real-time (as that would cause double lighting).
     
  13. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    Uhh..something like this, maybe I said it wrong (somehow) on my previous comment so i'll speak visually.
    $Trees_and_lightprobes.png
    The big one placed manually it's receive direct light and lightprobe as well (but the leaves didn't receive any dynamic light, only lit by light probe). And...the small one placed using terrain engine and it didn't receive any light from lightprobe. And those two came from the same prefab.
     
  14. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    I just realized if you guys are interested in faking specular highlights why not just do it with a cubemap? Use reflective bump spec, allow it to use the light probes... but at the same time have a cube map applied that looks like specular highlights... Sort of a ghetto hack , but it looks pretty good. I actually think thats the way halflife 2 did it back in the day. Not exactly accurate, but still looks better then nothing.

    Oh but one of these shaders will still need to be modified to take advantage of the diffuse with light probes that someone else was doing in these posts...

    Would love it if someone could post a diffuse bumped shader that also has a cubemap support to fake spec highlights ... I dont know how to write shaders. You can even change the cubemaps as you enter different rooms by using different instances of the material based on the character and his location.
     
    Last edited: Jan 11, 2012
  15. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    The problem is not adding cubemaps, but having multiple ones and blending between them, like lightprobes are doing.
     
  16. Chickenlord

    Chickenlord

    Joined:
    May 13, 2011
    Posts:
    381
    Light probes are applied as spherical harmonics, so they can be passed to the shader as only 9 float coefficients (i think it was 9 right?). This can be done separately for each object without creating multiple materials (and i'd guess without breaking the static batching as well). This was btw. the reason why i've asked how the probes values get applied and then i found the Renderer.SetMaterialBlock method. Using a cubemap, as far as i can see, you'd have to create a separate material for each object using the cubemap light probes and it would have a waaaaaaaay larger performance hit. Of course it's possible, but the benefit would probably be to small to justify it.
     
  17. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    i think u guys are overcomplicating my idea... forget the probes themselves for the cubemapping part... i just basically said fake a specular highlight by making an all black cubemap with a couple of blinn or phong shines in it... an image based highlight if u will.. May not be accurate, but its better then nothing.


    by the way.... any word on that shader from shadow gun? lightprobes are cool and all, but i mean seriously whats the point if u cant utilize your normal maps with them on?
     
    Last edited: Jan 12, 2012
  18. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    The tree on the left: If I understood correctly, the trunk is lit correctly with both direct light and probes. But the leaves, once you enable light probe, stop getting direct light. If so, it's probably a bug? Could you submit a bug report? Just zip up the scene submit via the bug reporter and post the case number here.

    The tree on the right: One of the things we're planning to use the light probes for is getting much better lighting on the terrain engine trees, but we didn't have time for that for 3.5, so it'll have to wait for one of the upcoming releases. The idea is that for a tree it would be very good to bake one (or several) probe exactly where the tree is, but with that tree 'taken out' (you don't want the tree to darken it's own probe, but you do want the surrounding trees to cast shadows on it). That requires support from Beast itself, because it's not feasible to bake the scene as many times as there are trees in a forest ;) So yea, stay tuned.
     
  19. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    You can do that and it will look pretty good, although of course the spec highlights will be coming from some seemingly random places, i.e. brights spots in the cubemap that don't correspond to actual light sources in the scene. But if you do a nice cubemap, no one will notice ;)

    To do it, you can just use the built-in reflective shader (that will give you per-vertex light probes) or you can combine the per-pixel light probe shader I posted in this thread earlier with the built-in reflective shader.
     
  20. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    Very true. Most games either just use one cube map for an entire scene, or switch between them (which gives a pop) or blend between two when transitioning (which can give a pop when there are more than two close by). It's basically a similar problem to light probe interpolation, with the difference that it's not a problem to blend 3 or 4 SH probes, but it is a problem to sample 3 or 4 cubemaps in the pixel shader.

    But maybe we'll come up with some nice solution to that at some point ;)
     
  21. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    It's 9 coefficients per rgb color channel, so 27. But they are pre-chewed before being set as shader uniforms, so they end up 7 x float4. There are some details in the 3.5 scripting docs for LightProbes.coefficients.


    Not really. You can't change gpu state half way through a batch and gpu state includes shader uniforms, samplers, etc. So, because two different objects will get a different interpolated light probe, they need different shader uniforms and they can't be batched - they behave in a similar way as if they had different materials. Same goes for two objects which want to use a different cubemap.
     
  22. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    Rej promised to put it up and I'm sure he will. You can poke him on twitter, but he's a really busy guy.

    Why would you say that? I posted a shader using light probes per-pixel earlier in this very thread. With pictures and everything.
     
  23. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    It's not that you can't. You can. There just isn't a shader that comes with unity that does it.

    (although, I wonder, why isn't there one built in? I mean even the one robert posted a few pages back would be fine and there would be less complaining all around)
     
  24. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    I'll just add a surface shader keyword perpixellightprobes or something like that, but not for 3.5.
     
  25. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    @robert : Done, Case 438105
    and there's a typo on it, sorry it's 2 am here.
     
  26. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    hmm... i saw a package on the asset store that, and correct me if im wrong, does light probes... with normal maps and spec... and even has realtime shadows working with the lightprobes and all that on...

    The only problem is it says not supported for mobile.

    There light probe system also seems to have much more features. I think unity team just got outdone by their own community... lol

    http://forum.unity3d.com/threads/116129-ShaderBox?highlight=shaderbox

    theres the link... so correct me if im wrong, but this seems to be everything that people want.
     
    Last edited: Jan 12, 2012
  27. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    Which is?
     
  28. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    Well... from what ive been reading ..

    1.)being able to control the colors of the probes themselves.. not just baking them.

    2.)The realtime shadows working with them..

    3.) the normal maps working with them... (spec.. dont know for certain, but from reading their pdf file it seems like it does have in fact have it.. along with a bunch of other features in the shader for it.)

    4.) Tools to layout the probes automatically. (Which i dont really care about but seems like a really nifty feature.)
     
    Last edited: Jan 12, 2012
  29. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    1.) There's API for that (LightProbes.coefficients), so it's easy to make a tool for painting/adding additional light into the probes. A free idea to anyone writing tools for the Asset Store ;)
    2.) They do in dual lightmaps. What other setup would you like to have?
    3.) They do work with normal maps (again, take a look at the shader I posted earlier, it's two lines of code you have to paste in). And ShaderBox doesn't take spec from the light probes either, they take it from a cube map and we discussed it here already as well.
    4.) That will be built-in in a future release. In the meantime it's possible to write a script using the nav mesh or other knowledge about how your characters move in your scene and place the probes there.

    All in all ShaderBox is neat, but their light probes are a super basic temporary solution while waiting for the 3.5 built-in probes.
     
  30. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    I dont know if the issue was commented on, but here is what dual mapping does with unitys probes.... the next image is a comparison of with the probes and without em... perhaps there was a workaround.. could have sworn i read one somewhere. Notice how with the probes the object utilizing the probes changes colors to be way darker and not match up.

    Now that i really look at it ... the probes for the directional light mapping makes the object super dark compared to the baked lightmapped... check out screenshot #2 for that one...

    Really at this point im not even sure im using these light probes as unity intended. I mean it was my impression that these things in a way would replace the lighting information from lights. And if not that contribute to ambience to the model when reacting with real lights. In which case i would not know how to set that one up with this pipeline since the lights sort of get disabled on bake... would be difficult. Perhaps you guys only intended for us to use lightprobes in dual light mapping mode with a super far shadow distance.
     

    Attached Files:

    Last edited: Jan 13, 2012
  31. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    Aside from that ill see what i can do about the shader and your code example, but im no shader writer... im lost the moment i open it.
     
  32. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    Now that i have been messing around with it.. i think the best workaround for unitys probes and directional lightmap is baking the scene with directional, switching to single light map... baking the probes with single lightmaps, then switching the mode back to directional lightmaps...
     
  33. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    659
    Here is my current BumpSpecProbed shader, you'll have to play around with the three Probe Spec parameters till you get a nice specular effect from the probe light.
     

    Attached Files:

  34. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    @Zomby138 hell yea dude you are the man!
     
  35. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    659
    Glad to help :)
     
  36. Stjohn909

    Stjohn909

    Joined:
    Aug 19, 2011
    Posts:
    35
    The lightprobes are awesome! And Zomby138 thanks for the shader, it's great! I'm kind of going about this a little backwards, I'm generating lightmaps as EXR's in Houdini, then bringing them into Unity through the lightmapper interface. But in order to bake the probes to fill them with any values that work, I have to plug the lightmap EXR image into an emissive material, assign that to my object, bake the probes, then return to my original material. Once the probes are baked with the emission, they work on other objects in the scene. Is this the way I have to do it if I bake my lightmaps elsewhere?

    Sequence is as follows:
    Lighting the probes with a self-illum shader with the lightmap as texture:


    Room object returned to diffuse shader/lighmapping, not using probes. The crate is using the probes.


    Here's the crate close up:


    This is probably a really roundabout way to do things, I know, but it sort of works.
     
  37. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    It looks like you're using deferred. If so, this bug has been fixed for beta 7 (and the fix will soon be available in the public version). What you're seeing is that behind shadow distance the object is no longer getting direct light, only indirect light from the probes, which is incorrect.
     
  38. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    That's one way to get indirect light into the light probes, but you won't get direct light that way. The "proper" way to do it is to setup the same lighting in Unity, but unless you write a script that can export the setup from Houdini and set it up in Unity, it's a lot of manual work. So - whatever works for you ;)

    The important thing to remember is that starting with 3.5 Unity has a very fancy way of filling out the unused space of a lightmap to minimize any bleeding you might get at the UV seams, especially with lower mip levels. The unused space has to have a negative color for Unity to recognize it. Right now it's just "< 0", but maybe we'll need to change it near future to be lower that some negative number, so stay tuned. In any way it would be good if you could make Houdini output negative background. Otherwise you'll have to make sure yourself that the unused space is filled with some colors that make sense.
     
  39. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    Ok, thanks. With the manually placed tree it was a bug and I fixed it for the next beta. So when you enable light probes on that tree, both the trunk and the leaves will get both the light from the probe and real time light from the directional light.
     
  40. Futurerobot

    Futurerobot

    Joined:
    Jul 13, 2011
    Posts:
    179
    I was wondering about something related to the probe data too. It seems kind of hard to see which probedata file a probe component refers to. And sometimes this causes mixups.

    We have a couple of levels in our game, and sometimes we have to branch a level off for testing. Or maybe we just need a duplicate to create a variation of the same level. However, when duplicating a level (or saving it with a new name), it seems that the probe data in the scene still points to the same file as the original level. Making any bakes apply to both the new and the old level. Which is a problem when the new level has dramatically different lighting, and we'd rather not have to recreate the probes for the entire level, when it's for the most part the same.

    Is there any way we can force a probe system to write to a new data file?
     
  41. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    The LightProbes.asset is always written into ./sceneName/LightProbes.asset, assuming the current folder is the one the scene is in. So if you branch off a scene, i.e. save it under a different name or path and then bake, new light probe asset (and new lightmaps) will be created in the folder next to that scene. So the original light probe asset won't be overwritten, even if the copy of the scene was still referencing it before the bake. After the bake it will reference the one in the correct folder.

    Also to avoid having the light probe asset overwritten, you can just rename it or move it to a different folder, but in your scenario that is not needed.

    Finally you can check which asset is being referenced by the scene by looking in the Maps pane of the Lightmapping window.
     
  42. Futurerobot

    Futurerobot

    Joined:
    Jul 13, 2011
    Posts:
    179
    Thanks, it's working now! Though with some modifications.

    It seems that it doesn't automatically refer to a new lightprobe asset file when saving a new scene. When saving to a new scene in a new folder it will still default bake to the lightprobe asset from the previous scene, we can see it's still referred to under Lightmapping - Maps. If we set the lightprobe asset to None before baking in the new scene, we get a new lightprobe asset file created with the scene-name and placement etc.
     
    Last edited: Jan 18, 2012
  43. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    Damn, sorry for the confusion - you were right and the bake did overwrite the referenced light probe asset, which was an incorrect behaviour and a bug. I just fixed it for the next beta, so now the behaviour should be exactly as I described in my post above, i.e. after the bake only the asset at default path for the current scene will be written to, no asset at a different location will be touched.
     
  44. Stjohn909

    Stjohn909

    Joined:
    Aug 19, 2011
    Posts:
    35
    Robert, thanks for your response.

    I'm still exploring the abilities of Houdini's UV rendering implementation, but I believe it's possible to fill the unused space with something other than zero.

    At the moment, I can export light position, direction, FOV, color, and intensity to Unity, so that's not a problem. I don't mind baking the probes with direct light from Unity, but I'm trying to achieve as close to 1:1 between Houdini/Mantra and Unity as I can.

    Also, am I limited to Beast's light probe functionality or if I have my own method, can I write light probe information to a file usable in Unity?
     
  45. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    I would imagine you can get closer to what Beast would produce by setting up the lighting you had in Houdini, but if can't get it to work, then at least you have your hack with emissive materials :)

    You can place light probes in Unity and bake them. That will create the LightProbes.asset, which contains all the spatial information plus the light information. The latter is stored as SH coefficients and can be modified with LightProbes.coefficients. So if you bake your probes at the same positions in Houdini and encode the results to SH, you can just write those coefficients into the Unity light probe asset and everything should work nicely.

    To have a look and mess around, you can select the baked LightProbes.asset and switch the inspector to debug mode - you'll see the modifiable array of coefficients.
     
  46. thempus

    thempus

    Joined:
    Jul 3, 2010
    Posts:
    61
    Is there any plan to make the process of baking light probes with custom light maps(not beast) easier in unity 3.5 or future releases?
    Would be good if the "hack" that Stjohn909 suggested could be implemented properly in unity instead leaving it open for the user to create their own process, or most likely wait for someone to release a tool on Asset Store.

    Is the workflow with custom light maps fully considered by unity, or is it just a consequence of the beast workflow?
     
  47. alexzzzz

    alexzzzz

    Joined:
    Nov 20, 2010
    Posts:
    1,447
    Is there a way to create new light probes in runtime?

    PS
    After messing around for a while, I guess the answer is no, and even if it were yes, it might be not a good idea because of performance.
     
    Last edited: Jan 21, 2012
  48. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    The proper way to do it is either to mirror the lighting setup from the external in Unity or to bake the probes in the external tool and then apply the SH coefficients. Both options are fully supported by the API.


    Beast doesn't influence the workflow at all - Unity just exports the scene to Beast and gets the lightmaps in return. So what you see is Unity's take on making something work in an integrated way but also allowing for control via the editor or scripting. One thing that could be potentially improved is how the rendering engine understands when a light has been already baked. If it hasn't, it needs to be rendered realtime. If it has, probably not anymore (depening on the lighting setup). That can potentially cause problems when using an external lightmapping tool, but is not hard to workaround.
     
  49. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    Quick Question. Is the lightprobe use the ambient light value?

    Edit: it's a yes, weird is there any ways to make the lightprobe not affected by theAmbient light? It's make the character brighter than the environment.
     
    Last edited: Jan 31, 2012
  50. Deleted User

    Deleted User

    Guest

    Hello all! *waves*

    I just wanted to chime in really quick and mention another way of getting specular lighting with SH as a simpler alternative to Zomby138's shader. MK vs DC Mortal Kombat 9 both used a method where they basically had the view direction act as a specular-only directional light (ie. pow(dotNE, specularPower)). They then colored the light using the diffuse ambient lighting sampled from their SH probes. Finally, they used standard material parameters for adjusting the lighting (ie. specular color power).

    Sadly, their technique has some limitations, as it has a limited range of specular powers before the highlight becomes too small and kills the effect. As a result, their system doesn't integrate nicely with dynamic local lights that have proper specular. Finally, it works better for round objects and normal maps with rounded details, and is much less effect for largely flat surfaces.

    Bottom line, it is dirt simple and dirt cheap if all you want to do is add a glossy effect for a system dominated by ambient lighting.
     
    Last edited by a moderator: Feb 3, 2012