Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Normal map shader capable of batching?

Discussion in 'Shaders' started by _Petroz, May 28, 2012.

  1. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    Hi, I know everyone is sick of threads about batching. The one that popped up recently reminded me of a problem I am having with batching.

    I have many small (and low poly) objects which I want have normal mapped, and dynamically batched, but I'm having trouble finding a shader that works. I started with "Mobile/Bumped Specular", that does not batch. When I switch to a simple shader such as diffuse I get batching.

    I don't really need the main texture or vertex colors. I cannot work out why this shader refuses to batch. I read the docs: http://unity3d.com/support/documentation/Manual/iphone-DrawCall-Batching

    I cant figure out what I'm doing wrong. Does normal mapping require multiple draw calls?

    Here is my modified shader. It is using the the main texture for both the color and normal map.

    Code (csharp):
    1.   Shader "Example/Diffuse Bump" {
    2.     Properties {
    3.       _MainTex ("NormalMap (RGB)", 2D) = "white" {}
    4.     }
    5.     SubShader {
    6.       Tags { "RenderType" = "Opaque" }
    7.       CGPROGRAM
    8.       #pragma surface surf Lambert
    9.       struct Input {
    10.         float2 uv_MainTex;
    11.       };
    12.       sampler2D _MainTex;
    13.       void surf (Input IN, inout SurfaceOutput o) {
    14.         o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;
    15.         o.Normal = UnpackNormal (tex2D (_MainTex, IN.uv_MainTex));
    16.       }
    17.       ENDCG
    18.     }
    19.     Fallback "Diffuse"
    20.   }
    Any assistance would be greatly appreciated.

    Edit:

    commenting out the line:
    o.Normal = UnpackNormal (tex2D (_MainTex, IN.uv_MainTex));
    does make it batch, but commenting out the other line does not. It seems like the normal map stage is breaking the batching. Still not sure what's going on though.
     
    Last edited: Jul 27, 2012
  2. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    If you want to use anything other than a directional light, yes, that's the way it's set up, in Unity.
     
  3. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    Hi Jessy, thanks for the reply. All I want is one directional light, and that is the only light I have in my scene.
     
  4. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    Then yes, batching is possible. I'm not sure what is preventing it from happening for you right now.
     
  5. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    Thanks for reply Jessy, it's good to know that it should be possible. I might try to make a simple repro outside my game and try to debug it further. I really enjoyed your YouTube videos by the way, taught me everything I know about shaders. :)
     
  6. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Like the new mad scientist avatar, Jessy :D
     
  7. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    I made a simple scene with two Unity cubes in it and I am seeing the same behaviour as inside my game. Any shader which has normal mapping does not get dynamic batching. Does anyone know for sure if it is possible to batch materials using shader with normal mapping?
     
  8. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    You're right, it's borked. As soon as you use tangents in a shader, batching stops happening. Sorry I misled you. However, the documentation misled me, by talking about tangents on the batching page, with the implication that they could batch. I just reported this bug.

    Object space or world space normal mapping should work fine, though, if that will suffice.


    OMG dude, I just tried to take a nice picture of me smiling. :*(
     
  9. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    Thanks for shedding some light on this.

    Unfortunately I don't think either of the other types of normal mapping will work. I am randomly generating the meshes (asteroids), they are moving and rotating in space, and they all share the same texture. From my elementary understand of the other types of normal mapping I don't think either would work for this purpose.

    All I want to do is add some "texture" (for lack of a better word) to the shape of my low-poly asteroids. Currently I am just using a bump map coverted to a tangent space normal map in GIMP. Do you have any ideas of how I can make generated meshes bumpy without breaking batching?
     
  10. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    It depends on what "asteroids" are. An object space normal map would make them look bumpy, but depending on how you make them, it could look totally wrong. If you're doing this for today's mobile devices, though, you probably should refine your expectations, and make that work. The pixel shader can be much faster without a space change matrix operation for the normal map, and tangent space maps require that.
     
  11. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    They're just UV mapped icospheres that I exported from Blender that I randomly deform by scooping out lots of craters. The main benefit of the tangent space normal map was that I was able to generate it from a simple grey scale height field using a GIMP plugin. From what I understand of object space normal maps, it isn't as straight forward as that.

    The main thing I like about the normal mapping is I can see subtle changes in the lighting as the asteroids slowly rotate. I guess I'll just go with the "Mobile/Diffuse" shader for now for performance.

    Thanks for all your help, it is greatly appreciated.

    Edit: I just tested out about 50 asteroids using the "Mobile/Bumped Diffuse (1 Directional Light)" shader on my Nexus S and it didn't have noticeable affect on the performace, still stead around 30 FPS so I might just make the high LOD use that shader lower LODs use something simpler.
     
    Last edited: May 30, 2012
  12. foxter888

    foxter888

    Joined:
    May 3, 2010
    Posts:
    530
    i tried dynamic batching with bumped diffuse and bumped specular and they work, the geometry has to be less than 150 poly's which is so low when using those shaders but just using the diffuse shader you can maybe go up to close to 300 polys.
     
  13. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    I tried those shaders on two Unity cubes in an empty scene and they are not dynamically batching for me. I am testing with 3.4.2f3. What version are you using?
     
  14. karljj1

    karljj1

    Joined:
    Feb 17, 2011
    Posts:
    440
    Maybe you could bake a light map using directional lightmaps with bumped spec on your asteroids. then swap them for diffuse. The lightmap info will still contain the bump map data. Not sure how it will look though :)
     
  15. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I am having the same issue with dynamic batching not working for unity default shaders such as "Specular"

    I am guessing that this is the same issue as discussed here and been reported as a bug officially?

    Unity has announced Unity 4 for pre order and I am afraid that Unity won't update 3.x versions with this fix.....

    Do you guys think / know that if Unity will fix this bug and release 3.5.2.x version ?

    This seems like a bug that is very important to be fixed ... and should be fixed soon.

    Anyone know?
     
  16. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    I haven't heard anything, I'm assuming it wont be fixed.
     
  17. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    Actually, even specular shader breaking Dynamic batch.
    Only Diffuse, vertex lit type (except reflective group), and unlit shader that batched (Using default built in shader).
     
  18. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Yes, even simple specular shader breaks batching. This bug is like.. wow... I can't even imagine how come this kind of bug is still in 3.5.2 and why there isn't fix for this yet. It's like here is dynamic batching , but haha, too bad ~ you can't use it. Unity should come forth and let us know their plan for the fix for this bug as dynamic batching is core functionality of Unity.
     
  19. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I have just installed Unity 3.5.3 But dynamic batch bug is still not fixed. Dynamic batch still breaks for any shader that uses normal map.
    This includes Unity builtin shaders.

    I thought this is critical bug and should have been addressed asap, but no.. still not fixed yet. Dynamic batch plays important role especially for mobile platform performance so it just beats me why this is not fixed yet.

    Also how can we check status / progress on this bug? Anyone know?
     
  20. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    sure its batching that breaks and not your math?

    Any object hit by a pixel light (by definition required for pixel lights) will be drawn once again for every light impacting it.
    This is on top of the 'flat drawcall'
    The same holds for any object hit by dynamic shadows or a projector.

    so with a light you will, assuming all objects are impacted, at least get numberofobjects + 1 drawcalls (if batching works), otherwise 2 * numberofobjects drawcalls

    and without the original reporter of the bug linking you to it you can't check the status

    the only light not causing additional drawcalls when hitting an object are vertex lights which will not work with normalmaps and specular maps, as they are not per texel lights
     
    Last edited: Jun 28, 2012
  21. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Dreamora.

    Yes, I am well aware of the pixel light's impact on the drawcall.

    However, (and unfortunately) that is definitly not the case because, you can very easily reproduce this by making a new scene with nothing but 2 cubes and each of them Bumped diffuse shader applied.
    There are no lights in the scene and batch breaks with 2 drawcall instead of one. I have just made bug report of my own as well. Windows PC platform.

    Bug case number : 473536

    and not sure if other people can follow the link but here it is :

    https://fogbugz.unity3d.com/default.asp?473536_l17v34liuv2gtm3s
     
    Last edited: Jun 28, 2012
  22. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,521
    Where you found 3.5.3? Is it out yet?
     
  23. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Humm I have access to the beta release because I work on console platform. So maybe there is no official 3.5.3 yet. My version is 3.5.3 release candidate 3
     
  24. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,521
    I see, well the good news is they have still time to fix the bug for the official release :)
     
  25. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Yeah, but my worry is if Unity hasn't fixed this bug for this long time, then I wonder if it will be fixed for official 3.5.3 cause if you search this on Unity sites, it seems that it has been going on for some time.
     
  26. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,521
    Maybe good idea to pm some of the shader guys at unity, perhaps they have overlooked it.
    I sure hope that unity will pay some attention to these kind of issues as mobile performance is key for me too.
     
  27. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    as I have suspected, official 3.5.3 version did not fix this bug....

    one thing that i can't understand is the lack of voice concerning this issue on the forum or unity answer...
     
  28. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I am still waiting for any reply from my bug report,... but nothing coming back yet. it has been 2 weeks.. I wonder how long it will take before I get any reply.
     
  29. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    I submitted a bug ticket a while ago for something else and as far as I know it still has not received any attention.
     
    Last edited: Jul 27, 2012
  30. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    True... it is just that this kind of bug should be considered as major and feature breaking bug , yet Unity is not replying to it and keeping its silent.
     
  31. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    I imagine the bugs for 4.0 release are probably mostly already allocated and being worked through. Once it's out they can go and look through the freshly opened ones :)
     
  32. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    If Unity decides to fix this kind of bug after dealing with majority of 4.0 bugs then well, it is decision by Unity so I can suck on it. However what I don't like is that if that is the case, (or any other) then Unity must let people who are concerned about the bug what their plans are. Worst case for any "customer" service is placing them in a dark room.
     
  33. ImogenPoot

    ImogenPoot

    Joined:
    Jul 2, 2012
    Posts:
    214
    Here is a more in-depth discussion ;)

    http://forum.unity3d.com/threads/102739-How-to-get-Dynamic-Batching-to-work

    I solved it with implementing a custom octree-batching algorithm. Have fun!

    PS: I see "Cameron" added something but that is still an issue of Unity. Batching is something totally independent of whatever you do with the rendering. As long as you don't need to preserve draw call order for some reason, say transparent shaders, rendering all objects with the same material in one call or in thousand calls makes no visual difference.

    The saying goes, "better make a few more draw calls, than having to copy geometry around in every frame". Okay, but when I hit the 4000 draw call limit, then this does not hold anymore. Because basically what we have then is "Better copy huge chunks of geometry around every frame, than having an unplayable game"... Unity definitely needs to adopt this latter philosophy, especially since copying geometry in optimized C++ is very fast compared to what you can do with the stupid UnityEngine.Mesh and C#.
     
    Last edited: Jul 27, 2012
  34. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Nope, I still don't have any responds from Unity.

    This is getting funny.. as in 4.0 this bug still exists...

    How can Unity "advertise" to have efficient dynamic batching system and then it just doesn't work like it is saying...

    Man... this is getting old now.
     
  35. ImogenPoot

    ImogenPoot

    Joined:
    Jul 2, 2012
    Posts:
    214
    Aras mentioned in the Unity 4 thread, that Dynamic Batching is working as expected. So I guess this is basically it. There doesn't seem to be a bug for UT, even though the feature is totally useless ^^. Unless UT is only testing their engine with mobile games, there is no way in hell they didn't run into this SERIOUS bug.
     
  36. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Do you have a link? I don't remember what I've said during all these years; but I'm pretty sure I did not say something like "you say batching with tangents does not work? I say it works, shut up!"... ;)


    Anyway from what I can see, we have:

    * Shaders with tangents (e.g. bumpmapping) do not dynamically batch on Windows Mac (they should statically batch just fine). We had to disable that on Mac due to some driver bugs; and I don't remember why we had to disable that on Windows frankly.
    * All should dynamically batch just fine on mobile platforms.

    We'll take a look at the case numbers posted in this thread (so far I see one?). If you see any other behaviour (like Specular shader is not batching), please let us know.
     
  37. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Update: just experimented with enabling dynamic batching when tangent vectors are involved:

    * On Mac: looks like that driver bug that caused us to disable it is fixed these days (I've tested on OS X 10.7.4), so I'll try to enable it after some more QA.
    * On Windows: looks like the reason that caused us to disable it on D3D9 is fixed now by somewhat-related other fixes, so I'll try to enable that as well.

    First I'll do the two above in 4.0 codebase, then some more QA to verify, and then I'll see about doing the same for one of upcoming 3.5.x releases.

    And again, if anyone sees something else (tangents cause no dynamic batching on mobile; shaders without tangents cause no dynamic batching; no static batching when expected etc.), let us know.

    That said, yes there are many reasons why batching might not work. The obvious ones are different shaders materials; less obvious are different object scales; presence of per-pixel additive lights; presence of shadows; deferred rendering in many cases etc. Many of these have to do with precision (batching transforms to world space, which produces a tiny difference in final vertex position - but enough to result in really, really horrible Z fighting in case of multipass/shadows/deferred).
     
  38. ImogenPoot

    ImogenPoot

    Joined:
    Jul 2, 2012
    Posts:
    214
    Well that sounds strange. Wouldn't double precision be an option?
    But anyway, I batch object in my game for deferred rendering, with dynamic shadows, tangents and complicated shaders, and there are no artifacts and everything works well. Just that it is much slower than what you could do in C++ code. Additionally, why not allow us on a per-material basis to force batching? I mean it is okay if you say that by default it might cause artifacts so we don't enable it. But I have no choice. I NEED batching regardless of the outcome because otherwise the draw calls will skyrocket.

    You practically clarified yourself ^^. But still I consider this a bug. CE3 and UDK will batch this stuff... well they must, because basically with all that S*** flying around when you blow things up, there is just no choice, just like in my case.
     
    Last edited: Jul 27, 2012
  39. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    Hi Aras,

    Thanks for taking a look at this. So the problem can be reproduced by creating an empty scene, adding two Unity cubes sharing the same material which uses a shader that has normal mapping. I testing this inside the Editor on Windows targeting Android. My actual game has one directional light only and everything else is batching nicely. The only thing which isn't working are the normal mapped meshes. If I am targeting Android, should I expect the graphics stats in the editor to show the batching for Android or PC?
     
  40. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    That is a good question.

    Anyway...

    So, the dynamic batch was partially disabled by Unity because of Mac and (D3D9 related) bug?
    Well, I guess now I know "why" it is not working at least.

    But the thing is that it took me this long to get some info about this... and I really think there should be note about this with every official release of the Unity version. (was there? I don't remember but I doubt it...)

    If there was official note / explaination with the release note on versions of Unity relased, then I would have not become so annoyed.:mad:
     
    Last edited: Jul 28, 2012
  41. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Like I said above; when you are in the editor, you're using either D3D9 (on Windows), or OpenGL (on Mac). Both of those currently (Unity 3.5.4) have dynamic batching disabled when tangent vectors are involved, for various reasons (and I'm looking into fixing that). It does not matter whether you're in Android platform mode or not; the editor is still using either D3D9 or GL.

    When you deploy to the Android device, dynamic batching should work with current Unity version (and quite a lot of previous versions).
     
  42. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    I guess internally here, we never saw a strong case where dynamic batching helps with performance on Windows/Mac anyway. Well, except maybe if you have a million of 3-poly objects or something. But in actual real projects we profiled, dynamic batching on PC was a bit faster in some cases, and a bit slower in other cases. On mobile it's a different story.

    That's why we never took serious effort at documenting all the cases where it will or will not work - because we haven't seen a strong case of it being very helpful on PC anyway. If your experience is different, that would be good to know (with the details about OS/CPU/GPU, number of draw calls, CPU times with batching without etc.)
     
  43. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I don't think it really matters if internal profile by Unity shows that dynamic batch helped on the Mac / PC or not. The point is that it is advertised feature (pretty strongly as well) but when it went disabled, information was not documented enough , communication was lacking to the end user which caused this confusion. If Unity indeed thought it was not serious that it didn't need to be documented in detail then it is fine, but then again, the end user do not know such reason.

    If Unity's internal profile showed that dynamic batch was indeed helped on some cases and some not, then THAT information should have also been documented and let it be known easily.

    I am sorry but I don't think any excuse can cover the fact that there was some lack of communication here especially when a paid feature of a software doesn't work properly.
     
  44. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    The problem is, whether it helps or not depends on almost literally, a million of factors. About the only way to know if it helps or not is to check with your specific game for whether dynamic batching is faster or slower (it can be slower!).

    ...dynamic batching is not a paid feature; it's in Unity Free as well ;)

    But point taken that we need more docs on when it does does not work. Will try doing that.
     
  45. Bork-Awesome

    Bork-Awesome

    Joined:
    Jul 2, 2012
    Posts:
    23
    Just submitted a bug with a simple test scene demonstrating non-batching of normal mapped objects in the iOS player; case number 498889.

    I'm as sure as I can be that I am following the batching rules; uniform scale, same material, no shadows etc. (what I have derived from forum threads and whatnot)...
     
  46. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    My original test case was really simple, two identical Unity cubes. It may be worth trying that, try with diffuse make sure it batches and then change it to bumped diffuse. If that doesn't batch then it is definately broken.
     
  47. Bork-Awesome

    Bork-Awesome

    Joined:
    Jul 2, 2012
    Posts:
    23
    @_Petroz

    Good call; so I've *just* updated to Unity 4.0.0f7 and tried the two Unity cubes - they batch as expected - but if I replace the two cubes with Unity spheres they don't batch, even with the default diffuse shader... does poly count effect batching?
     
  48. Bork-Awesome

    Bork-Awesome

    Joined:
    Jul 2, 2012
    Posts:
    23
  49. _Petroz

    _Petroz

    Joined:
    May 13, 2010
    Posts:
    730
    haha, i'm glad to hear it's working. :)
     
  50. MaT227

    MaT227

    Joined:
    Jul 3, 2012
    Posts:
    628
    Hey, I just found this old topic because I am having the same issue while using a surface shader that writes to Normal.
    Writing to Normal even without any lighting breaks batching.
    Any idea why ?