1. Help us improve the editor usability and artist workflows. Join our discussion to provide your feedback.
    Dismiss Notice
  2. We're looking for feedback on Unity Starter Kits! Let us know what you’d like.
    Dismiss Notice
  3. We’re giving 2017.1 beta testers a chance to win t-shirts and a Nintendo Switch. Read more on the blog.
    Dismiss Notice
  4. Unity 2017.1 beta is now available for download.
    Dismiss Notice
  5. Unity 5.6 is now released.
    Dismiss Notice
  6. Get further faster with the Unity Plus Accelerator Pack, free for new Unity Plus subscribers for a limited time. Click here for more details.
    Dismiss Notice
  7. Check out all the fixes for 5.6 on the patch releases page.
    Dismiss Notice
  8. Learn how you'll soon be able to publish your games to China in four simple steps with Xiaomi. Sign up now for early access.
    Dismiss Notice

Difficulty using normal maps to transform vertices in vertex shader

Discussion in 'Shaders' started by Iron-Warrior, Mar 19, 2017.

  1. Iron-Warrior

    Iron-Warrior

    Joined:
    Nov 3, 2009
    Posts:
    632
    Hi, I'm attempting to use a normal map to displace vertices on the ZX plane (in object space). The reason I'm doing this is to eventually be able to pass in a dynamically generated normal map for effects like grass being pushed in a specific direction, or water ripples. Here is what I currently have:

    Code (csharp):
    1.  
    2. Shader "Unlit/Grass"
    3. {
    4.     Properties
    5.     {
    6.         _MainTex ("Texture", 2D) = "white" {}
    7.         _Cutoff("Alpha cutoff", Range(0,1)) = 0.5
    8.  
    9.         _ForceMap("Force Map", 2D) = "black" {}
    10.     }
    11.     SubShader
    12.     {
    13.         Tags { "RenderType"="Opaque" }
    14.         LOD 100
    15.         Cull Off
    16.  
    17.         Pass
    18.         {
    19.             CGPROGRAM
    20.             #pragma vertex vert
    21.             #pragma fragment frag
    22.            
    23.             #include "UnityCG.cginc"
    24.  
    25.             struct appdata
    26.             {
    27.                 float4 vertex : POSITION;
    28.                 float2 uv : TEXCOORD0;
    29.             };          
    30.  
    31.             struct v2f
    32.             {
    33.                 float2 uv : TEXCOORD0;
    34.                 float4 local : TEXCOORD1;
    35.                 float4 vertex : SV_POSITION;
    36.             };
    37.  
    38.             sampler2D _MainTex;
    39.             float4 _MainTex_ST;
    40.  
    41.             sampler2D _ForceMap;
    42.             float4 _ForceMap_ST;
    43.            
    44.             v2f vert (appdata v)
    45.             {
    46.                 v2f o;              
    47.  
    48.                 o.local = v.vertex;
    49.  
    50.                 float2 coords = float2(v.vertex.x, v.vertex.z) / 10 + 0.5;
    51.                 float3 normal = UnpackNormal(tex2Dlod(_ForceMap, float4(coords, 0, 0)));
    52.                 normal.y = 0;
    53.  
    54.                 float4 transformed = v.vertex;
    55.                 transformed.xyz += normal;
    56.  
    57.                 o.vertex = UnityObjectToClipPos(transformed);
    58.                 o.uv = TRANSFORM_TEX(v.uv, _MainTex);
    59.                 return o;
    60.             }
    61.  
    62.             float _Cutoff;
    63.            
    64.             fixed4 frag (v2f i) : SV_Target
    65.             {
    66.                 // sample the texture
    67.                 fixed4 col = tex2D(_MainTex, i.uv);
    68.                 clip(col.a - _Cutoff);
    69.  
    70.                 float2 coords = float2(i.local.x, i.local.z) / 10 + 0.5;
    71.                 fixed4 normalColor = tex2D(_ForceMap, float4(coords, 0, 0));
    72.  
    73.                 return fixed4(coords.x, coords.y, 0, 0);
    74.             }
    75.             ENDCG
    76.         }
    77.     }
    78. }
    79.  
    I transform the vertex's ZX position into UV coords by first dividing by the mesh's bounds (I am using the default Unity plane to test, and it is 10x10) and then adding 0.5, as Unity's UV coords are in the range of 0-1 (I think), and the mesh is centered around the 0 (being -5 to 5). I then sample the Force map texture at these coords and unpack them into a normal. Here is the normal map I'm using to test:

    [​IMG]

    My assumption is that for tangent space normals, the pastel blue colors are straight up, while the red, dark blue and greens would be pointing towards the edges of the texture. I don't use normal maps much, so please correct any assumptions of mine that are not on the mark!

    So I zero out the Y of the normal, add the normal to the vertex and then transform it. My expectation would be to have the mesh stay centered around the 0, the vertices being pushed out into a circle (and the verts on the edge unchanged).

    [​IMG]

    It's kind of there, but two main issues: it's sort of a messy circle, which I don't entirely understand. As well, the entire plane is translated 1 unit up the z axis (I have a copy of the plane with the default material on it below for comparison), which again I'm confused about.

    Side note: Does anyone have any general advice on how to debug shaders properly? Outside of coloring normals and UVs, I can't seem to find any handy tools like Debug.Log or stepping the shader.

    Thanks for any help,
    Erik
     
  2. Iron-Warrior

    Iron-Warrior

    Joined:
    Nov 3, 2009
    Posts:
    632
    Figured it out! When you unpack a normal, the tangent normal is X/Y for left/up, and Z for coming out of the screen...which seems pretty obvious in retrospect, but I guess I was super in Unity mode and didn't think enough about the coordinates system.