Search Unity

Is 0 / 0 == NaN in Unity's shaders?

Discussion in 'Shaders' started by CHPedersen, Oct 28, 2014.

  1. CHPedersen

    CHPedersen

    Joined:
    Mar 2, 2011
    Posts:
    63
    I have this kind of weird use case where I actually need to deliberately generate a NaN value and return it from a function in HLSL/Cg. And I just can't seem to get it to work. :)

    According to Microsoft, division by zero should result in either +INF or -INF, except for 0 / 0, which should be NaN.

    So, armed with that information, consider this fragment shader:

    Code (HLSL):
    1.             float4 frag(vertOut fragIn) : COLOR
    2.             {
    3.                 if(isnan(0.0f / 0.0f))
    4.                     return float4(1,0,0,1);
    5.                 else
    6.                     return float4(0,1,0,1);
    7.             }
    It should return bright red, right? But it doesn't. It returns green, indicating that isnan evaluates to false on "0.0f / 0.0f". I don't understand why, especially since if you go to class System.Single in C#, you'll find its constants defined like so:

    Code (CSharp):
    1.     [Serializable]
    2.     [ComVisible(true)]
    3.     public struct Single : IFormattable, IConvertible, IComparable, IComparable<float>, IEquatable<float>
    4.     {
    5.         public const float Epsilon = 1.4013e-045f;
    6.         public const float MaxValue = 3.40282e+038f;
    7.         public const float MinValue = -3.40282e+038f;
    8.         public const float NaN = 0.0f / 0.0f;
    9.         public const float NegativeInfinity = -1.0f / 0.0f;
    10.         public const float PositiveInfinity = 1.0f / 0.0f;
    11. .
    12. .
    13. .
    I.e. in C#, the very definition of float.NaN is 0.0 / 0.0. So... What gives? Why doesn't isnan evaluate to true on the same? Is there another way to represent a NaN in Cg/HLSL, or provide some calculation that causes it to trigger one?
     
    SgerbwdGwyn likes this.
  2. CHPedersen

    CHPedersen

    Joined:
    Mar 2, 2011
    Posts:
    63
    Update:

    I can do it with sqrt(-1). That returns NaN as it should. But division by zero curiously doesn't. :)
     
  3. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Does sqrt(-1) generate a NaN?

    Hah, ninja'd with the same response.
     
  4. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    That's documentation for DX11. Are you using DX11?
     
  5. CHPedersen

    CHPedersen

    Joined:
    Mar 2, 2011
    Posts:
    63
    Yes, I am. :) I have the editor running in DX11 mode, but still, I think your suggestion hints at the problem. I had a look around the interwebs and found no evidence that Direct3D 9 implemented IEEE 754 for Floating Point Rules, which would have included the 0/0 = NaN part. Instead, I get the feeling from some blog posts here and there that IEEE 754 compliancy is something Microsoft introduced for Direct3D 10. DX10 is also the earliest page in their docs that lists the Floating Point Rules.

    So I think maybe the missing 0/0 = NaN I'm experiencing under DX11 is just a left-over in Unity from DX9, perhaps. There is this page from NVidia about their vs_2_x profile for DX9 where they explicitly mention IEEE 754, but I'm thinking that has more to do with what their profile and GPUs supported at the time than the actual DX9 specification.
     
  6. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    1,520
    In the DirectX 9 time graphics cards were commonly not IEEE 754 compliant. Not only in the way Inf and NaN are handled, but also in the way precision is maintained when doing calculations. The reasoning behind that was that it is beyond what is needed when doing rendering tasks, which seems reasonable to me.

    It's the GPGPU time that put in the need to be IEEE compliant, so it's not surprising to see this back in more recent Direct3D specifications.

    That Nvidia page mentions that the encoding is the same as in IEEE 754. They don't mention being IEEE 754 compliant.