Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Alpha blending in a shader different to readPixels

Discussion in 'Shaders' started by monark, Sep 13, 2009.

  1. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    I have a semi transparent shader on some objects which looks great in a render but when I encode it to a png via ReadPixels I get a different result and therefore its impossible to re-create the same image outside of Unity for example in photoshop. Is there a trick to fixing this?

    So in a Unity render it looks like the second image and after saving and comping it looks like the top one. How do I make the top one look like the bottom one?

    I can almost get what I want using "Screen" mode in photoshop but that messes up the non-transparent bits. Is there a way to modify the image alpha/colours before encoding it to png so it will comp the transparent bits like in screen mode but leave the rest untouched when just using a normal composite?

    The shader code looks like this

    Code (csharp):
    1.  
    2. Shader "My Shaders/XRay" {
    3.     Properties {
    4.         _Color ("Tint (RGB)", Color) = (1,1,1,1)
    5.         _RampTex ("Facing Ratio Ramp (RGB)", 2D) = "white" {}
    6.         _Tint ("Texture tint", Color) = (1,1,1,1)
    7.         _Rle ("Rle color", Color) = (0.5,0.5,0.5,1)
    8.     }
    9.     SubShader {
    10.         Cull Off //turn this on if using Blend One One
    11.         ZWrite Off
    12.         Tags { "RenderType"="Opaque" "Queue" = "Transparent" "VisibleInDepth"="On" }
    13.         //Blend One One
    14.         Blend One OneMinusSrcColor
    15.         Pass {
    16.            
    17.             CGPROGRAM
    18.             #pragma vertex vert
    19.             #include "UnityCG.cginc"
    20.  
    21.             struct v2f {
    22.                 V2F_POS_FOG;
    23.                 float4 uv : TEXCOORD0;
    24.             };
    25.            
    26.             v2f vert (appdata_base v) {
    27.                 v2f o;
    28.                 PositionFog( v.vertex, o.pos, o.fog );
    29.                 float3 viewDir = normalize(ObjSpaceViewDir(v.vertex));
    30.                 o.uv = float4( abs(dot(viewDir,v.normal)), 0.5, 0.0, 1.0 );
    31.                 return o;
    32.             }
    33.             ENDCG
    34.  
    35.             SetTexture [_RampTex] {constantColor[_Color] combine texture * constant}
    36.         }
    37.     }
    38.     // Fallback to Alpha Vertex Lit
    39.     Fallback "Alpha/VertexLit", 2
    40. }
    41.  
     

    Attached Files:

  2. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    This is basically a limitation of Photoshop's blending modes. If you can't do it with the built-in modes, you'll have to do your blending elsewhere. To help you discern whether you can use the built-in modes, take a look here:

    http://www.teamphotoshop.com/articles-The-Techniques-Layer-Blend-Modes-Explained!-5,8,65a.html
    (copy and paste due to forum limitations)

    or here for a more programmatic approach:

    http://www.nathanm.com/photoshop-blending-math/
     
  3. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    I don't think its a limitation of the blending modes I think its more a limition with how alpha and colour are stored when you grab the pixels from a render.
    The issue is that I want normal users to be able to save images and then put them on any background without having to have extensive photoshop expertise, our users are doctors and surgeons not digital compositors :cry:

    One solution I've toyed with but haven't had time to try is to write my own post process that somehow takes the image without alpha and then creates the correct alpha and colour values for these objects, that could be a lot of work so I was hoping someone might have a quick trick that would do it.
     
  4. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    I think I see what you mean now. If you want to save images with transparency, you need to have your shader record its alpha value without using any blending:
    Code (csharp):
    1. Blend Off
    Then make sure the clear colour for your camera has an alpha of zero, and when you use ReadPixels you'll get the alpha channel straight from the fragment shader rather than from the blending function.

    Of course, it will look wrong in Unity because you can't see the alpha channel in the render target. The resulting PNG will look right, though. If you need it to look right both in Unity and in saved files, you'll have to render with a different shader in each case.
     
  5. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    cheers I'll try that, that sounds like exactly what I want. I guess there is no way to dynamically switch the blend mode to off is there, you have to do it with a separate shader?
     
  6. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    rats, that doesn't work at all :cry: in fact it's worse....
     

    Attached Files:

  7. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    I've found a solution but its a horrible hack.
    I basically do a screen overlay before encoding to a png and saving but only for pixels that have a hue that matches my transparent shader, this obviously only works because all the transparent bits are using the same shader and would instantly break if I use other colours. Still it will have to do for now.
     
  8. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    What did you do to get the second image? It's not clear to me what you're showing. Shader code would be great, too.
     
  9. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    Shader code is in the first post.

    The key is

    Code (csharp):
    1. Blend One OneMinusSrcColor
    The default for this Xray shader when downloaded is

    Code (csharp):
    1. Blend One One
    But the issue with that blending mode is 2 fold, first I have the same issue when saving image i.e it's not WYSIWYG but also inside the Unity player it doesn't handle multiple layered surfaces overlapping very well as you get total burn out where the opaque bits overlay each other.

    Using
    Code (csharp):
    1. Blend One OneMinusSrcColor
    I was able to get a much more subtle xray effect where there isn't any burnout. The issue then became how to let users save and image in a way that they could then comp them onto any colour or any other image and have it look just as nice. I.e same colours, and no burnout or darking as in the first example.
     
  10. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Ok, but when you use Blend Off on a transparent background, and use ReadPixels and save the result as a PNG, you should be getting a decent export that can then be composited in Photoshop using whatever you want (regular complementary alpha blending being the default). Is that not working?

    Of course, it won't look right in Unity because it won't be blended properly, but it should look right in Photoshop, etc.
     
  11. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    When I do that I don't get any transparency on the xrayed objects so although the very front edges look right the internals don't feature any overlapping surfaces. See the pic above that is on black. That image is a comp not what I get in Unity but what I get after saving.
     
  12. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Ok, I'd forgotten about overlapping. I've been thinking about this for a while and it's confusing me. There must be a solution, but I'm not sure how to set up a shader so that the colour channel comes out properly. The alpha channel can be layered properly by starting with a transparent clear colour, and having the shader use:

    Code (csharp):
    1. Blend OneMinusSrcAlpha One
    as the blending function.
     
  13. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Ok, I think I've got it!

    This only works for alpha blending, but I think it works pretty well. That said, I haven't tried it.

    First, set your camera's clear colour to (0,0,0,0).

    Second, use a shader that uses two passes. The first pass uses:
    Code (csharp):
    1. ColorMask RGB
    2. Blend SrcAlpha OneMinusSrcAlpha
    The second pass uses:
    Code (csharp):
    1. ColorMask A
    2. Blend One OneMinusSrcAlpha
    Both passes perform the same texture operations and produce the same RGBA output.

    Finally, you need to divide the colour channel of your render target by its alpha channel. This will remove the black component introduced by alpha blending onto a black background. You can do this on a pixel-by-pixel basis after using ReadPixels, but remember that a lot of pixels will still have an alpha value of zero, so just ignore those ones.

    The result should have the correct values in all four channels, and will behave properly as a PNG. Even better, because the shader writes the colour channels normally, you don't have to use a separate shader for displaying to the screen and for saving to PNG.
     
  14. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    Close but no banana!

    I've run out of time at the moment to play with this but hopefully there is a solution in there somewhere, I'll try again hopefully next week. I can see your logic and it has given me some good ideas on what to try.
     
  15. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Did you try it? It worked as expected for me. Here is the modified shader:
    Code (csharp):
    1. Shader "FX/XRay Special Alpha" {
    2.     Properties {
    3.         _Color ("Tint (RGB)", Color) = (1,1,1,1)
    4.         _RampTex ("Facing Ratio Ramp (RGB)", 2D) = "white" {}
    5.     }
    6.     SubShader {
    7.         Cull Off //turn this on if using Blend One One
    8.         ZWrite Off
    9.         Tags { "RenderType"="Opaque" "Queue" = "Transparent" "VisibleInDepth"="On" }
    10.         Pass {
    11.             ColorMask RGB
    12.             Blend SrcAlpha OneMinusSrcAlpha
    13.            
    14.             CGPROGRAM
    15.             #pragma vertex vert
    16.             #include "UnityCG.cginc"
    17.  
    18.             struct v2f {
    19.                 V2F_POS_FOG;
    20.                 float4 uv : TEXCOORD0;
    21.             };
    22.            
    23.             v2f vert (appdata_base v) {
    24.                 v2f o;
    25.                 PositionFog( v.vertex, o.pos, o.fog );
    26.                 float3 viewDir = normalize(ObjSpaceViewDir(v.vertex));
    27.                 o.uv = float4( abs(dot(viewDir,v.normal)), 0.5, 0.0, 1.0 );
    28.                 return o;
    29.             }
    30.             ENDCG
    31.  
    32.             SetTexture [_RampTex] {constantColor[_Color] combine texture * constant}
    33.         }
    34.         Pass {
    35.             ColorMask A
    36.             Blend One OneMinusSrcAlpha
    37.            
    38.             CGPROGRAM
    39.             #pragma vertex vert
    40.             #include "UnityCG.cginc"
    41.  
    42.             struct v2f {
    43.                 V2F_POS_FOG;
    44.                 float4 uv : TEXCOORD0;
    45.             };
    46.            
    47.             v2f vert (appdata_base v) {
    48.                 v2f o;
    49.                 PositionFog( v.vertex, o.pos, o.fog );
    50.                 float3 viewDir = normalize(ObjSpaceViewDir(v.vertex));
    51.                 o.uv = float4( abs(dot(viewDir,v.normal)), 0.5, 0.0, 1.0 );
    52.                 return o;
    53.             }
    54.             ENDCG
    55.  
    56.             SetTexture [_RampTex] {constantColor[_Color] combine texture * constant}
    57.         }
    58.     }
    59. }
    And here is the script I used to capture the image and divide colour by alpha:
    Code (csharp):
    1. using UnityEngine;
    2. using System.Collections;
    3. using System.IO;
    4.  
    5. [RequireComponent(typeof(Camera))]
    6.  
    7. public class DivideAlpha : MonoBehaviour {
    8.    
    9.     public RenderTexture renderTexture;
    10.    
    11.     void TakePicture () {
    12.         Texture2D texture = new Texture2D(renderTexture.width, renderTexture.height, TextureFormat.ARGB32, false);
    13.        
    14.         RenderTexture.active = renderTexture;
    15.         texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0, false);
    16.         texture.Apply();
    17.         for (int x = 0; x < texture.width; x++) {
    18.             for (int y = 0; y < texture.height; y++) {
    19.                 Color color = texture.GetPixel(x, y);
    20.                 float alpha = color.a;
    21.                 if (alpha != 0f) {
    22.                     color /= alpha;
    23.                     color.a = alpha;
    24.                     texture.SetPixel(x, y, color);
    25.                 }
    26.             }
    27.         }
    28.         RenderTexture.active = null;
    29.        
    30.         byte[] bytes = texture.EncodeToPNG();
    31.         File.WriteAllBytes(Application.dataPath + "/../SavedScreen.png", bytes);
    32.         DestroyImmediate(texture);
    33.     }
    34.    
    35.     protected IEnumerator Start() {
    36.         yield return 0;
    37.         TakePicture();
    38.     }
    39. }
    Here is a picture of two spheres using the shader. As you can see, they composite properly even here on this forum.
     
  16. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    That looks great, I did try it but it didn't come out like that, the read and blue would have been black. I probably missed something in the code somewhere, I'm still working my way through shaderLab haven't figured it all out yet. Just looking at the code quickly it looks the same so I'll try out your version and see where I must have something wrong, at least I know it should work which is great.
    Many thanks for taking the time out to get a correct solution this is really helpful.
     
  17. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    Do you find it works in unity too? I think that assumption maybe wrong and I will need to return to the idea of swapping the shaders when outputting?
     
  18. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    There's something odd going on here, your posted image looks exactly like what I'm after (more or less) but when I use your code in my file it comes out much less like what I want, which was why i thought it wasn't working.

    This is a screen shot straight from a test file where I've tried to recreate your test. To me there is a black tint to the transparent parts which you don't get. I can only think this is somehow something to do with the ramp image I use? I'm at a loss.

    Below is a comparison of what I'm currently getting with your shader versus my current one and hacky screen overly fix.
     

    Attached Files:

  19. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    The shader produces the same result in Unity. If you need a different coloured background for real-time display, you can just switch it to black while taking the picture. This can be done invisibly by using Camera.Render after setting RenderTexture.active to your target texture.
     
  20. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Are you using a ramp with black in it? Try this one instead:



    (there's a PNG there in case it's not obvious)

    Your ramp looks like it goes from opaque white to transparent black, whereas mine goes from opaque white to transparent white. Using your old blending method, you didn't see the black because it was additive. With pure alpha blending, you'll see any dark colours in your ramp as well.
     
  21. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    AHHHHHHH...... yes I was. I never thought of that!! Unity seems to handle transparency slightly differently to what I'm used to, I've been caught out by this before, i should of thought of that. Brilliant. :p
     
  22. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,598
    One final word on this....

    Interestingly thanks to your help I've now discovered 2 ways to do this. :)

    1. Your way with 2 passes using the white ramp :wink: all looking lovely now

    2. My original shader with a white to black ramp and matching alpha channel but with the output also divided by the alpha just as you are doing.

    Previously I was doing a screen overlay with the colour channels, but trying to extract just the xrayed pixels, it worked sort of but had obvious issues and limitations, this new way is simpler as I just treat all pixels the same. It may not be mathematically correct but gives me what i want too. The two looks are slightly different so now its a straight choice about which one looks best.

    Many thanks again.