Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

chroma key shader

Discussion in 'Shaders' started by chippermonky, May 14, 2010.

  1. chippermonky

    chippermonky

    Joined:
    May 14, 2010
    Posts:
    7
    Hi, new here. Working on a game for class. Were trying to overlay video onto our game as a sort of meta game element. I pulled some shader code from this http://forum.unity3d.com/viewtopic.php?t=12228&highlight=colbert+green+screen+video thread to do basic binary chroma keying. It works but I only get boolean values of alpha out of it. By that I mean

    return half4(color.r, color.g, color.b, .49595);

    turns things complete opaque whereas

    return half4(color.r, color.g, color.b, .5);

    turns things completely transparent.

    I'd like a smooth alpha value based on the amount of blueness in each pixel. Is there any way to do this? I'm a complete shader noob but good at coding in general.

    Thanks!
     
  2. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Why not just use an alpha channel? Then you don't lose your blue.
     
  3. chippermonky

    chippermonky

    Joined:
    May 14, 2010
    Posts:
    7
    I'm applying the shader to a video texture which afaik does not support alpha channels correct me if I'm wrong.
     
  4. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Yeah, they do. Most videos don't use them, so most players don't play them. Alpha video channels are used all the time for video compositing, and of course special effects in games.
     
  5. chippermonky

    chippermonky

    Joined:
    May 14, 2010
    Posts:
    7
    Okay thanks, I'll give that a try.

    For general reference though, I'm still curious as to how to get the full range of alpha values from a shader.
     
  6. chippermonky

    chippermonky

    Joined:
    May 14, 2010
    Posts:
    7
    so I did some research and according to this thread http://answers.unity3d.com/questions/1600/how-do-i-use-movie-texture-with-alpha-channel unity does not support video alpha. btw I did try and export video with alpha and confirmed that it did not work. Of course it is possible I just did not export it right.

    I could use a second video as alpha channel as suggested in the video but that involves writing a shader as well (which brings me back to the same problem as before).

    In any case, that's still less than desirable as I'd much rather do the keying process directly in unity with a shader to save time.
     
  7. chippermonky

    chippermonky

    Joined:
    May 14, 2010
    Posts:
    7
    Okay seems like I resolved this problem. The shader I copied was using alpha testing instead of alpha blending. I'm still figuring out how shaderlab works together with cg but its slowly starting to make sense.

    Thanks!
     
  8. ChemiKhazi

    ChemiKhazi

    Joined:
    Nov 16, 2009
    Posts:
    27
    I've messed about with matting shaders before. The video + mask video route goes too slowly since theres two video files playing, might depend on your video size though.

    I've got another matting shader that does a simple color threshold and depending on your video it could make the matting look nicer.

    If you manage to make a matte shader this good please post your code in the wiki! Anyway, here's the matte shader with a simple threshold:

    Code (csharp):
    1. Shader "Matte/RGB Color"
    2. {
    3.  
    4. Properties {
    5.    _MainTex ("Base (RGB)", 2D) = "white" {}
    6.    _Matte ("Matte Color", Color) = (1,0,1,1)
    7.    _Thresh ("Threshold", Range (0,1)) = 0.0
    8.    _Cutoff ("Cutoff", Range (0,1)) = .5
    9. }
    10.  
    11. SubShader {
    12.  
    13.    Pass {
    14.       AlphaTest Less [_Cutoff]
    15. CGPROGRAM
    16. #pragma fragment frag
    17. #include "UnityCG.cginc"
    18.  
    19. struct v2f {
    20.     float4 pos : POSITION;
    21.     float4 uv : TEXCOORD0;
    22. };
    23.  
    24. sampler2D _MainTex;
    25. uniform half4 _Matte;
    26. uniform float _Thresh;
    27. uniform float _Cutoff;
    28.  
    29. half4 frag (v2f i) : COLOR
    30. {
    31.     //return _Matte;
    32.     half4 color = tex2D(_MainTex, i.uv.xy);
    33.    
    34.     half3 keyMinus = half3( (_Matte.r - _Thresh), (_Matte.g - _Thresh), (_Matte.b - _Thresh) );
    35.     half3 keyPlus = half3( (_Matte.r + _Thresh), (_Matte.g + _Thresh), (_Matte.b + _Thresh) );
    36.    
    37.     if ( ((color.r > keyMinus.r)  (color.r < keyPlus.r))  ((color.g > keyMinus.g)  (color.g < keyPlus.g))  ((color.b > keyMinus.b)  (color.b < keyPlus.b)) )
    38.     {
    39.         return _Cutoff;
    40.     }
    41.    
    42.     return float4(color.rgb, max(0, _Cutoff-0.01));
    43. }
    44.  
    45. ENDCG
    46.       } // pass
    47.    } // subshader
    48.    //Fallback "Transparent/Diffuse"
    49. } // shader
     
  9. BloodArk

    BloodArk

    Joined:
    Jun 27, 2011
    Posts:
    2
    Hello...

    I really need to display a video with transparency into my Unity project, could you please post a sample project doing that ? I don't know how to use the shaders you are talking about :(

    Thanks a lot

    BloodArk
     
    Archana Sarang likes this.
  10. segafan

    segafan

    Joined:
    May 21, 2009
    Posts:
    4
    Can someone upload any project sample/example of using video with chroma key, please?
     
  11. yannminh

    yannminh

    Joined:
    Jul 1, 2007
    Posts:
    43
    Double image with Alpha for Chroma Key in Unity 3D


    Hello,

    Maybe here is a trick I imagined, that could be usefull... it is not really chroma key, but it works fine...


    Make a movie with both, the foreground and the alpha key image on the same picture with After Effects or other compositing soft.

    Import this movie in Unity3D




    create a shader with the script written by LouLou from the french forum http://www.unity3d-france.com
    http://www.unity3d-france.com/unity/phpBB3/viewtopic.php?f=8&t=901

    Apply this shader to a material, and adjust the Tile and offset of the texture of this material to hide the white alpha image.


    Code (csharp):
    1. Shader "Custom/VideoAlpha" {
    2.    Properties {
    3.       _MainTex ("Base (RGB)", 2D) = "white" {}
    4.       _AlphaOffsetX ("alpha offset x", float) = 0.5
    5.       _AlphaOffsetY ("alpha offset y", float) = 0
    6.       _Cutoff ("Cutoff", Range (0,1)) = .5
    7.    }
    8.    SubShader {
    9.    AlphaTest Less [_Cutoff]
    10.          CGPROGRAM
    11.          #pragma surface surf Lambert
    12.    
    13.          sampler2D _MainTex;
    14.          float _AlphaOffsetX;
    15.          float _AlphaOffsetY;
    16.    
    17.          struct Input {
    18.             float2 uv_MainTex;
    19.          };
    20.    
    21.          void surf (Input IN, inout SurfaceOutput o) {
    22.             half4 c = tex2D (_MainTex, IN.uv_MainTex);
    23.             IN.uv_MainTex.x += _AlphaOffsetX;
    24.             IN.uv_MainTex.y += _AlphaOffsetY;
    25.             half4 d = tex2D (_MainTex, IN.uv_MainTex);
    26.             o.Albedo = c.rgb;
    27.             o.Alpha = (d.r*-1)+1;
    28.          }
    29.          ENDCG
    30.      
    31.    }
    32.    FallBack "Diffuse"
    33. }
    It works fine, it is much more better than hiding the green by script.
    A 600 pixel high sequence is fine.

    If you want to test here are some demo sequences to test

    http://www.noomuseum.net/mov/YannSteadyDoubleAlpha_674.mov
    (the .ogv file for Unity http://www.noomuseum.net/mov/YannSteadyDoubleAlpha_674.ogv


    Model Soizic Hess http://soizic-hess.com/
    http://www.noomuseum.net/mov/SoizicDoubleAlpha-674.mov
    (.ogv file for Unity http://www.noomuseum.net/mov/SoizicRotatekey-2-800ogv.ogv



    1) How could it be possible to make this texture "Unlit" ? does someone have the answer ?

    2) Is it possible to do the same, but with a video movie store in a web server, or in a local computer, because the videos are really too heavy to put inside the project himself...

    Yes, just put this video player script in the screen object, and put a similar jpg image from the movie in the material window.

    Code (csharp):
    1. private var url = "http://www.noomuseum.net/mov/YannSteadyDoubleAlpha_674.ogv";  // Lien de la video
    2. public var object:GameObject; // Objet sur lequel doit être afficher la video
    3.  
    4. function Start () {
    5.  
    6. var www = new WWW(url);
    7.  
    8. var movieTexture = www.movie;
    9. while (!movieTexture.isReadyToPlay)
    10. yield;
    11.  
    12. object.renderer.material.mainTexture = movieTexture;
    13.  
    14.  
    15. audio.clip = movieTexture.audioClip;
    16.  
    17. movieTexture.Play();
    18. movieTexture.Loop = true;
    19. audio.Play();
    20. }
    21.  
    22. @script RequireComponent (AudioSource)
    If you want to store the movie on a local URL on your Mac or PC, replace the URL address in the script by that kind of address : file:///Users/YannSteadyDoubleAlpha_674.ogv


    Here a level I have done with two video inside.. [/url] avec deux videos incrustées à l'intérieur en démo. (les vidéos sont des .ogv hébergés sur le web.)
    http://www.cyberesthesie.com/cyberesthesia-3d/Cyberesthesia.html

    it is a multiplayer test, click "Heberger" and "entrez dans les Noobunkers"

    Sorry, it is a heavy level..


    Yann, NooKeyer
     
    Last edited: Dec 16, 2011
  12. aubergine

    aubergine

    Joined:
    Sep 12, 2009
    Posts:
    2,878
    If i understand your question right, you want to make a mask selectively based on a defined color right?

    Something like this will do:
    Code (csharp):
    1. Shader "Hidden/Aubergine/Mask" {
    2.     Properties {
    3.         _MainTex ("Base (RGB)", 2D) = "white" {}
    4.         _MaskCol ("Mask Color", Color)  = (1.0, 0.0, 0.0, 1.0)
    5.     }
    6.  
    7.     SubShader {
    8.         CGINCLUDE
    9.             #include "UnityCG.cginc"
    10.            
    11.             half3 NormalizeColor (half3 color) {
    12.                 //return color / max(dot(color, half3(1.0f/3.0f)), 0.0001);
    13.                 return color / dot(color, fixed3(0.22, 0.707, 0.071));
    14.             }
    15.            
    16.             half4 MaskColor (half3 mCol, half3 cCol) {
    17.                 half4 d = distance(NormalizeColor(mCol.rgb), NormalizeColor(cCol.rgb));
    18.                 return (d > 1f) ? half4(0.0) : half4(1.0);
    19.             }
    20.         ENDCG
    21.  
    22.         Pass {
    23.             ZTest Always Cull Off ZWrite Off Lighting Off Fog { Mode off }
    24.            
    25.             CGPROGRAM
    26.                 #pragma vertex vert_img
    27.                 #pragma fragment frag
    28.                 #pragma fragmentoption ARB_precision_hint_fastest
    29.  
    30.                 uniform sampler2D _MainTex;
    31.                 uniform float4 _MaskCol;
    32.  
    33.                 half4 frag (v2f_img i) : COLOR {
    34.                     half4 col = tex2D(_MainTex, i.uv);
    35.                     half4 mask = MaskColor(col.rgb, _MaskCol.rgb);
    36.                     //return col * mask;
    37.                     return mask;
    38.                 }
    39.             ENDCG
    40.         }
    41.     }
    42.     Fallback off
    43. }
     
  13. yannminh

    yannminh

    Joined:
    Jul 1, 2007
    Posts:
    43
    Hello Aubergine,

    thanks very much for your shader script,
    Iam sorry, I just try it, but I don't understand how it works... I have made a Shader with it, and use this shader on my video material.. but nothing happen...

    But it doesn't matter, because, as I explained just before I found a way to solve this problem of chroma key in unity...
    Using scripts to make a chroma key in Unity by using a color like green as mask is not fine, because they are green pixels around the foreground image... (very ugly)

    The best way, I found to make a "chroma key" effect, is to use the old video art technic of making the chromakey with a powerful device, like Ultimate, or Soft like After Effect, specialy built to generate a special video file, with the foreground image and a white mask.
    Loulou, from the french unity forum, wrote a script that works fine..

    Thanks very much...

    Yann
     
  14. aubergine

    aubergine

    Joined:
    Sep 12, 2009
    Posts:
    2,878
    It is a full screen image effect, you define the mask color and a threshold (its hard coded in there as return (d > 1f), 1 is the threshold here)
    then if any color is near or close to the value you defined comes out as white and any other colors come out as black and you get your mask

    After that you can do whatever you want with that mask.

    Anyhow, its not like using after effects offcourse.
     
  15. DaveA

    DaveA

    Joined:
    Apr 15, 2009
    Posts:
    310
    I have need of such, but it should work on a mesh, not full-screen. Will any of the above apply?
     
  16. Virtualware

    Virtualware

    Joined:
    Sep 17, 2012
    Posts:
    75
    Hey there!

    We have just released our new Chroma System for Unity. You can check this thread for further details.

    If you're interested in chroma key features in Unity you have to check it out!

    Thanks!
     
  17. Hacksdens

    Hacksdens

    Joined:
    Apr 29, 2013
    Posts:
    2
    Hello

    I appreciate your work aubergine, the shader you provided works great for me but it only works on windows build can you help me how can i get it to work on mobile. when i try to use it on android it doesnt work. Can you please help me with this
     
  18. aubergine

    aubergine

    Joined:
    Sep 12, 2009
    Posts:
    2,878
    I dont have android pro, so i cant see why it wouldnt work. You are on your own.
     
  19. Hacksdens

    Hacksdens

    Joined:
    Apr 29, 2013
    Posts:
    2
    Thanks for the reply i will try to get it to work on android will post if i will be successful.
     
  20. sadiq@xstpl.com

    sadiq@xstpl.com

    Joined:
    Feb 27, 2013
    Posts:
    6
    Can you share the video shader for making black background transparent in unity .

    Thanks
    sadiq
     
  21. Arbelzapf

    Arbelzapf

    Joined:
    Sep 30, 2013
    Posts:
    58

    This is great! Thank you very much.

    Can anyone tell me what to change in order to make this shader use alpha blending instead of alpha testing?
     
  22. detomato

    detomato

    Joined:
    Aug 16, 2013
    Posts:
    54
    Sorry to bump up this old thread. Just wonder if this script work with current version of Unity? Coz I cant seem to get it to work.
    Any help would be appreciated.

    Thanks in advance.

    edit: Eventually it doesn't works on Android. Just too bad.
     
    Last edited: Feb 25, 2014
  23. pain_gwar

    pain_gwar

    Joined:
    Apr 14, 2013
    Posts:
    22
    I know that i'm a little late but maybe someone could be interested.

    You just need to modify line 26 in your shader script from "o.Albedo = c.rgb;" to "o.Emission = c.rgb;"

    Hope this could help someone.
     
  24. pain_gwar

    pain_gwar

    Joined:
    Apr 14, 2013
    Posts:
    22
    I'm using it with the new Unity 4.5 (Pro) and it seems to work fine.

    I think "Pro" is required because it use MovieTexture to play the movie.

    On Android it couldn't work because MovieTexture is not supported on that platform (nor iOS).

    If you need to reproduce movie on mobile you can check out this plugin: http://u3d.as/content/defiant-development-pty-ltd/mobile-movie-texture/2w1
     
  25. rodripf

    rodripf

    Joined:
    Feb 11, 2014
    Posts:
    2
    This shader of mine http://u3d.as/6A4 does the job. It works in android, but only with images, since MovieTexture is not supported.
    Cheers.
     
  26. BabilinApps

    BabilinApps

    Joined:
    Dec 28, 2014
    Posts:
    40
  27. Lucas7CL

    Lucas7CL

    Joined:
    Sep 28, 2015
    Posts:
    7
    @BabilinApps how do you use it in mobile? what class do you use instead of MovieTexture?
    I have working the plane with custom shader for alpha Movie Texture, everything ok, but it fails when tryied to deploy to Mobile: "error CS0246: The type or namespace name `MovieTexture' could not be found. Are you missing a using directive or an assembly reference?"
     
  28. BabilinApps

    BabilinApps

    Joined:
    Dec 28, 2014
    Posts:
    40
    The shader is for alpha clipping for green screen. It doesn't play a video by itself. You can use plugins like https://www.assetstore.unity3d.com/en/#!/content/10032 to render the video on mobile.
     
    Lucas7CL likes this.
  29. jbedo

    jbedo

    Joined:
    Mar 9, 2015
    Posts:
    5