LUT based color correction destroyed my scene

Hi,

The corresponding pixel shader code snippet is:

Code (CSharp):

1.    
2.                 processedColor.rgb = saturate(processedColor.rgb);
3.                 half sliceNumber = floor((processedColor.b * 256)/16);
4.                 half2 baseUV = half2((sliceNumber * 15)/256, 1);
5.                 half redContribution_OnThisSlice = (1/16) * processedColor.r;
6.                 half greenContribution_OnThisSlice = processedColor.g;
7.                 baseUV.r += redContribution_OnThisSlice;
8.                 baseUV.g = 1 - greenContribution_OnThisSlice;
9.    
10.                 processedColor.rgb = tex2D(_LutTex, baseUV);
11.  

I am using this LUT texture.


When I apply effect, my scene changes from this:


into this,



Any idea why is this inferno happening?

 

I'm not sure if your math is correct, but even if it was, sampling a LUT in its unwrapped 2D form has a few major problems. First is bleeding. For values right next to slice boundaries, the color from the other slice can bleed through. The mipmapping also needs to be disabled on the texture, or alternatively, you could use tex2Dlod instead. The second problem is that you're not interpolating between the individual slices. This results in very visible steps in the blue channel. You'd have to sample the LUT twice in neighboring slices and blend between the colors manually.

The safest option, rather than trying to fix the above, is to convert the unwrapped 2D LUT into a proper 3d texture. You can do this rather easily with a script and the Texture3D.SetPixels() function. Sampling it is trivial:

Code (CSharp):

1. processedColor.rgb = tex3Dlod(_LutTex, float4(saturate(processedColor.rgb), 0));

 

Hi,
Thanks for the suggestion. AFAIK mobile platform does not like texture3d. So I had to live with the 2d texture.
http://kpulv.com/359/Dev_Log__Color_Grading_Shader/
This article helped a lot. I modified previous shader code as below.

Code (CSharp):

1.                        
2.             fixed4 gradedPixel = sampleAs3DTexture(_LutTex, processedColor.rgb, 16);
3.             gradedPixel.a = processedColor.a;
4.             processedColor = gradedPixel;
5.  
6.  

The correspoding function is given below:

Code (CSharp):

1.  
2. float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width)
3.         {
4.             float innerWidth = width - 1.0;
5.             float sliceSize = 1.0 / width; // space of 1 slice
6.             float slicePixelSize = sliceSize / width; // space of 1 pixel
7.             float sliceInnerSize = slicePixelSize * innerWidth; // space of width pixels
8.             float zSlice0 = min(floor(uv.z * innerWidth), innerWidth);
9.             float zSlice1 = min(zSlice0 + 1.0, innerWidth);
10.             float xOffset = slicePixelSize * 0.5 + uv.x * sliceInnerSize;
11.             float s0 = xOffset + (zSlice0 * sliceSize);
12.             float s1 = xOffset + (zSlice1 * sliceSize);
13.             float yPixelSize = sliceSize;
14.             float yOffset = yPixelSize * 0.5 +  (1 - uv.y) * (1.0 - yPixelSize);
15.  
16.             //yOffset = 1 - uv.g;
17.             fixed4 slice0Color = tex2D(lutTex, float2(s0, yOffset));
18.             fixed4 slice1Color = tex2D(lutTex, float2(s1, yOffset));
19.             float zOffset = frac(uv.z * innerWidth);
20.             fixed4 result = lerp(slice0Color, slice1Color, zOffset);
21.             return result;
22.         }

Now with the standard LUT texture applied, scene image is:


While without LUT, the scene image is,


As you can see, there is still some artifacts. Any idea why is this happening?

 

Yeah, that looks like bleeding because of linear interpolation. You should be able to avoid it by clamping your sampling area to the centers of the border pixels (0.5/16 .. 15.5/16). Another option is to modify the LUT itself and duplicate every column of pixels that's next to another slice and change the uvs in shader accordingly. Also, make sure your wrap mode is set to Clamp.

 

Can you explain this a bit further? I have no idea how could I do that.

 

Code (CSharp):

1. uv.x = clamp(uv.x, 0.5 / width, (width - 0.5) / width)

By the way, I'm not fully following the code you have there... It looks you have some half pixel offsets in there for some reason, so I can't say if the above is going to work with it or not.

 

I am afraid, that did not work.

Now with clamp, the modified code of that function is:

Code (CSharp):

1.  
2.         float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width)
3.         {
4.             //uv.x = clamp(uv.x, 0, (width -0.5) / width);
5.             float innerWidth = width - 1.0;
6.             float sliceSize = 1.0 / width; // space of 1 slice
7.             float slicePixelSize = sliceSize / width; // space of 1 pixel
8.             float sliceInnerSize = slicePixelSize * innerWidth; // space of width pixels
9.             float zSlice0 = min(floor(uv.z * innerWidth), innerWidth);
10.             float zSlice1 = min(zSlice0 + 1.0, innerWidth);
11.             float xOffset = slicePixelSize * 0.5 + uv.x * sliceInnerSize;
12.             float s0 = xOffset + (zSlice0 * sliceSize);
13.             float s1 = xOffset + (zSlice1 * sliceSize);
14.             float yPixelSize = sliceSize;
15.             float yOffset = yPixelSize * 0.5 +  (1 - uv.y) * (1.0 - yPixelSize);
16.  
17.             s0 = clamp(s0, 0.5 / width, (width - 0.5) / width);
18.             s1 = clamp(s1, 0.5 / width, (width - 0.5) / width);
19.             fixed4 slice0Color = tex2D(lutTex, float2(s0, yOffset));
20.             fixed4 slice1Color = tex2D(lutTex, float2(s1, yOffset));
21.             float zOffset = frac(uv.z * innerWidth);
22.             fixed4 result = lerp(slice0Color, slice1Color, zOffset);
23.             return result;
24.         }

And this is the output with standard LUT.

 

Here's my quick implementation of the function.. see if it works with that

Code (CSharp):

1. float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width) {
2.     uv.x = clamp(uv.x, 0.5 / width, (width - 0.5) / width);
3.     uv.y = 1.0 - uv.y;
4.  
5.     float sliceBase = saturate(uv.z) * (width - 1.0001);
6.     float sliceFrac = frac(sliceBase);
7.     float2 slice0 = float2((sliceBase - sliceFrac + uv.x) / width, uv.y);
8.     float2 slice1 = float2((sliceBase - sliceFrac + 1.0 + uv.x) / width, uv.y);
9.  
10.     float4 slice0Color = tex2D(lutTex, slice0);
11.     float4 slice1Color = tex2D(lutTex, slice1);
12.  
13.     return lerp(slice0Color, slice1Color, sliceFrac);
14. }

edit: Also, double check that the wrap mode of the LUT is set to clamp and mipmapping is turned off.

 

Unfortunately the problem still persist. The import setting of the LUT texture used is:



And the result using latest respective sampleAs3DTexture() function:
Shader code:

Code (CSharp):

1.  
2. float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width)
3. {
4.         uv.x = clamp(uv.x, 0.5 / width, (width - 0.5) / width);
5.    
6.         float sliceBase = saturate(uv.z) * (width - 1.0001);
7.         float sliceFrac = frac(sliceBase);
8.         float2 slice0 = float2((sliceBase - sliceFrac + uv.x) / width, 1.0 - uv.y);
9.         float2 slice1 = float2((sliceBase - sliceFrac + 1.0 + uv.x) / width, 1.0 - uv.y);
10.    
11.         float4 slice0Color = tex2D(lutTex, slice0);
12.         float4 slice1Color = tex2D(lutTex, slice1);
13.    
14.         return lerp(slice0Color, slice1Color, sliceFrac);
15.     }
16.  

Scene image:



For second option, I converted LUT texture:

Now instead of 256x16, its 256+16 = 272x16
Any idea of how the respective function should change to cope with?

 

As far as I can tell from that sampleAs3D code it's already taking into account pixel buffer by scaling the UVs instead of clamping.

What settings are you using on the LUT texture for import?

 

Here is the most odd thing I just dont understand. If I hard code the previous function into this:

Code (CSharp):

1. float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width)
2.         {
3.  
4. float innerWidth = 15;
5.     float sliceSize = 1.0 / 16; // space of 1 slice
6.     float slicePixelSize = 1 / 256; // space of 1 pixel
7.     float sliceInnerSize = 15/256; // space of width pixels
8.     float zSlice0 = min(floor(uv.b * 15), 15);
9.     float zSlice1 = min(zSlice0 + 1.0, 15);
10.     float xOffset = (1 / 256) * 0.5 + uv.x * (15/256);
11.     float s0 = xOffset + (zSlice0 / 16);
12.     float s1 = xOffset + (zSlice1 / 16);
13.     float yPixelSize = 1 / 16;
14.     float yOffset = yPixelSize * 0.5 +  (1 - uv.y) * (1.0 - yPixelSize);
15.  
16.     float4 slice0Color = tex2D(lutTex, float2(s0, yOffset));
17.     float4 slice1Color = tex2D(lutTex, float2(s1, yOffset));
18.     float zOffset = frac(uv.z * innerWidth);
19.     float4 result = lerp(slice0Color, slice1Color, zOffset);
20.     return result;
21. }

The bleeding problem is gone! But the overall color does change into something this weird:



When we are expecting just normal unchanged color of the scene with standard LUT, just like without this image effect. This:

 

Please see the previous replay, I posted the snapshot of inspector.

 

The reason the bleeding problem is gone in your red tinted example is because it's caused by red values too close to 0 and 255. I thought the clamping to pixel centers would have solved it, but for some reason it doesn't...
Anyway, the modified LUT needs to have duplicate columns on both sides, producing a texture that is 18 * 16 pixels wide. Then simply replacing the first clamp line with this should work, unless I miscalculated:

Code (CSharp):

1. uv.x = saturate(uv.x) * (1.0 - 2.0 / (width + 2)) + 1.0 / (width + 2);

 

So I duplicated each slice's left pixel column(right one already I have done earlier) and the modified LUT texture(which is 288x16 now, with each slice being 18x16) is:


The I replaced the clamp code with newest, so the function becomes:

Code (CSharp):

1.  
2. float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width) {
3.         uv.x = saturate(uv.x) * (1.0 - 2.0 / (width + 2)) + 1.0 / (width + 2);
4.         uv.y = 1.0 - uv.y;
5.    
6.         float sliceBase = saturate(uv.z) * (width - 1.0001);
7.         float sliceFrac = frac(sliceBase);
8.         float2 slice0 = float2((sliceBase - sliceFrac + uv.x) / width, uv.y);
9.         float2 slice1 = float2((sliceBase - sliceFrac + 1.0 + uv.x) / width, uv.y);
10.    
11.         float4 slice0Color = tex2D(lutTex, slice0);
12.         float4 slice1Color = tex2D(lutTex, slice1);
13.    
14.         return lerp(slice0Color, slice1Color, sliceFrac);
15.     }
16.  

Still bleeding has not gone. Weird!

 

Hmm that's really weird... try messing around with the border value here:

Code (CSharp):

1. float border = 1.0;
2. uv.x = saturate(uv.x) * (1.0 - (border * 2) / (width + 2)) + border / (width + 2);

See how much you need to increase it to get rid of the bleeding.

 

@Dolkar When the border is roughly 8.78, the bleeding is gone. But the scene color changes into this:

Now everything is reddish.

 

8.78?! And no less? That would mean it's bleeding even though you're sampling pixels nowhere near the borders! It makes no sense...
One important note.. are you testing this on PC or phone?

Can you try what effect changing filtering to point has? I'm running out of ideas here...

 

I am testing it on both editor and android phone. Looks like if I use this function:

Code (CSharp):

1. float4 sampleAs3DTexture(sampler2D lutTex, float3 uv, float width) {
2.  
3.             uv.x = saturate(uv.x) * (1.0 - (_borderAreaLUT * 2) / (width + _widthAddLUT)) + _borderAreaLUT / (width + _widthAddLUT);
4.             uv.x = clamp(uv.x, 0.5 / width, (width - 0.5) / width);
5.             uv.y = 1.0 - uv.y;
6.        
7.             float sliceBase = saturate(uv.z) * (width - 1.0001);
8.             float sliceFrac = frac(sliceBase);
9.             float2 slice0 = float2((sliceBase - sliceFrac + uv.x) / width, uv.y);
10.             float2 slice1 = float2((sliceBase - sliceFrac + 1.0 + uv.x) / width, uv.y);
11.        
12.             float4 slice0Color = tex2D(lutTex, slice0);
13.             float4 slice1Color = tex2D(lutTex, slice1);
14.        
15.             float4 lerpedColor = lerp(slice0Color, slice1Color, sliceFrac);
16.             //lerpedColor.r -= 0.1098039215686275;//hack
17.             //lerpedColor.g -= 0.0313725490196078; //hack
18.           //  lerpedColor.b -= _BlueCorrection / 255;
19.  
20.             return lerpedColor;
21.         }

The bleeding is gone with standard neutral LUT image, but scene is a bit blueish.


As you can see in commented code, no matter what I do, scene does not go normal if using standard LUT. Bleeding is gone with "this setup", will be broken as hell if I modify standard LUT in photoshop; bleeding appears again.