Image Effect: Edge Detect Normals Colours [rel]

(Now updated to work up to 5.4.0f3)
As I was working on my project, I was looking into edge detecting and the current image effects edge detection doesn't offer the ability to change the colours of the edges it detects. Instead it only renders it black.

I've edited the shader and editor to add another image effect called Edge Detection Color. It allows you to edit the colour of the edges, changing them from black to any other color, selectable in the editor. The required files are attached to this post.

How to install: Unity Image Effects files are required (Import Unity Effects package)
Put EdgeDetectionColorsEditor file in the Editor/ImageEffects folder,
Put the other two (EdgeDetectColor.cs & EdgeDetectNormalsColor.shader) anywhere else.
Add EdgeDetectionColour to camera through Image Effects > Edge Detection > EdgeDetectionColor / searching for EdgeDetectionColor


Note: only works for triangle depth normals and robert cross depth normals

A quick example of what it lets you do:
Before:

After:

 

@pmurph03 this is just want I wanted, but I'm getting an error: "Missing shader in Main Camera when" when I attach the script to my camera, do you have any suggestions as to how I can fix this?

 

I was able to get it to work by adding:

Code (csharp):

1.  
2. public override bool CheckResources ()
3. {
4.     CheckSupport(true);
5.  
6.     if (edgeDetectShader == null)
7.     {
8.         edgeDetectShader = (Shader)Resources.Load ("EdgeDetectNormalsColor");
9.     }
10.     .
11.     .
12.     .
13. }
14.  

 

pmurph03 hey, thanks for scripts!
very helpfull for me, looks so nice

 

@pmurph03
THANK YOU! This is exactly what i have been looking for. I was using the unity edge detection scrpt, and it wasnt doing the job like i wanted. this is exactly perfect. You are the man!

 

This is excellent. I was really struggling on how to do this so thank you.

Does anyone know if it is at all possible to either - 1) have the color key off say for instance an objects material color or 2) exclude certain objects from the edge detection effect?

 

I've followed the instructions, but I'm getting an error:

Unfortunately, I can't figure out what the issue is. Anyone had the same problem?

EDIT: Fixed the issue. I misplaced the shader, now it works just fine. Sorry for the inconvenience!

 

Nice work.

Thanks.

 

When trying out your scripts, I'm receiving two errors.

Assets/Editor/ImageEffects/EdgeDetectionEditor.cs(8,11): error CS0101: The namespace `UnityStandardAssets.ImageEffects' already contains a definition for `EdgeDetectionEditor'


NullReferenceException: Object reference not set to an instance of an object
UnityStandardAssets.ImageEffects.EdgeDetectionColor.OnRenderImage (UnityEngine.RenderTexture source, UnityEngine.RenderTexture destination) (at Assets/Scripts/EdgeDetectionColor.cs:81)

Seeing as how i'm trying to learn how shaders and things work, I'm not sure how to go about finding the cause of these issues. The first error really has me confused, since I have searched the EdgeDetectionEditor.cs script and find no instance of "EdgeDetectionEditor" in the script itself. So i'm not sure where it's finding that definition.

 

@Rottiger I know it's been several months since you solved your problem, but do you remember what you did exactly? I have the exact problem you did, and I've tried a bunch of different things (including shuffling the shader around) but I can't seem to fix it. (Also I'm on Unity Personal v5.3.3f1, if that has any significance)

 

#1. You can't have two scripts with the same name in your project. Either delete the duplicate or rename one, both the file and the namespace.

#2. Look at line 81 of the script just like the error tells you. Likely you have a missing pointer and the script is trying to utilize some component but you haven't told it what to use.

 

Bump.

 

@Kayos Yeah, the issue seems to be that it can't track down the shader. I was able to get it to work by doing the following:

• Place your "EdgeDetectNormalsColor.shader" in Standard Assets > Effects > ImageEffects > Shader
• Add the following line of code to "EdgeDetectionColor.cs" at line 40/41 (just after CheckSupport(true); )
• Code (csharp):

  1. edgeDetectShader = Shader.Find ("Hidden/EdgeDetectColors");

 

I apologize for the lack of support! I didn't realize that anyone had used this, so I'm very sorry for the late update.

I just quickly updated this shader to work in Unity 5.4.0f3, as well as fixing some bugs that people pointed out. I'll also update the original post to include more information about proper installation. Original post attachments have also been updated to the latest version.

Attached are files for 5.4.0f3

 

Thank you pmurph03. Works well on mobiles too.

 

Hi, I'm unity 5.6 and I kept the EdgeDetection image effect but it has a bad issue: the thickness of lines changes based on screen size.
Does this effect have the same issue?

 

I believe if you modify the script that goes on the camera to multiply the sample distance based on screen resolution there shouldn't be a change of percieved thickness based on screen size.

 

Hi pmurph, I am using Unity 5.6, I added these files in the correct place as well as importing Unity's image effects but I get the error:
The type or namespace name `PostEffectsBase' could not be found.

Am I missing something else? This class didn't seem to be included in Unity's Effects package.

Cheers!

 

Okay I found the issue, I ended up on a wild goose chase back to the Legacy image effects which aren't included in the newer versions of Unity: https://www.assetstore.unity3d.com/en/#!/content/83913

The effect is perfect, thanks for releasing this!

 

How get the edge detect shader to work on backfaces as well?
It seems to ignore backfaces.
Crease effect also ignores back faces.

Other image effects like bloom or contrast enhance do process back faces.

 

That's not a problem with the image effect, that's a problem with the two sided shader you're using. Or more specifically is a problem with how Unity generates the depth normals texture when using the forward rendering path, which requires manually editing a built in editor shader.

Switching to deferred should fix the issue, or using a depth only outline image effect (which this image effect doesn't allow you to select, it's the Sobel option in the original Unity image effect), or by adding a double sided pass to the internal-DepthNormalTexture.shader

 

I have tried to adapt the Internal-DepthNormalTexture.shader - by adding "Cull Off" into the opaque subshader, but it did not help. I guess it might be not exactly what you meant.

But the deferred mode and Sobel option worked.

 

What render type is the two sided shader using?

 

Some are using opaque, others alpha-test.

 

If you're directly modifying the shader in the editor folder you have to delete your project's shader cache for it to take effect. If you add the modified copy of the shader to your assets folder and set it as the override in your graphics settings then it should work with that change.

 

would it be possible to assign different colours to object based on tag and or layer?

 

I would just like to modify this effect shader to outline negative normal areas i.e. to detect where normals change from positive to negative values.
I have tried to modify the CheckSame function in EdgeDetect shader:

Code (CSharp):

1.         half CheckSame(half4 center, half4 sample) {
2.             half2 centerNormal = center.xy;
3.             float centerDepth = DecodeFloatRG(center.zw);
4.             half2 sampleNormal = sample.xy;
5.             float sampleDepth = DecodeFloatRG(sample.zw);
6.            
7.             // difference in normals
8.             // do not bother decoding normals - there's no need here
9.             half2 diffNormal = abs(centerNormal - sampleNormal) * _Sensitivity.x;
10.  
11.             half ndot = dot(centerNormal, sampleNormal);//!!!!!!!!!!!!!!!!!
12.  
13.             int isSameNormal = (diffNormal.x + diffNormal.y) < 0.1;
14.             // difference in depth
15.             float diffDepth = abs(centerDepth - sampleDepth) * _Sensitivity.y;
16.             // scale the required threshold by the distance
17.             int isSameDepth = diffDepth < 0.1 * centerDepth;
18.            
19.             // return:
20.             // 1 - if normals and depth are similar enough
21.             // 0 - otherwise
22.  
23.             return ndot>0? 1.0 : 0.0;//!!!!!!!!!!!!!!!!!!!!
24.  
25.             //return isSameNormal * isSameDepth ? 1.0 : 0.0;
26.         }

But after these modifications the shader doesn't show any outlines at all. Please give some hint if you can.

 

What exactly are you trying to test for that the original could not? Also you're testing the encoded normals which are values in 0...1 ranges, thus a dot product will basically always we greater than zero. You need to decode the view normals before you use them in a dot product, and even then it's going to be be fairly infrequent for that dot product to be negative.

half3 centerNormal = DecodeViewNormalStereo(center);
half3 sampleNormal = DecodeViewNormalStereo(sample);

 

Yes, that works great.
But I am missing the viewing direction - dont't know how to get it at the moment.
Basically I need to check the dot(viewdir, normal).
I guess it could be a matter of simple calculation, basing on texture.uv and it just equals to view space in the middle of the view (0.5, 0.5) and then diverges towards the edges.

 

can you merge outlines of two objects together with this?

 

@pmurph03 thanks a lot!!!
that exactly what I was looking for
small questions - is it possible for edges to fade with distance from camera? (and possibility to tweak that distance)
thx!

 

any way to control detected edge thickness?

 

Is there a version that doesn't require ImageEffects?
Also - antialiasing?

 

Is there anyway to change color and stack colors?

 

Is this still usable somehow in modern versions of Unity and on Universal Render Pipeline? "Unity Image Effects" no longer seems to exist.

 

The same base techniques are still usable, but organized in new ways:
https://alexanderameye.github.io/outlineshader.html

 

Does anyone know of a way to achieve this effect with HDRP?

 

I have two questions for this method. I want to use it in vr project. First problem is anti aliasing. Lines are flick too much.I could not find any solution for this. Second problem , It is add everywhere lines, UI included. How can we assign specific object. Maybe It should assign only layers. Otherwise, It is working great. If this problems solved , It can be best solution.

 

For HDRP you have to make this a custom post process.
https://docs.unity3d.com/Packages/c...efinition@7.1/manual/Custom-Post-Process.html
The approach used above is the legacy Built In Rendering Path image effect method ... because it's from 6 years ago and predates even the current BIRP Post Processing Stack stuff which is similarish to what the HDRP uses.

Yeah ... that's going to be a hard nope.

Some VR games do use outline methods similar to this (the PC version of Echo Arena for example), but you have to use TAA at the same time to get clean outlines. And Unity's TAA is ... a bit rough to use with VR, especially as there's no (easy) way to disable jitter which you don't really want to have on for VR TAA.

Yep. That's the explicit goal of this specific implementation. Everything gets and outline. You cannot avoid that. If you have UI elements that are rendering in the opaque queue, then they're going to get outlines too. Again, you cannot avoid this, both because of how this particular effect is written and because of how Unity handles post processing.


If you want outlines that work well with MSAA, you cannot use a post processing effect like this that makes use of depth and normals because the camera depth and normals textures, and those are do not (and cannot) use MSAA. That's not something Unity supports doing at all.

If you want just an outline, you might want to look into using old school inverted hull outlines. Like this:
https://github.com/chrisnolet/QuickOutline

For Falcon Age I used a post processing based outline that works a bit like this asset:
https://github.com/cakeslice/Outline-Effect

For the Quest version of Falcon Age any kind of post processing was way, way too slow. The QuickOutline above probably would have worked, but I instead used a technique that renders each object 4 times slightly offset in screen space to produce an outline. This was more expensive than using an inverted hull, but still not as expensive as I feared it could be, and let it work on any mesh without any kind of additional setup step like the QuickOutline does. But I don't have an example shader / project of that to share.


These won't do the interior edge lines though, only the exterior outline. The only way to do that nicely for MSAA is to draw bespoke "edge" lines as geometry, or bake it into custom per mesh textures.

 

Thanks for answer. After I try them , I see that It is difficult to decide which method is suitable. We are working on quest game too. I need to select cheapest one. I will look at your other advise too.