Search Unity

Temporal Anti-Aliasing

Discussion in 'Image Effects' started by Tim-C, Jun 21, 2016.

Page 2 of 5

  1. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    This is correct.
     
    Tim-C, Jul 20, 2016
    #51

  2. rbmcginnis

    rbmcginnis

    Joined:
    Feb 3, 2014
    Posts:
    3
     
    rbmcginnis, Jul 20, 2016
    #52

  3. rbmcginnis

    rbmcginnis

    Joined:
    Feb 3, 2014
    Posts:
    3
    OK, Thanks for the installation tips. I did get it installed and operational. I am working on a driving simulator. The TAA script was great at removing the aliasing effects several hundreds of meters in front of the car, but the downside is the bluriness of the scene 30 meters in front of the car. The one control in the TAA script did not have any effect on this. Any advice?
     
    rbmcginnis, Jul 20, 2016
    #53

  4. blackbird

    blackbird

    Joined:
    Aug 9, 2011
    Posts:
    592
    in terms of performance is this similar to fxaa3 or more heavier ?
     
    blackbird, Jul 28, 2016
    #54

  5. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    Hey,

    We have been trying TAA in a couple of projects, and currently are finding it pretty unusable. On static objects, we are often losing a significant amount of texture detail, and on skinned meshes the entire object becomes a blurry mess. I have used Keijiro's motion vector debugger to ensure skinned meshes are actually writing velocity vectors, and it appears that is all correct.

    I understand some of these issues are innate to temporal anti-aliasing techniques, but what we are experiencing seems to be much worse than expected.

    The edges however, are lovely :)

    Just a bit more info. We are deferred rendering in linear mode, using 5.4.0f2. Removing all other image effects doesn't help the situation either. The screenshots are taken without camera movement, and only get worst with any movement whatsoever.


    SMAA
    Screenshot_SMAA_Crop.png


    TAA
    Screenshot_TAA_Crop.png

    Thanks
     
    Chris_Webb, Jul 28, 2016
    #55
    eskovas likes this.

  6. eskovas

    eskovas

    Joined:
    Dec 2, 2009
    Posts:
    1,373
    wow :eek:
    I have to say that looks amazing!
    Sorry to be off topic, but what game is this? do you have a link?
     
    eskovas, Jul 28, 2016
    #56

  7. chiapet1021

    chiapet1021

    Joined:
    Jun 5, 2013
    Posts:
    605
    I'm finding similar issues with loss of detail to go with the awesome edge softening. Adding a second sharpening post effect helps with that, but that obviously adds additional graphics overhead.
     
    chiapet1021, Jul 28, 2016
    #57

  8. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    Hi, is there anyway you would give us a copy of this scene from this angle (your exact test setup). Feel free to strip everything out apart from the meshs / materials / textures. We'd love to use this internally to help improve the effect. If you raise a bug with the project attached or just get a URL for where we can grab it that would be so so helpful.
     
    Tim-C, Jul 28, 2016
    #58
    Martin_H and chiapet1021 like this.

  9. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    No problem. I'll strip down the project and try and get that to you as soon as possible. Thanks a lot.


    haha thanks. The game is Hand of Fate 2
    http://store.steampowered.com/app/456670/
     
    Chris_Webb, Jul 29, 2016
    #59
    Kronnect, eskovas, Arganth and 2 others like this.

  10. eskovas

    eskovas

    Joined:
    Dec 2, 2009
    Posts:
    1,373
    Oh nice! The first one was pretty awesome :)
    Looking forward to the second one. It's looking really good
     
    eskovas, Jul 29, 2016
    #60

  11. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    Hey Tim. I've submitted a stripped down project. The issue is 818538. Feel free to let me know if you need anything else regarding this.


    Thanks a lot :)
     
    Chris_Webb, Jul 30, 2016
    #61

  12. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    Sure. Probably won't hear anything from us for a week or two on this one unfortunately, the dev who works on TAA is at siggraph then taking some summer time. He'll action when he gets back.
     
    Tim-C, Jul 30, 2016
    #62

  13. ArchVizPRO

    ArchVizPRO

    Joined:
    Apr 27, 2014
    Posts:
    457
    Hello
    When i use Taa with SteamVr i got a black screen.
    I am using Unity 5.4.0f3 and latest Taa from bitbucket.
    Same issue with Forward and Deferred, single pass stereo render disabled.

    Please someone let me know is this is only a my issue.
    Thank you!
     
    ArchVizPRO, Aug 1, 2016
    #63

  14. movra

    movra

    Joined:
    Feb 16, 2013
    Posts:
    566
    I've found another TAA on the Asset Store that advertises itself as being much sharper than Unity's Cinematic Image Effects TAA. The stills are indeed in favor of that other asset, though when moving in the web demo it's also blurry.

    https://www.assetstore.unity3d.com/en/#!/content/65513

     
    movra, Aug 2, 2016
    #64

  15. chiapet1021

    chiapet1021

    Joined:
    Jun 5, 2013
    Posts:
    605
    I haven't bought that asset, but I suspect the sharper look is by adding the sharpening pass, which is a configuration noted in that asset's documentation link and adds an additional draw call. You could potentially do the same thing by adding a separate sharpening effect (e.g., Beautify) on top of Unity's TAA.
     
    chiapet1021, Aug 2, 2016
    #65

  16. blackbird

    blackbird

    Joined:
    Aug 9, 2011
    Posts:
    592
    @movra i had a small chat with the dev of TAA from asset store and told him about the blur issue he told me you have to keep the framerate over 30 frames if it drops below it become fuzzy/blurry
     
    blackbird, Aug 2, 2016
    #66

  17. movra

    movra

    Joined:
    Feb 16, 2013
    Posts:
    566
    Right, it's called Anti-Blur. I wonder what algorithm TAA uses and if it's different from Beautify's. Because there are several types of sharpening, eg. unsharp mask and image restoration through deconvolution.

    Now that you mention it I remember reading TAA sharpening needs enough frames to work with. The framerate of the WebGL demo must be below 30 fps on my machine, so I can't verify it.
     
    movra, Aug 3, 2016
    #67

  18. larsbertram1

    larsbertram1

    Joined:
    Oct 7, 2008
    Posts:
    6,902
    that web player is pretty much worthless, sorry. taa was introduced to antialiase geometry and specular artifacts but the demo deos not show any bump mapped specular/smooth surfaces at all.
    naughty dog has just published the efforts on taa in uncharted 4 – and as they mention a lot of caveats concerning texture animations, shaders using vertex displacement or mirror reflections etc. i think taa should be a globally (unity authored) available solution tested with many different assets in many different scenarios.
    not to mention other shaders to take advatage of taa – like dithered alpha shaders.
     
    larsbertram1, Aug 3, 2016
    #68

  19. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Excellent post, fully agreed.
     
    hippocoder, Aug 3, 2016
    #69

  20. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    We have some changes coming very soon that make some nice improvements:
    *Added sharpen filter
    *Improved heuristic for blending history buffer (better convergence / much less crawling on static frame)

    No sharpen:


    Sharpen:
     
    Tim-C, Aug 3, 2016
    #70
    chiapet1021 likes this.

  21. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    Awesome, can't wait to try it out :)
     
    Chris_Webb, Aug 3, 2016
    #71

  22. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    That's sweet - hope the sharpening amount is configurable (ofc it is, duh!)

    Is that one of the secret of secrets?
     
    hippocoder, Aug 3, 2016
    #72

  23. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    Of course.

    Nope.
     
    Tim-C, Aug 3, 2016
    #73

  24. artur-leao

    artur-leao

    Joined:
    Feb 3, 2015
    Posts:
    66
    This is magic and I'm pretty happy to see this in Unity, but is it just me or the result is very very blurred? I've tried it on a project and it looked extremely blurred. Sample images posted here look fine though.
     
    artur-leao, Aug 3, 2016
    #74

  25. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    We have an update that will drop in a day or two that will fix a bunch of the blurriness :)
     
    Tim-C, Aug 3, 2016
    #75

  26. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    We just tried the latest version and the result was significantly better. Awesome work with it, and thanks a lot! :)
     
    Chris_Webb, Aug 4, 2016
    #76

  27. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    My dreams of actually having something better than FXAA are realised on console.
     
    hippocoder, Aug 4, 2016
    #77

  28. topofsteel

    topofsteel

    Joined:
    Dec 2, 2011
    Posts:
    999
    The cinematic TAA breaks my WebGL build in 5.4. I want to say it worked in 5.3 but I havent do
     
    topofsteel, Aug 4, 2016
    #78

  29. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    TAA will never work in 5.3 as it's not supported in 5.3. We are not sure if taa will be available on webgl, we have not done any testing there yet.
     
    Tim-C, Aug 4, 2016
    #79

  30. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    In case anyone is interested, here are the current performance numbers for us on PC for comparison.

    TAA Post Effect- 0.61ms
    Motion Vector pass with about 50 drawcalls- 0.14ms
    TAA Total- 0.75ms

    SMAA(high)- 0.62ms
    FXAA3- 0.2ms

    I will try to get console values some time also, but so far performance is better than expected.
     
    Chris_Webb, Aug 5, 2016
    #80
    Martin_H likes this.

  31. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,795
    Considering that's the most effective post AA I've seen so far (assuming the issues that have cropped up are fixed), the performance is pretty good.
     
    AcidArrow, Aug 5, 2016
    #81

  32. Chris_Webb

    Chris_Webb

    Joined:
    Mar 18, 2014
    Posts:
    128
    Somewhat unrelated, but I'm getting a bunch of issues related to the camera depthTextureMode, and other effects (which are enabled/disabled though out the game) interfering with TAA (I was wondering why occasionally it looked terrible. Turns out motion vectors were being turned off when other effects changed the depthTextureMode).

    Is there a proper solution to this? I can't really think of a good way to do it unless the buffer requirements were a stack, and effects could push/pop them.

    Obviously this is something I can sort out by changing a bunch of effects, so it doesn't effect me greatly, but other people will certainly have this issue, and having depthTextureMode be more usable across packages is probably a good thing.

    The current quick solution is just to change the OnEnable code in TAA to run on update
     
    Last edited: Aug 5, 2016
    Chris_Webb, Aug 5, 2016
    #82

  33. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    The only real solution is to have something that runs at the start of every frame (pre render maybe) that scans the image effects and builds a list of the needed flags.

    Basically an interface like:
    Code (csharp):
    1.  
    2. //interface
    3. DepthTextureMode neededDepthFlags{get;};
    4.  
    5. var mode = DepthTextureMode.None;
    6. foreach(var effect in effects)
    7. {
    8.     mode |= effect.neededDepthFlags;
    9. }
    10. camera.depthTextureMode = mode;
    11.  
    The depth texture stuff is pretty old and pretty badly designed :/

    Yeah they need depth textures, if you enable motion vectors it will tell you you are enabling depth also as a warning.

    Uber effect will fix this :)
     
    Tim-C, Aug 5, 2016
    #83
    hippocoder and Chris_Webb like this.

  34. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    on PS4 1080p we are seeing TAA take 1.6ms heavily texture bound, we are looking into how to make this faster (hoping to get to 1ms).

    on xbone 1080p we are seeing TAA take 1.3ms also heavily texture bound.

    We think that given GPU's etc that ps4 should be faster. We still need to investigate this a little. One thing to note is that currently we have been trying to get the effect correct first. Now that we are happy with how it looks we are going to start trying to claw back as many ms as we can with some optimisation work :)
     
    Tim-C, Aug 5, 2016
    #84
    hippocoder and Chris_Webb like this.

  35. blackbird

    blackbird

    Joined:
    Aug 9, 2011
    Posts:
    592
    so it's cpu bound ?
     
    blackbird, Aug 5, 2016
    #85

  36. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    Nope. It's texture bound. This means that in the shader it's waiting for texture reads before it can do more processing using the results.
     
    Tim-C, Aug 5, 2016
    #86
    blackbird likes this.

  37. fffMalzbier

    fffMalzbier

    Joined:
    Jun 14, 2011
    Posts:
    3,276
    Now that 5.4 is finally out , what is the plan on making it VR Compatible?
    VR is currently limited to Forward rendering because all the defered rendering Antialiasing options do not work for VR.
    Heaving temporal anti-aliasing work in VR would bring the graphical possibility for VR applications forward in a immense way.
     
    fffMalzbier, Aug 5, 2016
    #87

  38. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    We've started porting the effects (AO, and bloom so far :) ) and will continue to port the ones that make sense (motion blur is great for making you feel sick but not much else for example). We'll get to TAA soon enough, but it needs a little work on stereo motion vectors first.
     
    Tim-C, Aug 5, 2016
    #88
    fffMalzbier likes this.

  39. ArchVizPRO

    ArchVizPRO

    Joined:
    Apr 27, 2014
    Posts:
    457
    It was removed from Asset Store.... why ?
     
    ArchVizPRO, Aug 5, 2016
    #89

  40. ArchVizPRO

    ArchVizPRO

    Joined:
    Apr 27, 2014
    Posts:
    457
    Totally agree... for VR it would be awesome
     
    ArchVizPRO, Aug 5, 2016
    #90

  41. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    The latest verions on repository looks awsome! I just whish you guys could make it work for VR. I have been unable to find a good AA solution for VR in deferred rendering mode :(
     
    olavrv, Aug 5, 2016
    #91
    Alverik likes this.

  42. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    Unfortunately it also depends on motion vectors which currently don't work well in VR. So we need to fix motion vectors for stereo rendering (part of the engine codebase so it will come in a future Unity patch/release) and then fix TAA.
     
    Chman, Aug 5, 2016
    #92

  43. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    Fantastic! A lot of pieces of the puzzle have been solved, and this will definitely be a major one. Its a bit of a mess with some 3rd party assets working only on deferred.. keep up the good work!
     
    olavrv, Aug 5, 2016
    #93

  44. ArchVizPRO

    ArchVizPRO

    Joined:
    Apr 27, 2014
    Posts:
    457
    Latest release is a great step forward ! Looks awesome in many situations.
    But on trasparent objects (standar shader) it fliker a lot ( really a lot ).
     
    ArchVizPRO, Aug 6, 2016
    #94

  45. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    It runs after opaque and not on transparents by default. you can make it run on transparents by removing the [ImageEffectOpaque] flag in the c# file.
     
    Tim-C, Aug 6, 2016
    #95

  46. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Regarding transparent queue things - does it treat them as "background" when performing motion blur? that would be ideal in our case I guess - either all or nothing (nothing for UI) ? Or what plans are there for these cases including particles? I don't need accuracy but I also don't need things that render out of place in an obvious way.
     
    hippocoder, Aug 6, 2016
    #96

  47. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,225
    You can run taa after transparent rendering and it mostly doesn't look terrible. The issue is that it kind of mutes a lot of small particles due to using the history buffer and not having motion vectors to make up for it. A good example it to take unity labs and look at he room with the particles falling from the ceiling. It looks okay with taa after transparency, but you lose a lot of the really tiny sparks. There currently isn't a good taa + transparency solution.
     
    Tim-C, Aug 6, 2016
    #97

  48. karonte

    karonte

    Joined:
    Jun 2, 2013
    Posts:
    48
    it's no longer available...they are not serious developer. Keep in mind their name studio and never buy from them again.
     
    karonte, Aug 7, 2016
    #98

  49. semaphore

    semaphore

    Joined:
    Mar 21, 2011
    Posts:
    49
    How exactly do I get the Temporal AA effect?

    When I download the package from the asset store I only have FXAA and SMAA, and in SMAA component there is a "temporal" tick. Is this what being discussed here? Because it looks different than the component shown in this video




    And the result I am getting is quite shaky in all my test scenes.
     
    semaphore, Aug 24, 2016
    #99

  50. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,352
    See the first post of this thread.
     
    bgolus, Aug 24, 2016
    #100
Page 2 of 5