Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Temporal Anti-Aliasing

Discussion in 'Image Effects' started by Tim-C, Jun 21, 2016.

  1. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Temporal AntiAliasing.

    Hi everyone, for the past few months we have been working on an Temporal Anti Aliasing solution for Unity. Temporal anti aliasing is a modern form of antialiasing that uses previous frames data to infer more about edge and highlight information. We first showed the TAA in the Adam demo and it made it look really great!

    TAA required a few changes to the core of Unity for it to be supported in the engine (specifically motion vectors and some surrounding support), in 5.4 we have added this functionality and implemented a version TAA. This will also allow us to ship a really great motion blur quite soon also.

    If you are curious about the technical side of things you can read about some general TAA things here: https://bartwronski.com/2014/03/15/temporal-supersampling-and-antialiasing/


    You can get the new TAA from here:
    https://bitbucket.org/Unity-Technologies/cinematic-image-effects/branch/taa
    This branch also contains the latest version of our other effects.

    You will need Unity 5.4b22 for this to work properly. Here are some images showing how this looks for us. NOTE: You will need to enlarge the images too see the difference.



    Click this link to see before and after.





    Let us know how it goes for you!
     
    Last edited: Jun 24, 2016
    elmax, ArthurT, Jingle-Fett and 12 others like this.
  2. Mahdi-Jeddi

    Mahdi-Jeddi

    Joined:
    Dec 21, 2012
    Posts:
    27
    You should do a view to showcase the REAL benefits of temporal AA. also, some numbers to compare the performance of these to the FXAA and other solutions would be awesome.
     
  3. jashan

    jashan

    Joined:
    Mar 9, 2007
    Posts:
    3,307
    Looks awesome, thank you for sharing!

    Would TAA be useful for VR? In other words: Does it not impact performance too severely for typical VR games, and does it address the specific issues you're facing with non-AA'ed scenes in VR?

    As flickering is one of the issues that's super-annoying in VR, it seems like TAA might be a perfect fit for VR. I'm just concerned about the performance hit.
     
  4. iivo_k

    iivo_k

    Joined:
    Jan 28, 2013
    Posts:
    314
    Enabling Temporal (Experimental) makes me see vertically mirrored jittering ghosts of the scene objects.
     
  5. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Any plans to implement a project wide texture mip bias to handle the texture over blurring of TAA?
     
  6. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    This is MSAA, not temporal AA. you need the scripts linked from the original post.
     
  7. chingwa

    chingwa

    Joined:
    Dec 4, 2009
    Posts:
    3,789
    Must. Try. ASAP.
     
  8. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Note: We just discovered a weird issue on OSX. We'll have a fix as soon as we can, still feel free to try it there, it's just a little more blurry than it should be.
     
  9. Ippokratis

    Ippokratis

    Joined:
    Oct 13, 2008
    Posts:
    1,521
    Great results, cheers !
     
  10. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Pushed a fix for this. Latest version should be good on all platforms.
     
  11. iivo_k

    iivo_k

    Joined:
    Jan 28, 2013
    Posts:
    314
    I downloaded the repo from BitBucket and used the AntiAliasing script from CinematicEffects(ALPHA)/AntiAliasing. What am I doing wrong?
     
  12. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    You need the Temporal Anti Aliasing script. Make sure you click the link in the original post as it's only on the TAA branch of the repository.
     
  13. iivo_k

    iivo_k

    Joined:
    Jan 28, 2013
    Posts:
    314
    Whoops, thanks a lot!

    Edit: Right, it seems downloading the repo just gives you the main branch, I guess I need to setup Mercurial.
     
  14. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Click on download, click on branches, click on "zip" next to taa
     
    Martin_H likes this.
  15. iivo_k

    iivo_k

    Joined:
    Jan 28, 2013
    Posts:
    314
    Turn out I'm very stupid, thanks!
     
  16. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Nah, it's just really confusing.
     
    Martin_H likes this.
  17. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    So, some people asked for comparison with FXAA/SMAA... This is what it looks like on a specular-heavy scene (worst case scenario for AA) :



    FXAA and SMAA tend to work well on non-HDR scenes without too many specular highlights, but in this case TAA helps a lot. That said, TAA is still under heavy development so keep an eye on the repository for updates !
     
  18. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Also TAA advantages become really apparent when the scene is in motion and you have crawling edges and small speculars (normally caused by high frequency PBR materials).

    Here is a video where I toggle the TAA on and off. Please turn to 1080p to see the difference.


    At about 40 seconds in the video jitters a little bit, that just my terrible capture.
     
  19. Fuegan

    Fuegan

    Joined:
    Dec 5, 2012
    Posts:
    20
    Hi,
    I just have a simple question, is this working only with forward renderring or deferred is supported?
     
  20. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Both are supported. We haven't tested forward / MSAA yet, it might be a little weird.
     
  21. iivo_k

    iivo_k

    Joined:
    Jan 28, 2013
    Posts:
    314
    Are there some specific kinds of scene setups that TAA works best and worst with? I read the Bart Wrongski blog but I'm not sure how much of that applies and which parts specifically are problematic. It did mention having to customize lots of stuff for TAA, is that still the case, or should it be just something you slap in a scene and expect to work?

    I tried TAA with a scene that has a lot of small, moving creeps with small limbs that are animated by changing the mesh manually (to avoid Mecanim overhead & use instancing) and the result wasn't good. I don't know if it's an inherently difficult use case for which it's better to use some other technique, or if it can be improved.
     
    Last edited: Jun 22, 2016
  22. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Yeah, that's a use case that will probably fall down pretty hard with TAA. TAA for moving objects needs two things:
    • The position of the object last frame (Translation Matrix)
    • The position of each vertex relative the that transform each frame
      • Easy for rigid objects (verts don't move)
      • Harder for skinned objects (verts move)
    So in your case (as you are rendering your object manually (i'm guessing using "DrawMesh"). Unity does not know the position last frame (as it's a custom draw), and it also does not know the vertex locations from last frame (as it hasn't gone through our skinning system that tracks these).

    Currently we don't support DrawMesh with any last frame information (we will be adding the ability to pass transform information, but it won't make 5.4). When this is in you can use some shader magic and write a custom motion vector pass for your custom vertex data. Something like packing that information into a free vertex stream and then using that in your custom motion vector shader to generate the difference between last frame and the current frame.
     
    Adam-Sowinski likes this.
  23. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    We haven't done a pass on TAA for VR yet but it should work for non single pass stereo rendering (have not tested, but given that there are two cameras). On a decent video card the cost is about 2ms a frame, so maybe double that for VR. It all depends on where you want to spend your frame budget. I've heard mixed reports about if you should use it on VR though... we'll see how it looks soon and then start to make a call :)
     
  24. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Note: We pushed a new version to the repository. We perform a tonemap operation now when doing calculations, and a detonemap before returning from the rendering. It fixes some small highlight issues and in general will help a little with image stability.
     
    tatoforever likes this.
  25. tatoforever

    tatoforever

    Joined:
    Apr 16, 2009
    Posts:
    4,364
    I gave it a quick go and deleted it. No options to tweaks, performs less better than the TAA made by LIMBO guys. But it's a bit better on moving objects though (probably because the use of motion buffer). I'll give it a new go once we get some options to tweak jitter parameters.
     
  26. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Sounds like you would possibly prefer to use MSAA?
     
  27. tatoforever

    tatoforever

    Joined:
    Apr 16, 2009
    Posts:
    4,364
    TAA give me better antialiasing than anything else I've tried (better than MSAA and FXAA).
    If i have control over temporal buffer and jitter parameters to tweak it would be a lot better than LIMBO guys TAA as the results on moving objects it's a lot better but current implementation jitter buffer it's unstable to my taste (that's why i need some params to tweak it). :D
    [EDIT] I'll browse the sources and see if I can expose something and/or tweak the jitter buffer.
     
    Last edited: Jun 23, 2016
  28. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I'm guessing they haven't done a performance pass on any of the post really (maybe it still wants to be merged or something) - looks WIP to me but lovely results so far... perf is a worry indeed.
     
  29. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    Didn't try it yet, but if the reason it's "slow" is that it generates the motion buffer, maybe the cost will be offset by other effects that need motion vectors (motion blur, any filter with temporal filtering)?
     
  30. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Performance has been pretty good so far in our testing (it's completely texture operation bound). It takes 2.1ms for TAA on PS4 @ 1080p. We'll be trying to make the faster (obviously). But it's not slow at all. In addition motion vector generation is also very cheap from our testing. What hardware / testing setup are you using? How are you getting your frame timings?

    The code for the TAA is pretty simple, so just jump in and modify some stuff. You can modify jitter in the OnPreRender function, and the shader has lots of #defines that you can use to change behaviour. We haven't figured out what we want to expose to UI yet. If you have some suggestions after your tweaking let me know.
     
    Martin_H and tatoforever like this.
  31. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Note: Just pushed a change:

    Reduced the general case TAA from 3 blits to 1 blit:

    https://bitbucket.org/Unity-Technol...mits/6d0e05d848782bbfbc4fe77273f810f24abe9ee8

    Instead of copying to history buffer as a separate step we double buffer (to avoid multi read / write), and write to the new history buffer / color target at the same time using MRT.

    Anyway, general case is now 1 blit / frame for TAA. 2 blits if the current image effect target is the backbuffer (as we can't bind backbuffer as MRT target).
     
    hippocoder and tatoforever like this.
  32. tatoforever

    tatoforever

    Joined:
    Apr 16, 2009
    Posts:
    4,364
    Hey Tim,
    Yeah I'm downloading the version you just pushed and will see what I can change then let you know my suggestions about what what can be exposed.
     
  33. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Some PS4 numbers from @Chman since the change:

    Awesome :)
     
  34. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Hi Tim, I'm on PS4 as well, so I need all I can get. Appreciate all your work, means a lot to me.
     
  35. jashan

    jashan

    Joined:
    Mar 9, 2007
    Posts:
    3,307
    1.4 ms sounds pretty good. 2.8 ms, of course, is fairly intense. Can't wait for the days when everything will just work with single-pass stereo rendering ;-)
     
    Alverik likes this.
  36. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    ehhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh o_O
     
    DMeville, xenius, willpower and 3 others like this.
  37. Crzed

    Crzed

    Joined:
    Oct 19, 2013
    Posts:
    25
    Would this have any application for 2d sprites or sprite tiling? The current AA method in unity brings out lines between tiles that shouldn't be there. Would TAA be any less susceptible to this?
     
  38. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    Try it and let us know? ;)
     
  39. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    You shouldn't be needing AA with 2D. If you're mixing it with 3D then a better option is to render the 2D to texture I guess or use multiple cameras? Can MSAA be toggled between camera renders? never tried it.
     
  40. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    only if you use a render texture on those cameras with msaa. I prefer to do it this way than to use the graphics settings due to finer control.
     
  41. LightStriker

    LightStriker

    Joined:
    Aug 3, 2013
    Posts:
    2,717
    Any plan on having options on it? On your example video, in most case, I would prefer it off since it gives a kind of blurriness to the whole scene, smudging the details in the texture.

    And yes, 2+ ms is intensive for someone who aims at 60FPS on PS4.
     
  42. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    It's at 1.4 ms right now. Which I think is reasonable for such a high quality AA.
     
  43. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    It won't be the only cost though, bloom is a must along with tonemapping, grading and motion blur. So I'm hoping that further savings will come with them all combined. I probably will skip SSRR and AO and probably will skip DOF if it's too expensive too... Kind of want to keep all my post <3ms if possible.

    That's just the AA, right? need to factor in the motion vectors cost as well. But since I absolutely must have motion blur, that at least gets used twice...
     
  44. LightStriker

    LightStriker

    Joined:
    Aug 3, 2013
    Posts:
    2,717
    Which is why having some options to reduce the quality would be nice. We have ton of other post effect to factor in; bloom, lense flare, lense dirt, tonemap, AO.

    SSRR is out of question for 60FPS, we learned that the hard way.
     
  45. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I think once they combine the post, the savings will be bigger, I wonder if there's a roadmap for cinematic effects?
     
  46. FPires

    FPires

    Joined:
    Jan 5, 2012
    Posts:
    151
    Tested it out,I'm very happy that we're finally getting TAA in Unity. The current version is looking pretty decent.
     
  47. rbmcginnis

    rbmcginnis

    Joined:
    Feb 3, 2014
    Posts:
    3
    Is there any way to get access to the TAA filter without going through bitbucket. I am not allowed to set this up on my PC.
     
  48. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    You can get it from the asset store if you like. Should be updated by now. And what do you mean "not allowed to set this up" ? it's just a project you can download and copy over the folders...
     
  49. Obsurveyor

    Obsurveyor

    Joined:
    Nov 22, 2012
    Posts:
    277
    People in a corporate setting aren't always allowed to change anything outside of their My Documents folder. This includes the Unity installation folder. I personally don't like doing it because I want a "clean" install of Unity at all times. Is there any actual way to build things from bitbucket and override them on a project-by-project basis? I tried with Unity Networking once and it just broke everything until I reinstalled.
     
  50. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    The bit bucket site has a download link with a zip file that you just extract into your project's asset folder. Nothing should be going into the unity editor folder or requiring any special building. The only difference between the asset store version and the bit bucket version is the bit bucket version is a little newer and the asset store is packaged as an asset package.