Temporal Anti-Aliasing

I tried the GitHub version and it's the same. I can't find the temporal AA image effect. Just a single AA component that gives me the option between FXAA and SMAA.

 

Go to the bitbucket page, click on downloads, click on branches, click on the zip next to "taa"
https://bitbucket.org/Unity-Technologies/cinematic-image-effects/downloads?tab=branches
https://bitbucket.org/Unity-Technologies/cinematic-image-effects/get/taa.zip

Downloading branches is a little confusing since going to a branch then clicking on the download button takes you back to the non-branch download.

 

1. Bitbucket
2. https://bitbucket.org/Unity-Technologies/cinematic-image-effects/get/taa.zip
   
3. TemporalAntiAliasing.cs

 

https://github.com/gokselgoktas/tem...mmit/f76748e1417dec683d1dd07f2d6fd6ead82e202d

 

Damn, that's awesome! Can't wait. Our experience so far has been excellent.

 

Mad props.

 

Does the effect have a sharpening setting? Manually added sharpening with photoshop in this example.

 

AFAIK they were working on adding it.

 

There's the strangest slight indentation going vertically down the dead center of the screen - known issue?

 

Hi guys, just to be sure I understand correctly: is temporal antialiasing available for 5.4? I mean it doesn't require 5.5 beta, right?

 

(Thank you Tim, I've replied to your message with a question )

 

Hi there,

I only found out about this recently, and I'm very happy that Unity is implementing TAA

I noticed a couple of issues, though:

1. The first one is the one that hippocoder has already mentioned: a vertical artifact along the middle of the screen. Although, after some investigating it looks like it might be a general Unity issue (maybe a bug in 5.4.1), because I tried rendering without TAA, and the issue seems to still be there. Please see the image below - it's really not apparent, but in some situations is looks quite ugly. And after zooming in, you can see that all pixels along the vertical middle line are repeated. Has this already been reported? Should I submit a general (non-TAA related) bug?


2. All objects that are rendered in the transparent queue seem to be constantly shaking (I'm using the 2016-09-21 TAA version from the Unity repository). Am I missing something here? Can that be prevented? Of course, I can remove the [ImageEffectOpaque] flag, but then all of those objects become blurry, which I don't need

 

Thanks for the reply Would you guys consider implementing the option to selectively choose, which objects to shake/jitter in the scene (instead of shaking everything)? It would make a lot of sense for 2.5D games, where environments are 2D, but characters and some of the other dynamic objects are 3D and need anti-aliasing.

 

On the latest TAA version (Sep 21th) I'm having full screen pixelated trails when moving/rotating the camera.

 

Hi guys!
Thanks for the amazing work! I've tried a bit the latest TAA versions and it's great (there's only a bit of flickering in some parts of the model, I'll investigate..).

I'd like to use that in my WebGL project but it simply doesn't works! I can make the build (without error) and then in the browser I get a shader error and a black screen. Deleting the "#pragma exclude_renderers gles flash" in the shader, I can see my model but it's not really working...
Do you think you will support WebGL? Do you know when?
I founded a TAA asset on the assetstore weeks ago but now it's no more available for purchase!

 

Did anyone see this yet?

Siggraph 2016 Advances in Real-time Rendering in Games
http://advances.realtimerendering.com/s2016/index.html

Featuring a presentation about TAA in Uncharted 4.

But also Aggregate G-Buffer Anti-Aliasing (AGAA):

 

This is good stuff, we are currently using it in a 2.5D game we're making and pretty happy with the results (as long as you avoid the common pitfalls).

I ported an anti-aliasing comparison tool I made ~3 years ago to Unity and included TAA as one of the options in the comparison:
http://blog.metaclassofnil.com/?p=851
If it's of any interest for testing I can also upload the project (it's rather tiny).

 

Based on this, Unity's TAA is not far off from being almost as bad as UE4's. (Every iteration just serves to make some facet of it look worse without actually improving the underlying big issues.) Not even the 8xMSAA+TAA mode looks good. It does a very poor job at resolving what's supposed to be there and in motion just as bad. A slow camera pan does not at all really show off what it would look like in actual gameplay nor put it in a stress test scenario with fast motion. Where it probably , like most TAA, falls apart.


Really, most Temporal AA is totally mediocre and introduces unacceptable artifacts or fail to tackle a lot of actual temporal aliasing, or even static aliasing.
Often resulting to using an absurdly wide resolve that doesn't at all resemble any kind of ground truth to try and tackle these issues the correct way. And then tacking a sharpening filter on top to try and regain face. In the case of Doom 2016 or UE4's TAA, the sharpening method introduces even more artifacts on top statically and in motion. Instead of trying to gain back information lost in resolve it just distorts it into a smudgy mess (This GOW4 screenshot shows the issue readily http://images.nvidia.com/geforce-co...active-comparison-19x10-001-ultra-vs-low.html any and all actual detail has been scrubbed into almost looking like individual paint daubs making up the scene. Doom 2016' TSSAA has a similar problem. But not nearly as bad looking, while still resolving the majority of detail very well ). Then also emphasizes the remaining aliasing that exists.

I don't understand why anyone hasn't pushed to make a more costly and vastly higher quality TAA solution that fixes all of it's issues for future proofing. (Or create a more hybrid approach using actual super sampling in a smart way combined with it instead of just temporal super sampling alone) Then again, this is the industry that, 8/10 times can't even implement any decent SSAA solutions. All lazy, inefficient, poorly resolved OGSSAA methods. Faced with how bad those often look, I can see why more turn away from attempting to make good SSAA.
Everyone's so focused on getting things running as fast as they can get per frame that quality is sacrificed too much.

Heck, if you happen to be using Forward+ (Which would make more sense in VR.)AMD already has a great solution made for you.
https://github.com/GPUOpen-LibrariesAndSDKs/SSAA11

When there's TAA that doesn't introduce awful looking artifacts and can come close to SGSSAA or AMD's above SFAA demo statically, and ESPECIALLY in motion. I'll be jumping for joy. But I doubt i'll see that any day soon.
http://www.mediafire.com/file/8o36djz5fe7yj55/FFXIV+AA.7z (Compares FXAA vs 4xSGSSAA in FFIXV, a game with an insane amount of aliasing. 4xSGSSAA properly resolves 90% of the information and does so almost completely temporally stable without introducing artifacts of any kind)


SMAAT2x is pretty decent when properly implemented. (Ethan Carter UE4 is the best example. It exhibits essentially no added artifacts at all, there is no ghosting, no smearing. When combined with DSR instead of the linear filter used in the in game resolution scale. It essentially looks perfect. Dead rising 3 is probably one of the worse examples. Totally poor implementation. Introduces artifacts, image wobbling, resolves even less information than FXAA does. Homefront The Revolution's SMAAT2x also seemed very poor, broken actually when I tested it in on a free weekend. The alternative TAA option on the other hand was very good without a ton of artifacts or an overly wide resolve or absurd amount of sharpening that added no artifacts. Sharpening was user adjustable as well)

I'd rather everyone in the industry just use that instead of trying to do more for less (Again trying to gain better quality for little cost or combining it with poor to mediocre quality MSAA. Which is pointless, if you are going to be using MSAA you might as well be using or offering the option to use SGSSAA or AMD's SFAA from the above link as well. ) and just making thing's look worse instead with more and more artifacts. As most TAA is so terrible, i'd rather just use basic PPAA instead or use that + decent quality OGSSAA (DSR/GeDoSaTo).
SMAAT2x is probably the most well rounded, artifact free TAA out there on a consistent basis.
You should aim to go in it's footsteps instead of the likes of UE4.

Necessities for decent TAA
1.) Resolve filter width can not be so absurdly wide to make motion more stable in order to make up for the failings of the rest of your technique. Improve the core issues here not try to compensate for them. If anything else offer a much higher quality alternative that costs more performance.
In addition to the options to combine it with SSAA in proper 1.x X 1.x aspect ratio correct ratios(IE: 2.4x2.4=5.76x or 1.7x1.7=2.89x with the resulting render resolution maintaining the exact to the number DAR as output resolution. In addition to not needing any additional rounding to get a whole number resolution. Lot of resolution scales in games and everything but 2.25x/4.00x DSR use dirty math that potentially mess something up)but not lazy standard OGSSAA with a linear resolve as used in 8/10 games. Get creative to make it higher quality and potentially more efficient.

It will be appreciated in the future.

Actually try to solve the aliasing and get closer to the ground truth.
2.)There should be zero ghosting of any kind. Do a fast pan test with a mouse. Take a bunch of screenshots as you do so. Then pan the camera even faster and do the same. If there's any kind of visible ghosting. You are doing it wrong. (Ethan Carter UE4 SMAAT2x, Homefront The Revolution ;PC ver; key examples here. Neither have any ghosting.)
One where UE4 frequently fails, is around the silhouette of any visible first person models.
3.)Flickering should be minimized if not zero.
4.)If you are adding sharpening, make it user adjustable to the point it can be added to any and every game without issue. Add multiple techniques. Ideally sharpening shouldn't really be needed.|
Also, if when sharpening it's bringing aliasing back into the mix visibly that wasn't solved somewhere but was hidden by wide resolve softness. Fix that. (UE4,Mafia III specific examples of this)

Nvidia's AGAA seems a really promising alternative that at least in stills looks good.Resolving information without introducing unnecessary artifacts. I'd really like to try a demo of that in something as much raw aliasing as UE4 has. (Seriously, it's insane how much aliasing there is in UE4 without any AA. I don't think any game or engine has ever had as much aliasing issues as this)
http://advances.realtimerendering.com/s2016/AGAA_UE4_SG2016_6.pptx

 

UE4 is aliased without anti-aliasing in large part because of TAA, or more specifically the expectation that TAA will be used. Physically based shading tends to lead to a lot of aliasing from geometry edges and normal maps which many games try to solve with a handful of texture pre-processing techniques, usually referred to under the catch-all phrase "Specular Anti-Aliasing", like Toksvig, CLEAN, or more recently Frequency Domain Normal Map Filtering. Valve also uses some specular anti-aliasing techniques that take into account the model's geometry.

UE4 didn't go with any of these techniques, but rather relies completely on TAA. UE4 actually intentionally makes specular aliasing worse by decreasing the texture mip bias which results in the game sampling textures at a higher resolution than it should for the screen resolution it's being rendered at. But oddly this is what they should be doing as without that TAA would make things even blurrier.

There are also a ton of games out today or coming out soon that without TAA would look like noisy aliased messes to the point of being completely unplayable. Playdead's Inside and Media Molecule's upcoming Dreams are good examples of games that without temporal super sampling / anti-aliasing would not look anywhere near as good as they do, or might not exist at all.

However I totally agree that TAA adds a lot of artifacts that don't work well for all games. MSAA or CSAA/EQAA and specular anti-aliasing techniques can usually get much cleaner results, but those also don't work well or at all with deferred rendering.

SGSSAA, or Sparse Grid Super Sampling Anti-Aliasing, curiously shares a lot in common with TAA. You could think of SGSSAA as being the same thing as TAA without the temporal aspect. Both take several frames with the camera slightly shifted and combine them into a final image, the only difference is SGSSAA renders all of those additional frames all at the same time instead of collecting them over time, so 4xSGSSAA is rendering the entire game view 4 times for every time it's displayed. SGSSAA is also an attempt at getting closer to to the ground truth techniques that movies use (which is usually some form of Monte Carlo "Random Grid Super Sampling Anti-Aliasing" with adaptive variable sample counts). Performance wise rendering using 4xSGSSAA on a 1080p display is the same performance cost as rendering at 4k and downsampling the image to 1080p, or the equivalent of 2x super sampling (aka 4x Ordered Grid Super Sampling Anti-Aliasing). The results from SGSSAA are great, but it's not very "smart" as everything gets anti-aliased even if it didn't need to be. SGSSAA is also generally only used with forward rendering afaik.

Interesting the same causes of the aliasing that you see in UE4 without AA should be happening when using SGSSAA for the same reasons.

AGAA is great but can be more memory intensive than SGSSAA or OGSSAA as it renders the deferred GBuffers at high resolution, then down samples the gbuffers intelligently so the lighting calculations are done at lower resolution than the original GBuffers, but still higher resolution than the final screen resolution. This makes a lot of sense for physically based rendering where the cost of calculating the lighting is significant, and it gets decently close to SGSSAA. The "2xAGAA" or "4xAGAA" is hiding the fact that it's rendering the GBuffers at significantly higher resolution and the number in front of the "AGAA", and 2xAGAA is rendering the gbuffers at 4xSSAA or 8xSSAA and 4xAGAA is for sure at least 8xSSAA.

SMAAT2x is pretty awesome, especially compared to FXAA, but it actually adds a ton of noise to the edges of things. This works pretty well at higher pixel densities as it mostly reads as dithering, but it can be pretty ugly for something like VR.

I guess the TLDR is anti-aliasing is much harder now than it was when shaders and lighting models were relatively simple. Super sampling was the first anti-aliasing ever used and every technique since has been trying to figure out how to make that cheaper / better. We're kind of stretching the limits of what can be done with rasterization techniques alone.

 

I don't have the link handy, but CoD's recent presentation on how they did their AA was seriously impressive. Was basically "Cinematic" temporal SMAA, they did some wizardry to avoid ghosting on FP, minimized blurring, and it was super cheap because it had to be.
Also, pretty sure CoD's TAA doesn't do this when you move the camera around fast (using default static/motion blend settings):

 

As of now, the current implementation of Unity TAA is only good for static images. As soon as color changes or camera moves there's a huge amount of noisy artifacts inserted in the final image, pretty much useless.
Plus the fact that it currently don't work with transparent images it's kinda useless 2x.

 

Another form of temporal anti-aliasing. This is still fairly blurry but the ghosting reduction is very nice, http://j4lley.com/content/publications/pdf/pg2016-phlm.pdf

 

that looks quite promising!

 

I was actually just looking at this solution:
https://github.com/playdeadgames/temporal

It looks pretty nice, but the image does look a bit blurry. Sampling at a higher mip level like bgolus notes should solve that.

Edit:
I just did some tests with sampling at a higher mip level. Unfortunately it leads to an unstable image with the TAA implementation above. The exponential sample distribution makes the weight for the latest sample higher than the others, so the aliasing in the spatial sampling is not fully balanced by the temporal AA.

I used a 16 tap jitter for the TAA, so my initial texture bias was -2. But even -1 and -0.5 are unstable on textures with high contrast. On others it's fine, so it could be fixed with a per material setting. (Or just overriding the mipmaps.)

I've attached some test screenshots. (Sorry for the jpg artifacts, but png was too large.) Left column is without TAA and right column is with. Top to bottom are mipmap bias 0, -1 and -2. (Where -2 should be the proper value for a 16 tap jitter, so top left and bottom right should be the most "correct".)

Has anyone compared the play dead games solution above to the Unity TAA implementation?

 

There are some references to playdead's version in Unity TAA's source code. In theory the latter should improve upon the original, especially since it can benefit from new features in the Unity Engine such as motion vectors.

 

Hi all, e try Unity's TAA and we got good results in editor no blur effect or some kind of motion blur, but after build we got lots of motion blur effect on screen, can somebody tell me if its known issue or me setting are wrong, we wont use TAA for elimination specular jitters on geometry edges and in editor is everything fine but in build it's we have horrible motion blur. Thanks for answer or advice.

 

that's mean you have framerate issues in the final build your framerate should be high and stable

 

@blackbird
How high? Because I have same results with only one room, with all post process turn off as in like whole level with all post process turn on, And doesn't matter if I have 60 or 150 fps results are same, that why I ask people here, because if this is what should be doing Unity TAA, that way this plugin is useless.

 

Hey Tim-C, just wanted to ask you a question...hope you get to read this

I was wondering if the current implementation of TAA also had supersampling in it - so TSSAA, and if not, whether or not it is a planned future.

I ask this, because after trying the current TAA from bitbucket and the latest CTAA from lavenda (which according to their claim is supersampled TAA). I've come to an objective conclusion that CTAA, as of now, is better than TAA (image quality wise, haven't had the info and specs to compare performance). I've come to this conclusion after trying the CTAA demo (standalone demo downloaded from Lavenda) and applying and trying TAA to my own scene. More specifically, TAA had noticeable horizontal specular aliasing popping up between frames and also had mid-ranged objects aliasing - up close, both TAA and CTAA rocked, but as objects gained distance from the cam, TAA fell short.

I understand that TAA is still a WIP and honestly think it is impressive but just wanted to know if SS will be implemented - or maybe it already is and i have despicable eye balls

 

Hey,

We updated to the latest TAA recently, and have noticed a lot more artifacts/ghosting/blurring compared to previous versions. At some point I'll test all of the revisions we skipped to figure out where it started. The biggest issue is when the camera is moving backwards, which creates quite a lot of stretching on the bottom of the screen.

Thanks

 

Hey,

It's the latest from the Cinematic Image Effects repo. We are avoiding using the post stack for now for several reasons.

It also seems that performance has degraded on xbox one. It's now 2.16ms which is pretty significant.

 

Yeah the latest TAA is considerably blurrier than the one I had downloaded before.

 

I've noticed that trees that have gone past their fade distance are jittery, and very shaky when using TAA

 

TAA doesn't currently work with transparent materials. The trees in the backgrounds that renders as impostor falls back to some semi-transparent default material which TAA make them shake in the jitter buffer.
[EDIT] Looks like they do the jitter after the transparent pass and AA before transparent pass... Looks incorrectly bad yeah.

 

What about unity 5.6? There some big problem with TAA - Image the sliding. How you think -it's possible to fix?

 

I also have a real struggle with aliasing and embed TAA with 5.6.

 

4K Screenshot from 5.6 with TAA enabled. I've a lot of ghosting? not sure it's the right term?



More picture there: https://forum.unity3d.com/threads/4k-screens-unreal-engine-infiltrator-in-unity.447068/#post-2893350

 

When I incorporated the playdead TAA, I added a simple tonemapping operator before the blend weights are determined. This might also help here. Else, when in HDR mode, the blending is done on raw lighting information, which might not match the final image much. (Specially in dark and light regions, which seems the case here. Blending too much in the dark regions and not enough in the light regions.)

I also had some additional issues when adding another post process step. This was related to the fact that from a CommandBuffer, you identify temporary RenderTextures with an integer ID. So you have to somehow make sure that the same ID's are not used elsewhere. I do wonder why this ID approach is taken over a simple pointer to the actual RenderTexture. The latter seems a lot safer to me. Outside the CommandBuffer you release temporary RenderTextures by supplying a pointer to the RenderTexture. Why was the choice made to make this an integer ID in CommandBuffer?

Edit: Here's my TAA result. (Note it is scaled down from 3840x2160 and sharpened afterwards.) Only between the flower and the sky on the right there is a significant TAA bleed visible. Without a simple tonemap before the TAA blending, there would also be issues around darker parts in the scene.

 

Not enough battle testing on actual projects leads to limited and quite often poor design. I have the same issues with other CommandBuffers due to some of them sharing the same generated integer IDs.

 

I'm using TAA from the post processing stack (from github, downloaded latest commit yesterday) and I hope someone can clear up something for me.

I'm pretty sure I'm doing something compeltely wrong somehow, because judging from the images in this thread TAA should really have excellent quallity, unfortunately it looks absolutely hideous in my demo scene...

In the video you can see that either I have to reduce the "spread" so much that there's nothing gets antialiased (not even static and nonmoving geometry), or that all transparent stuff gets smeared all over the screen.

The smearing is most visible in the world space canvas, but the grass is affected even more.
I didn't dare to actually move the camera but I will add a screenshot so you guys can see whats going on.

Video (50mb): https://mega.nz/#!gBBRzQqA!5DfJXSD8fJam9qaCBHdu0IXiojGca4KlYMhXgewqTaE


No AA: http://i.imgur.com/Q09xFW9.png
SMAA: http://i.imgur.com/nu0FO0S.png
TAA: http://i.imgur.com/Xrrtfnq.png

Scene description: Camera was moving and rotating at moderate speeds, nothing too slow or too fast that could possibly explain the TAA result in my opinion.

The SMAA is from unitys "old" bitbucket cinematic effects pack, the TAA from the github postprocessing stack.

From the way the TAA image looks I very much doubt that it will be able to reach SMAAs superiour quality anytime soon.

The scenes in my screenshots and the video make no use of motionblur.

 

I used the playdead source, but that did not work 100% out of the box. Unity used the same source I believe, so the issues might be the same. I noticed that movement towards an object that was marked to have motion, led to a strong horizontal movement recorded in the motion buffer. If you enable motion blur, this becomes extra visible. I don't remember exactly what the fix was, but some adjustments were needed.

So, without the velocity buffer tag the motion is correct, but with that tag, it differs in the original implementation.