Search Unity

Mist System - Highly efficient flowing mist, fog or dust

Discussion in 'Works In Progress - Archive' started by TemplarGFX, Sep 18, 2014.

  1. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Mist System
    Alpha

    What Is It?
    Mist System is a C# script and a single quad mesh object that creates a circular field around a central point of fog or mist, and then simulates the movement of the mist over the terrain according to terrain slope, wind velocity and obstacles.
    The Mist has the ability to pool into shallow/recessed areas to create a longer lasting "fog bed" on the terrains surface.
    Finally Mist System can be used to create a field of fog that follows the player, or can be used to place individual specific areas of custom fog.

    How Does It Work?
    Mist System creates a pool of "clones" to your specified size to be used to simulate the mist. Each clone is a simple quad mesh with an Additive shader and a texture attached.
    These clones are then spawned within the field and simulated according to the settings you have chosen.
    Clones are activated, live their life then deactivated back into the "free" pool for use again later.
    Simulation of the mist itself is done through simple normal, slope and height terrain data calculations and is very efficient as a result.

    How Configurable is it?
    There are two core modes for the Mist System. Full and Mobile.
    In Mobile mode, the number of simulated clones per update can be controlled (and as little as 1 per update) with all remaining clone movement mimicking the last simulations results. This allows a simpler form of Mist System to be used in your game with practically no Main Thread overhead.
    In Full mode, every active clone is simulated to its fullest in every update. While much more expensive than mobile mode, due to the efficiency of the methods used, the impact on the Main Thread is minimal.
    Once you have chosen the appropriate mode for your game, you will next be greeted with well over 100 options and value to take full control of the Mist System and its implementation.
    Terrain Flow mechanics can be tweaked completely, or disabled, along with Wind mechanics and Collider Mechanics.
    Spawn Field size, count and update rate can be configured along with clone counts, creation conditions, life conditions and death conditions.
    Mist System has being designed specifically to allow the maximum amount of configuration with the minimum amount of limits directly from the inspector
    The Script itself is broken down in many (!) methods each for a specific task. This should make it very easy to modify or expand on a specific feature without having to dive into the rather complicated simulate methods.

    What Impact Does It Have?
    Performance Impact can be scaled from practically non-existant to quite intensive depending on the set of features and the quality of each feature chosen.
    Mobile / Full mode has the largest impact on performance, followed by Shadow checks, then Collider Avoidance, then Flow Mechanics.

    Features
    Clones can be anything!
    One of the coolest features of the Mist System is that the "clones" do not have to be a single quad with a texture. They can be anything you like! Just attach the object you want spawned to the CloneMaster field and that object will be spawned in place of the default clones.

    Spawn Field
    The field in which mist is spawned and simulated is fully configurable and can be small or large, updated in realtime, or only every few frames. It can be configured to have only a small number of spawn points for fast processing, or thousands of spawn points for additional variation in the field

    Location Independent
    Mist System can be configured to follow an object (such as the player), or be placed directly into the world to create a field in a specific place on your map. Multiple Mist Systems can be active at once allowing a lot of options!

    Mist Clones
    Clones can be configured to use a random material on activation, randomize between two colors, and have a random size and rotation within specified limits. Clone lifetime, creation change, fadein/fadeout speeds, maximum movement limits, and ground offset can be configured to tailor the look of the mist.

    Mist Pooling
    Clones that pool together in locations can be set to convert themselves into a "sub clone" which is just like a normal clone, but lying flat with the ground. These sub-clones are long lasting effects that are used to fill up shallow areas with a thicker more uniform fog. The specific timing and conditions for pooling are all configurable.

    Flow Mechanics
    Mist flows over terrain and pools into shallow areas by using the current terrain normal along with changes in height and slope. Flow Speed can be configured to a specific level, or be a factor of wind movement for more dynamic flowing

    Wind Mechanics
    Mist can be effected by wind through attaching your Wind Source to the WindMaster field and enabling wind in the inspector. Any changes to the wind object will automatically be taken into consideration by the system.

    Collider Mechanics
    Mist can avoid and/or build up at colliders. If enabled, mist can be configured to be "repelled" by colliders when in motion so that mist flows around objects, and slows down when moving between objects close to each other.

    Mobile Mechanics
    A special "Mobile Mode" is included which is designed solely to reduce the amount of CPU overhead to its absolute minimum, even at the cost of accuracy. Mobile mode simulates only a small number of clones at a time, and then predicts the movement of each clone to significantly reduce processing. This simplified mode does not have as many features as Full mode, however the core effect can be created in Mobile Mode for practically no cost.



    Planned Features
    Shadow Checking
    Already implemented in a costly fashion, this system allows the adjustment of the individual clone's color and strength if it is currently in (or about to move into) a shadow. This creates a much more realistic look to the mist field and really makes in integrate with your scene. Currently the system calculated this per clone, and is very expensive, however I am working on a cached system based around the spawn field instead of individual clones which should reduce this features cost significantly.


    Videos of the System

    Flow Mechanic (Extra bright mist for visualisation)


    Clone Replacement (please ignore draw calls, unoptimized terrain in use)


    Draw Call example on optimized terrain


    Collider Mechanic Example


    Mist Color Changes


    Updates to come!!
     
    Last edited: Sep 23, 2014
    Jaqal likes this.
  2. janpec

    janpec

    Joined:
    Jul 16, 2010
    Posts:
    3,520
    Yes defenetly i would be interested as long as it has good performance. Could you elaborate what do you mean by 5-7FPS , is that actual speed that its runing or is this how much FPS is eating from your determined FPS rate?
     
  3. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Hi there. The FPS hit on my HD4200 I5 laptop was 5-7fps compared to moving around in the same world without the system activated. That is with the system at pretty much maximum quality. ~90% of clones active at any time, and every active clone simulated in every frame. These settings can be tweaked, so for example you can set it to only process 10% of the active pool in a single frame instead of 100%. This slows the fog down a little and other settings need increasing to compensate. Alternatively you can lower the maximum clones, or the amount active at any one time to reduce performance hit.

    There are no rigid body physics or raycasts used here at all so it is very quick. the system is basically simulated in 2D and then "projected" onto the terrain.

    It creates a pool of objects on start up, and then re-uses those objects to create the field of mist around the player. The performance hit past creation is very low as only simple math calculations are used to animate the mist, and no objects are destroyed or created during the run.
     
    Last edited: Sep 18, 2014
  4. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Here is another video showing off how the flow system works and its interaction with a wind source!



    the jerkiness was added by fraps running on my win8laptop.
     
    Last edited: Sep 19, 2014
  5. janpec

    janpec

    Joined:
    Jul 16, 2010
    Posts:
    3,520
    Thanks for explanation, i will add it on purchase list.
     
  6. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Today I optimized the code to reduce the number of transform.position calls in the primary loop to approximately 1/5th previous giving a huge boost to performance. The average % usage of the script's functions in the profiler went from 4.2% to 1.2%!!

    I am in the process of finishing off the sub-clone system which takes clones that have pooled together and reduces them to a single horizontal clone. This will have the dual effect of increasing performance by free'ing up clustered clones and creating a thicker longer lasting low fog in channels and craters. continuous clones pooling and converting will slowly build up this underlying fog. The sub-clones themselves can be further effected independently by the wind and flow system!

    Finally Im also adding in the ability for the clones to have randomly selected materials from a set pool of cached materials to give the mist much wider diversity in its appearance!
     
  7. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,684
    This is very nice! But is there any way to minimize the horizontal lines as the effect impacts the terrain? That's a significant detractor from the effect.
     
  8. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Hello hopeful!

    If you watch the first video you can see that the edge of the clone where it intersects with the terrain is almost impossible to see. This is how the system should be used. The Second video I have purposefully pushed the visibility of the clones to a very high level to make it easy to see their individual movement. As it uses addative blending to add the effect into the world, you wouldnt have it so high!

    Texture choice and configuring the ground offset and rotation speeds to complement the chosen texture would also reduce this intersection if you require a stronger effect.



    Update :

    Last night I came up with, and implemented a very efficient way to have the individual clones avoid object colliders when moving! This means that the mist will slow down/build up around trees, be stopped by rocks/buildings etc. and best of all. FLOW around the player!
     
    hopeful likes this.
  9. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,684
    Well ... there are some visible lines in the first video if you watch at 720 in full screen mode. For instance, around the 3:50 min mark, on both left and right.

    I'd be curious to see what this looks like going through blades of grass, bushes, and trees.

    There are limitations we all have to work with, and it's a nice effect. Whatever can be done to minimize those lines is good, though. :)
     
  10. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Unfortunately I think the lines are always going to be present to some degree, at the end of the day, its still a quad intersecting with the world geometry :p
    However I havent even looked into shader creation for unity yet. When I purchase Shader Forge I am sure I will be able to create a shader to assist with this drawback!

    It is highly configurable however. I use it intersecting the terrain (ground_offset at -2.3) for a very low whispy mist look. You can put the ground offset to positive so they do not intersect with the terrain, play with rotational amounts and create a texture to suit. The avoid colliders option would keep them out of most objects.

    You could also replace the clone with a more complex custom textured object if thats your thing :p


    Here is a very quick video using a sphere with a simple diffuse shader on it.

     
    Last edited: Sep 20, 2014
  11. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Here is another video with better optimized terrain draw calls showing a thicker fog setup.

     
  12. Jaqal

    Jaqal

    Joined:
    Jul 3, 2014
    Posts:
    288
    Very cool. Something I've been wanting for awhile now. I am certainly interested in this!
     
  13. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103

    Thanks for the Feedback!


    I spent the weekend feverishly coming up with methods to update the clones for mobile or very low performance situations. After 2 failed attempts, the 3rd system works quite well! While no where near as accurate as the real-time simulation, this "mobile" mode simulates only 1 (or a set number) clone every 10 or so seconds. During this time the clone just heads in the simulated direction at the simulated speed without changing. It actually works fairly well, but does have drawbacks. Fast moving mist tends to fly over small crevices instead of pooling into them due the time between simulation updates. Still its a great option for those that want every bit of performance they can get!


    Here is video showcasing the intended use of the collider avoidance system. The Collider effect's power is quite low here, only having a small impact on the overall movement of any one clone. Each tree has a collider for its trunk, and the result is the fast moving wind-blown mist slows down and pools under trees. When enabled, mist will never enter a collider, and if close by be pushed away (but only when they are in motion)




    The SUB-Clone system should be finished today which will be the final piece before last-minute performance tweaking.

    I am also working on a webdemo so you can see it in action yourself.
     
    hopeful likes this.
  14. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    While I wait for my work day to end I thought I'd do a quick video showing color changes. Gotta love that toxic fog!

     
  15. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Just a small update today.

    The Sub-Clone pooling system was implemented last night and its awesome! I don't have a video yet as there are a few issues that need ironing out, but the potential for this feature is impressive!



    Flow mechanics are not yet applied to the sub-clones, which is why they are sitting a little higher than I would like, but as you can see, once they are placed properly in the crevices of the terrain it will be a really nice effect.

    Sub-Clones have almost no calculations done to them past creation, so they are extremely cheap.


    Here you can see the fog is rolling down the hill and pooling into this area infront of the camera
     
    Last edited: Sep 23, 2014
  16. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Here is a quick video showing the pooling system in action!



    The sub-clones don't yet have any textures, so they are using standard clone textures which are a little too think around the central area causing the easily visible intersection lines in the above video.

    Almost there! Still needs a bit of tweaking on how it forms and the transition itself but not only does it create a really nice effect in game, but it increase performance by removing pooled up clones!


    While obviously set to an over-the-top brightness, here you can the flow simulated sub clones in full action!



    And from an FPS perspective (with a blueish color)

     
    Last edited: Sep 23, 2014
  17. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    I am looking for one or two people to try this out in their project to give me some feedback on the performance of the system in its current state. This will really help me to optimize the system before I actually release it as at the moment, I have only run it on my laptop!

    Send me a PM if you are interested!
     
  18. janpec

    janpec

    Joined:
    Jul 16, 2010
    Posts:
    3,520
    Nice, so if i understand correctly this sub-clones are very light on performance? I would love to test it but as i dont have yet any level design built it wouldnt make sense. I am sure you will find someone though.
     
  19. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Hi Janpec! You are correct. Each sub-clone does a little bit of calculation when its activated, then it just sort of "glides" over the terrain for the rest of its life. This means its only impact is really just the draw call (or calls depending on material) for the sub-clone! The number of sub-clones, how many clones they use up to do the conversion and their life time are all configurable to ensure their impact is as low as possible
     
  20. Teila

    Teila

    Joined:
    Jan 13, 2013
    Posts:
    6,932
    Wow, this is beautiful. I must have it. :) Thanks!
     
  21. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Thanks for the support Teila!


    Here is another video showing the system Disabled, in Mobile, Full (No advanced features) and Full (all features) to give you an idea of the performance impact.

    Mobile Mode - 1000 Clones, 100 per update, 10 second life
    Full Mode (Basic) - 1000 Clones, Flow, Wind, Effected by Sun, 10 second life
    Full Mode (All) - 1000 Clones, Flow, Wind, Collider Avoidance, Clone Avoidance, Sub Clone Conversion, Effected by Sun, Effected by Shadow, 10 Second Life (30 seconds life for Sub Clones)



    The Alpha test package has been built and uploaded and I am looking for 1 or two more people to give the system a test. Please note this is not a test of a "finished" system. It is still very much WIP!
     
  22. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    A bit of a "show off" video this time. A nice low shot of an area with fog building up.

     
  23. janpec

    janpec

    Joined:
    Jul 16, 2010
    Posts:
    3,520
    I am thinking this could work with deserts, using it as desert sand movement too. Performance impact is quite low wihch is good. The only disadvantage of this system is that where cloud planes go trough terrain, it is seen trough sharp plane edge, have you been thinking maybe adding some sort of falloff to it when mist plane touches terrain it would falloff with alpha texture or somethng which would decrease those sharp edges into terrain and objects.
     
    hopeful likes this.
  24. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103

    Hi There!

    I have several ideas which I am going to investigate on how to get around this issue, because it is quite noticeable if you want fog or anything thicker while the clones are underground. Moving them so they are just above the terrain obviously removes the problem all together, but you really have no choice but thick fog in this setup (awesome on fast wind settings! sandstorm?).

    I have yet to try creating a texture which is much like a grass texture, with the smoke at the bottom. Then when sitting on the terrain it will still be low fog and the intersection would be reduced alot (I might try this now as its quick).
    Another option would be an overlay or decal applied to the texture or for pro, using a render texture.
    Shaders, which I haven't looked into yet (and plan to) could lead to an efficient and robust solution also.

    Really, Mist System is a custom particle system that is solely designed to create a terrain based fog effect. It gives you billboard objects simulated with wind and gravity and a bunch of conditions, and you choose the material and texture you want the clones, or "particles" to use. You can use a transparent diffuse shader and turn shadow casting/receiving on if you want, and you can get some cool evil-fog effects going with multiply shaders!
     
    sashahush likes this.
  25. Jaqal

    Jaqal

    Joined:
    Jul 3, 2014
    Posts:
    288
    If you are still looking for someone to test I am interested in using this on my current project.
     
  26. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    Hi there Jaqal!

    Yes I am. I wanted 5 people to test it in varying situations and I have some room left :p

    The quietness in here of late has been from me re-writing the entire simulation classes (mobile and full) using the feedback I have had so far.

    Significant improvements have been in performance compared to the version above. I did not know about the "Deep Profile" option until just recently and it helped me cut the simulate ms time in half! Once I break it up further and optimize each function further improvements should come!


    I also came up with an interesting idea which I am going to test this weekend. At the moment, each clone faces the camera object directly and constantly to ensure you are seeing the texture face on (a-la billboards). In close proximity to the camera, the effect of them rotating can be quite noticeable if they are intersecting the ground.
    I am going to add a "buffer" if you will that will allow you to set the number of degrees off the clone can be from your facing. This should not only significantly reduce the visible rotation of the clones, but should increase the variation of the fog due to the now varying viewing angles on each clone!
     
    hopeful likes this.
  27. Jaqal

    Jaqal

    Joined:
    Jul 3, 2014
    Posts:
    288
    Sounds really nice. I've been wanting this since I started my project! I have a swamp and desert area I am working on that this would fit perfectly.
     
  28. TemplarGFX

    TemplarGFX

    Joined:
    Aug 21, 2014
    Posts:
    103
    It certainly would! I am finding that I am liking it more and more for visualizing the wind in my environment. If you have it on a really low strength and just a small number of clones its barely noticable, but enough that you can look around and determine both direction and strength of the wind visually.

    I think the best setup will end up being a single Mist System following the player to do this wind vis. And then stationary Mist Systems placed in your environment to do localised effects like sand or bog fog.

    Updates coming today!
     
    janpec, Hikiko66 and Jaqal like this.
  29. casimps1

    casimps1

    Joined:
    Jul 28, 2012
    Posts:
    254
    Very nice looking effect. I can see this making for some great moody atmosphere.
     
  30. janpec

    janpec

    Joined:
    Jul 16, 2010
    Posts:
    3,520
    Whole system is very solid and good, i hope though you find solution for blending clouds with terrain becouse rough edges is the only thing that is downgrading it a bit.
     
    Hikiko66 and hopeful like this.
  31. Soul-Challenger

    Soul-Challenger

    Joined:
    Dec 30, 2010
    Posts:
    152
    This is looking very nice!
     
  32. rcalt2vt

    rcalt2vt

    Joined:
    Jun 6, 2009
    Posts:
    36
    This looks awesome man, flagging this to watch the progress. I'd be willing to help test this as well if you still need testers.
     
  33. Rico21745

    Rico21745

    Joined:
    Apr 25, 2012
    Posts:
    409
    Wow, found this by accident. Definitely very interested as I have a way of doing fog atm, but it's very much faked in a nice way. Depending on performance and complexity, I'd be interested in it for my project.

    Let me know if you need additional testers as well.
     
  34. Jaqal

    Jaqal

    Joined:
    Jul 3, 2014
    Posts:
    288
    You still plan on releasing this? I'd love to test it out!
     
  35. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    240
    Hello,
    is the project alive? Will it be released?
     
  36. mons00n

    mons00n

    Joined:
    Sep 18, 2013
    Posts:
    304
    I'm up for testing on OSX!
     
  37. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    240
    Great! Looking forward.