Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[WIP] VRTFX - Volumetric Rendering, Titling & Effects

Discussion in 'Works In Progress - Archive' started by sschaem, Nov 7, 2016.

  1. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here is a preview of an experimental 3D Titling system for Unity.

    Its still at an early stage, but the core engine seem to be working well.

    I plan to update this thread with progress and examples.

    Here is a (growing) list of features:

    - Multiple controllable Chamfer type : Round, Inner round, Flat, Steps
    - A few Boolean operators for solid geometry modifications (to create shell, inset, etc..)
    - UV generator (and material property) for the various defined parts of the solid geometry
    - Built in real-time local ambient occlusion (can be used with deformed/morphing shapes)
    - Run-time TTF parser with analytic quadratic curve evaluation
    (The API could be made to accept generic vector shapes)
    - Seamless LOD (this come by design)

    All properties are keyframeable for run-time animation.

    I will update/refine this top post with progress and include answers to questions you guys might have.

    If you are still reading, you might be interested in a technical overview.
    a) The system generate signed distance field volumetric data from vector shapes at runtime
    b) The data is processed/manipulated via sphere tracing (aka raymarching)

    So no classic geometry exist, only 3d bounding volumes.
    This makes the generation of models dynamic, where surface modeling is evaluated per pixel.

    vrtfx1.png
     
    Last edited: Nov 15, 2016
    Marco-Sperling and Nova-Shade like this.
  2. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Last edited: Nov 14, 2016
  3. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here is a modeling progression (keyframeable shader properties) using some Font Awesome symbols

    1) Extruded flat
    2) Colorized
    3) Boolean operator applied
    4) Local ambient occlusion

    fontcolor.jpg fontcolor2.jpg
     
    Last edited: Nov 12, 2016
  4. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    zyzyx likes this.
  5. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Closeup of the boolean operators with the font https://www.fontsquirrel.com/fonts/vitamin
    You can also get a sneak peek at the geometry for the letter 'T'...

    Some animation (morphing) should follow... stay tuned

    Here, the operator is used to hollow the extruded stencil.ttf font,
    it is then filled with a transparent volume.
    (This scene total 3 draw call and 48 vertices : solid, transparent, shadow)
    filled.jpg

    vrtfx4.jpg

    vrtfx2h.jpg
     
    Last edited: Nov 11, 2016
  6. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here is an example of unicode support with arialuni.ttf

    uni1a.jpg
    uni1b.jpg
    uni5.jpg
     
  7. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Rigid body (and video capturing) test using a box collider per glyph.

    If you peek full screen at the stats you can see the scene max out at 9.5K vertices.
    (most, about 4K, used by the Unity skybox)



    mess2.jpg

    mess1.jpg

    Stress test with 32K vertices


    Closeup of a glyph
    closeup1.jpg
     
    Last edited: Nov 15, 2016
  8. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    A simple example of the dynamic system used for players names.
    (TTF font used : armalite-rifle.ttf "player1", impact.ttf "versus", arialuni.ttf "player2/subtitle")

    Since all is done at run-time, things like localization, player names, etc.. is possible.

    titlea.jpg

    titleb.jpg

    This all look great in VR as the LOD system is seamless.
    titlec.jpg
     
    Last edited: Nov 13, 2016
    manpower13 likes this.
  9. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here is a function based deformation test, where a 3D sin wave is applied to the distance data.



    The deformation can be subtle
    snow.jpg

    And also applied to a translucent volume
    loading.jpg
     
  10. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Morphing proof of concept. The target here is a sphere, but it can be 100% arbitrary.
    The sample I ultimately want to make is the word "Welcome" morphing into multiple languages,
    one to another to another.

     
    zyzyx likes this.
  11. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    love it
     
  12. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Arial.ttf with some displacement

    scale1.jpg
    scale2.jpg
    scale3.jpg
    scale5.jpg
    scale6.jpg
     
    Last edited: Nov 16, 2016
    yc960 and Stephan-B like this.
  13. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    how is the performance?
     
  14. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Performance really depend on the feature set enabled, and its 99% related to on screen pixel coverage.
    I will double check, but I think the last scene (spiked metal) run at 200+fps on a r9 290x (very heavy)
     
    yc960 likes this.
  15. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    Is there a beta/release date?
    Is it limited to fonts?
    Let's say I want a map to appear volumetric, would it work?
    Would it be easy to include custom shaders/maps into yours? (I assume this is a shader based asset)
    Would it work with unity's particle system? Vertex Color and all?
    Would it work with unity's UI system? Alpha Mask and all?
    How easy is it to add custom time base animations, like a wave like effect?
     
  16. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    No date, I'm still trying to see if there is interest in this.
    But I would estimate the work involved to be about 2 month for a solid beta.

    The code currently only support TrueType fonts, but any raw vector data could be supported
    (as long as the data is built around quadratic bezier splines)

    For customizing the shaders you would need to edit the shader files to get tweaks done.
    Maybe when Unity integrate something like shader forge, I can design modules, making it trivial to customize.

    I haven't looked at the particle system and how both would interact.

    Currently I dont plan to do anything specific with the Unity UI.

    I should have some sample script that provide basic text animation next,
    it should be no different then other scripts that manipulate objects.
     
    yc960 likes this.
  17. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    I would love to help you test this if you need the help.
    Let's say I converted a graymap to vector data, would that work with the shader? How would it look?
     
  18. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    The vector data is transformed to volumetric data. This is what define the implicit surface of the 3D model.
    This data can then be extruded, modeled, textured and animated in the shader.

    In this scene the sphere is the default Unity sphere, the other 3 are VRTFX using http://fontawesome.io/cheatsheet/
    objects.jpg

    Modeled & Textured (100% on the GPU, all parametric via shader properties)
    objects2b.jpg

    For beta testers, first let see if its even a viable thing to work on :)
     
    Last edited: Nov 17, 2016
    yc960 likes this.
  19. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    I see great potential in this. If the gpu performance is similar to mesh models, I think this will be a must have for projects. What is the likely compatibility? I assume Desktop is a given, console? Mobile? VR? Webgl? What is the required shader model?
     
  20. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    It goes all the way down to dx9 shader model 2.0, but some quality options are not available.
    Now even so it works with shader model 2.0, loops get unrolled, so compile time takes a hit.

    I dont think this modeling technique will replace traditional optimized mesh for performance.
    I'm using it because its very flexible and offer some neat features for real-time animation.

    Update : target 2.0 now fails. So the minimum is 2.5 (shader model 3.0)
    Here is a quick test : forward, Dx9, shader target 2.5
    s25.jpg
     
    Last edited: Nov 18, 2016
    yc960 likes this.
  21. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here is an in-game example of run-time localization.

    The red decorative letter is using the rothenbg.ttf font
    The wood/metal sign is from CloisterBlack.ttf font

    scene6.jpg

    scene4.jpg


    Performance overhead.
    (CPU not shown because its identical, geometry added to the scene is 64 vertices total, and no script execute)
    perf.jpg
     
    Last edited: Nov 23, 2016
    yc960 likes this.
  22. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    Love it, indeed, runtime localization! I did not think of that! Any other magic use for this wonder asset?
     
  23. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Thank you.

    Beside localized title / load screens, score page, pref / settings etc..
    VR is a good candidate for 3d/volumetric text.

    Here is displacement + HDR. (For people reading bottom to top, this is runtime modeled in Unity from a TTF)
    magic3.jpg

    Here is how it look before tweaking the material properties.
    magic3b.jpg
     
    Last edited: Nov 23, 2016
  24. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    A variation of above, and to created this scene:

    1) Select the font "CloisterBlack.ttf"
    2) Type your text
    3) Tweak your material
    4) Move the character around

    The crucibles are the "......." of the text.
    beat2.jpg
    beat2b.jpg
     
    Last edited: Nov 23, 2016
    yc960 likes this.
  25. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    Awesome, what are the compatible rendering paths? Forward, Deferred or both? How does it play with image effects like the Unity's official post process stacks?
     
  26. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    I shown both so far (forward and deferred), but forward is currently only supporting a directional light.
    (Unity 5.6 or later could support a better forward mode)

    note: In forward rendering, there is a Unity "bug" where dynamic batching is not working correctly.
    So I had to disable "dynamic batching" in the forward mode example.

    But at this stage, everything works seamlessly in deferred (including HDR).
    So you can stack post processing FX as you want.

    - The first image in this thread is using the Unity screen space reflection
    - The last example use the bloom post effect
     
    yc960 likes this.
  27. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Title screen example with localization.

    main : Blade 2.ttf
    subtitle : SHLOP.ttf / Arialuni.ttf

    blade1.jpg
    blade2.jpg
    blade3.jpg

    Material tweak example (raw UI, not final)
    mat.jpg
     
    Last edited: Nov 24, 2016
    yc960 likes this.
  28. zelmund

    zelmund

    Joined:
    Mar 2, 2012
    Posts:
    437
    can you make a Mortal Combat logo ? will be awesome )))
     
  29. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    To make the logo look like the full detailed original you would need a displacement map.

    Here is my attempt with : mkarmageddon.ttf

    edit: fixed the parser mk5.jpg
    mk4.jpg

    Tweaked the displacement map. (and more efficient)
    mk3.jpg

    Here is the vector data used
    MK.png
     
    Last edited: Nov 29, 2016
    Griffo, zelmund and yc960 like this.
  30. zelmund

    zelmund

    Joined:
    Mar 2, 2012
    Posts:
    437
    holy... :eek:
     
    Griffo likes this.
  31. QFSW

    QFSW

    Joined:
    Mar 24, 2015
    Posts:
    2,906
    This is looking pretty neat. followed
     
  32. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    Can we have some transparent examples?
     
  33. zombiegorilla

    zombiegorilla

    Moderator

    Joined:
    May 8, 2012
    Posts:
    9,042
    Love what I am seeing. Following.
     
  34. SarfaraazAlladin

    SarfaraazAlladin

    Joined:
    Dec 20, 2013
    Posts:
    280
    Yeah, this looks amazing. Totally following.
     
  35. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here is a quick proof of concept. (The black box is a standard unity box to show depth interaction with the smoke)
    The smoke look better animated, I will post a video later.

    note: I have 3 transparent mode at the moment. Shell, Solid, FX . I will post more examples to show the various usage.

    smoke2.jpg
    trans1.jpg
     
    Last edited: Dec 2, 2016
    yc960 likes this.
  36. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    I could be interested if:
    1-Work on mobile even low end. If it cannot run fast, maybe an option to preload and cache.
    2-Can avoid runtime allocation for simple scenarios.
     
  37. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    1- Right now the target is 2.5 (shader 3.0), so should be pretty much all mobile, beside early windows phones?
    But from what I can tell ATM, forward mode rendering will only support 1 directional light until Unity 5.6 is out. But some hack could probably be made to support 4 point light... (Deferred as no limitation)

    2- Some allocation free code path could be possible.
    But having your usage scenario detail would be needed to confirm this possibility
     
    Last edited: Dec 2, 2016
  38. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    This is not final (quality or performance)...

    Here the density is directly applied to the transparency and the color is solid white.


    smoke3.jpg



    smoke4.jpg
     
    Last edited: Dec 2, 2016
    yc960 likes this.
  39. yc960

    yc960

    Joined:
    Apr 30, 2015
    Posts:
    228
    I am now convinced that this is a must have, when can we get our hands on it?
     
  40. gian-reto-alig

    gian-reto-alig

    Joined:
    Apr 30, 2013
    Posts:
    756
    Just to add my 2 cents:

    Holy ****! This is amazing!


    Would I be interested in this if it was on the asset store? Of course! Price has to be right, but given its not grossly overpriced I would buy it on a whim just to play around with it. I am certain there are a ton of uses just for the 3D text alone.


    Keep up the good work.
     
  41. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    I think that there is a solid 4 to 8 weeks of work to have a private beta worth it.

    Right now its still in the "pet project" category,
    but I think as more people show interest I prioritize this more.
     
  42. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    I will keep posting my progress, thank you.

    For the price, I'm unsure how to position it.
    I dont want to play around with the feature set and make a watered down version,
    so I might look to see if I can make it "big mac" cheap for the educational users VS commercial.
     
  43. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    Here the smoke is contained inside the carved letters. (Should look better when the lighting is correct)



    smoke5.jpg
     
    Last edited: Dec 2, 2016
  44. Player7

    Player7

    Joined:
    Oct 21, 2015
    Posts:
    1,533
    Start off with a Happy Meal release, and then progress to a Big N'Tasty :p
     
  45. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    +
     
  46. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I can't be the only one thinking that this should be more than just restricted to text...
     
    one_one, pixelsteam and QFSW like this.
  47. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    This is nice, i was going to just use SVGs and pre-created models but this is more interesting.
     
  48. sschaem

    sschaem

    Joined:
    Feb 14, 2014
    Posts:
    148
    I currently only support quadratic bezier (and line segments) since this cover all the requirements of TrueType.
    I looked into supporting postscript font, so adding cubic bezier would be manageable.
    But anything beyond that is not even on my radar.

    Also this might be important to you, but no mesh data is accessible to the CPU or vertex shader.
    So the surface data is only available in the fragment shader.
     
    yc960 likes this.
  49. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    That is fine. Your system enables the data transforms for prettiness and such but exterior to that, the data, regardless of what it looks like represents something else to my app. That keeps those things easy & separate. I look forward to this product.
     
  50. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    is it compatible with textmesh pro? I imagine so as this is a shader...