Skyshop - Image-Based Lighting Tools

Discussion in 'Works In Progress' started by monkeyscience, Mar 9, 2013.

Thread Status:
Not open for further replies.
  1. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    I've been working on bringing Marmoset Toolbag shaders into Unity, and along with it, a suite of tools for processing HDR skies into IBL data.

    Introducing: Marmoset Skyshop!
    [​IMG]

    Skyshop takes sky images (panoramas, crosses, or cubemaps) as input and will generate diffuse and specular IBL cubemaps from them.

    Web-Player Demo: http://marmoset.co/skyshopdemo/

    Mipmapped Gloss-
    The specular cubemap also stores varying gloss levels in the different mip-map levels, allowing for blurry specular reflections and gloss-maps in the IBL shader.
    A custom cubemap importer saves and loads these mip levels from sub-assets of the cubemap for now. Eventually I want to use DDS cubemaps for everything.

    Sky Objects -
    The IBL cubemaps can be stored and managed by a custom Sky object in your scene. Sky objects are treated as ambient light-sources in your scene and work along-side Unity's direct lights. They also take care of binding all cubemaps, skyboxes, and exposure settings to our IBL shaders globally, making swapping out skies and messing with exposure through scripting very easy.

    HDR(!) -
    High dynamic range skies are handled with a .pbm importer and turned into RGBM textures. All the shaders handle RGBM cubemaps as well, allowing for super light-weight HDR without floating-point textures or any hardware restrictions really; HDR that'll run on a phone!
    I plan to write more file formats later, .pbm was just one I had code for already.

    Gamma Correct-
    Much care is taken to do all the fancy maths and light convolution in linear color-space. All the output textures will work in both Linear and Gamma color-space so both Pro and Free versions of Unity are supported.

    [​IMG]

    Shaders-
    Along with the IBL tools, Skyshop will come with all the permutations of shaders and material types you've come to expect from Unity:
    diffuse, Blinn-specular, normalmaps, and transparency. The IBL overhead for a shader is really just two cubemap lookups so adding new shader types is easy and I plan to do more of it.

    The shaders I've written so far conform to what Toolbag uses and include parameters for:
    - Diffuse Color
    - Diffuse Intensity
    - Specular Color
    - Specular Intensity
    - Specular Exponent
    - Specular Fresnel

    And channels for:
    - Diffuse map (RGBA)
    - Specular (RGB) Gloss (A)
    - Normalmap (RGB)

    Support-
    Edit: Skyshop now works in both free and pro versions of Unity 3.5 and 4.
    Last edited: May 21, 2013
  2. God at play

    God at play

    Member

    Joined:
    Nov 3, 2006
    Messages:
    278
    So exciting to see Marmoset finally make its way to Unity! Looking great so far :)
  3. mstevenson

    mstevenson

    Member

    Joined:
    Sep 24, 2009
    Messages:
    139
    That's fantastic, I've been searching for exactly this sort of toolset.
  4. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    This is KICK ASS!!! I have custom shaders I made a while back...and I wonder if my shaders will work with this... and how much is this?
  5. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Thanks!

    I'm sure it will, using IBL cubes in a shader is not hard, once you have the data. Generating the data and making sure Unity does not eat your mipmaps along the way can be tricky though. We'll be polishing and selling the whole workflow, if that makes sense.
  6. mstevenson

    mstevenson

    Member

    Joined:
    Sep 24, 2009
    Messages:
    139
    Do you have an ETA for release?
  7. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    Well, the shaders were made in Strumpy... but if your shaders are flexible, this should not be a problem... My shaders do accept cubemaps so this transition will be painless. Also will this work with Beast?
    Last edited: Mar 10, 2013
  8. janpec

    janpec

    Member

    Joined:
    Jul 16, 2010
    Messages:
    3,330
    This is all well and great but we need screens!

    Cant wait:D
  9. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Whoa Firefox does not display my attached screenshots, fixed! I've also put up a webplayer demo at http://www.reinot.com/skyshop/
  10. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    ETA is in the coming weeks, its close :).
  11. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    I am very impressed with the webplayer...it definitely has the quality I am looking for too... how is it under strain during game play... and will this work with Light Probes, Beast, and skinned meshes? I have custom shaders that will accept cubemaps, and this will be a great addition... keep up the good work :)
  12. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Quick update: Unity 4 support, hazaa!

    I've also cleaned up the UI quite a bit and added an overlay of little bulb icons for all the directional lights in the scene. Very handy for positioning a direct light-source for the sun. You can even color-pick the hue and intensity of the light from the input image.

    [​IMG]
  13. God at play

    God at play

    Member

    Joined:
    Nov 3, 2006
    Messages:
    278
    Fantastic work :) Being able to visualize the placement of a light for the sun will make it worth using over the default skybox!
  14. God at play

    God at play

    Member

    Joined:
    Nov 3, 2006
    Messages:
    278
    Is it possible to rotate the skybox without going through the trouble of setting up another camera? I'd love to be able to do that, but for some weird reason, there's no skybox rotation with the default setup.
  15. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Yes, actually! The skybox and lighting orientation is hooked to the Sky object transform. So if for example you wanted to make the lighting environment spin around your model, you can script the Sky object transform to spin.

    Not as important for full games perhaps, but an IBL environment spinning slowly makes for great turn-table presentations, for example:
    Last edited: Mar 19, 2013
  16. BossaDigital

    BossaDigital

    New Member

    Joined:
    Mar 23, 2013
    Messages:
    2
    Oh wow, this is perfect for a project we're working on! You mention it working on pretty much any hardware; do you know if it'll play nice with exporting to Flash?
  17. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Good question, I tried it out in 3.5 this afternoon and Unity Flash has trouble even with the cubemap skybox. It will need a little more love before I can say we have Flash support, I'm afraid.
    Last edited: Mar 24, 2013
  18. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    One last pre-GDC update, I've made sure light probing and mapping all works. I also wrote a little UI for adding HDR skies into the Beast Settings XML file. Now any time you bake lightmaps or light-probes and Beast's raytracer misses your geometry, it samples your sky panorama!

    [​IMG]
    Beastly light-probing, featuring Joe Wilson's boss little Vespa

    p.s. We will be showing off Skyshop and our IBL Unity shaders at GDC13, albeit informally, like nomads on the expo floor. Look for scruffy fellas in "Marmoset Toolbag" T-shirts and say hello :).
  19. derkoi

    derkoi

    Member

    Joined:
    Jul 3, 2012
    Messages:
    1,221
    Looks great, I don't suppose it'd work on mobile very well?
  20. BossaDigital

    BossaDigital

    New Member

    Joined:
    Mar 23, 2013
    Messages:
    2
    In my own 4.0 tests, I've found Flash able to handle pretty much anything shader-related as long as you don't go above 4 texture lookups. Cubemaps work fine from what I've tested, so if your IBL technique only uses two cubemap lookups, then that'll leave 2 texture lookups left for whatever else :)
  21. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    This is great and I am glad this will also work in 3.5... I use Beast a lot in my work and this will be handy in lighting my levels... just need to make sure this will work with my custom shaders... I am very impressed with the sun placement... it is a PAIN to place the sun to match the sun spot...
  22. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    You also have to make sure this will not conflict with the Lightmap Extended addon... I use this instead of the regular Beast UI
  23. sc3

    sc3

    Member

    Joined:
    Nov 2, 2010
    Messages:
    101
    This looks great!
  24. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Lightmap Extended looks cool indeed. Nice work mstevenson! Maybe I can make skyshop work in tandem with it instead.
  25. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    PLEASE?!!! And could I send you one of my shaders to test with this tool?
  26. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Sure, send away or PM me.
  27. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    Sent...
  28. MatzeOGH

    MatzeOGH

    New Member

    Joined:
    Mar 20, 2013
    Messages:
    10
    How is it coming along?
  29. nukeD

    nukeD

    Member

    Joined:
    Feb 12, 2009
    Messages:
    270
    feed us!
  30. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    I've been working on gamma correction over the last few days and have finally arrived at a solution that will work with both linear and gamma-space projects. Some mobile platforms will not support gamma-correct rendering so it is important for Skyshop to still look good without it.

    Along the way I stumbled on a way to add more Dynamic Range to my High Dynamic Range, pow(2.2)'s worth of more range :). This means the brightest pixel in an RGBM sky can be ~51.5x brighter than white. Maybe not 2^16 as float16 would get you but decent enough for even direct sunlight.


    [​IMG]

    Gaze into my shiny, gamma-space orbs!
  31. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    This is looking spectacular... I can finally make use of the panoramas I make in Vue for my skies... and if you guys are bringing in the skin shaders from Marmoset, I will be a happy man and this will be a DAY 1 purchase if the price is fair, considering I own a marmoset licence :)... by the way, have you had a moment to test out the custom shader I sent out?
  32. Korda

    Korda

    New Member

    Joined:
    Jan 3, 2013
    Messages:
    37
    This looks really great! I am curious as to how much of a performance impact this has?
  33. angelodelvecchio

    angelodelvecchio

    Member

    Joined:
    Nov 15, 2012
    Messages:
    111
    I Just cant sleep since i see this, cant wait for release.

    Many thanks developer!, i am addicted to unity and just cant wait ;)
  34. Migueljb

    Migueljb

    Member

    Joined:
    Feb 27, 2008
    Messages:
    95
    Hey monkeyscience. Very interested in this. Any ETA when you could have this ready for purchase for us unity artists? I pretty much just use stock unity shaders which can only look so good and go so far. Ready to take unity to the next level with shaders like yours. Thanks looking great so far.
  35. kenshin

    kenshin

    Member

    Joined:
    Apr 21, 2010
    Messages:
    687
    Great tool!

    You have a customer and if needed a serious tester for your ETA.... just PM me!
    Kenshin
  36. derkoi

    derkoi

    Member

    Joined:
    Jul 3, 2012
    Messages:
    1,221
  37. JVaughan

    JVaughan

    Member

    Joined:
    Feb 6, 2013
    Messages:
    13
    Just caught up with this thread. Feel free to PM as soon as you release this. I'm a day one customer.
  38. janpec

    janpec

    Member

    Joined:
    Jul 16, 2010
    Messages:
    3,330
    Me too, please feed us with updates.
  39. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    A Quick Update
    Yesterday I finished up integrating Lightmapper Extended with some help from MStevenson. All the Beast global illumination controls are accessible straight from Skyshop, and change the same XML file as LMExtended without conflicting. I'm also going to bundle the full LMExtended package with Skyshop for the million other render quality and baking options it exposes.

    There is also a button in the output cubemap GUI to export an .HDR image and send it to the lightmapper's IBL sky field. And for that, I added .HDR support :). The importer works in much the same way as .PBM, an asset post-processor script parses .HDR files and generates RGBM .PNG files from them.

    [​IMG]
    "Use as GI" button sets cubemaps to be used as global illumination when baking.

    The ETA
    with those Lightmapper Extended changes I dub Skyshop feature-complete. I am combing over the mobile shaders, making sure they still run on iOS (yes, there will be mobile shaders :wink:), and we are gathering up all the marketing material, demos, and pretty pictures required for release.
  40. janpec

    janpec

    Member

    Joined:
    Jul 16, 2010
    Messages:
    3,330
    Awesome! So glad for this i was really looking up solutions how to make it the same way that Overgrowth team was doing it, and pretty much your tool is exactly the same. How much will it cost on Asset store?
  41. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    I second Janpec... I may finally ditch my strumpy made shaders for these... And I hope that the skin shaders have the great quality that I have been looking at for quite some time... I have been meaning to ask you, if the textures will be RGB(A) or will you do something like embed the Spec in lets say the Red Channel or Gloss in the green... I recently learned how to do this in photoshop so it would be useful... and this will definitely be a day one purchase :)

    And I am very happy to know that you got this working with Beast :)

    EDIT: How would I go about making these RGBM maps?
    Last edited: Apr 26, 2013
  42. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    Spec maps are specular intensity (RGB) gloss (A), colored specular ftw! :)

    RGBM conversion is all done internally. Skyshop is going to come with an image post-processor script which runs every time a .HDR or .PBM file is imported into the project, and adds another .PNG file to the project containing RGBM data.
  43. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    Good to know, since I really f'ed up my textures for converting for the Pre Integrated Skin Shader... and I need to really test the skin shader for marmoset, because I have NO idea how to set it up

    EDIT: It was easier than I thought, it is just a desaturated diffuse version of the skin. Once this is brought into Unity, I am DEFINITELY dropping my strumpy shaders...
    Last edited: Apr 26, 2013
  44. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    Also one more item to bring to your attention: When you move the marmoset skin shaders over to Unity, there may be an issue with light probes... you may lose the scattering feel to the skin if you use them... just a heads up... I am using 3 different skin shader packages and I am getting the same issues with all 3...
  45. nipoco

    nipoco

    Member

    Joined:
    Sep 1, 2011
    Messages:
    1,558
    Looks rad man!

    Can't wait to get my hands on these.
  46. JVaughan

    JVaughan

    Member

    Joined:
    Feb 6, 2013
    Messages:
    13
    :cool: Oh man. You guys are killing this. So hard. In the face. Can't wait for this to launch, I hope the Unity internal are taking notice and will tout this heavily.
  47. janpec

    janpec

    Member

    Joined:
    Jul 16, 2010
    Messages:
    3,330
    Oh just figured out this should be perfect for my metal texture problems. Considering first skull with heavy cubemap and metal look is all i need.

    So if i understand correctly you are importing Marmoset shaders, i am wondering what about lighting? Lighting setups of MT are really nice is there any possibility you would include them too?
  48. Neptune_Imaging

    Neptune_Imaging

    Member

    Joined:
    Jan 5, 2010
    Messages:
    2,677
    +1
  49. monkeyscience

    monkeyscience

    Member

    Joined:
    Dec 13, 2011
    Messages:
    572
    The iOS shaders are up and running! I'm waiting on some ShaderLab bug fixes in Unity 4.2 to get gloss-maps fully supported so the mobile shaders will likely ship as pre-compiled and in a "beta" status. Gloss-maps will work but I'm adding extension functions to compiled shader files by hand right now >_<.

    Here's a nice teaser on my iPad 3:
    [​IMG]
  50. angelodelvecchio

    angelodelvecchio

    Member

    Joined:
    Nov 15, 2012
    Messages:
    111
    I will have an Heart Attack, good luck! I really hope to see this great asset soon!, Many thanks!
Thread Status:
Not open for further replies.