Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Runtime shader compilation

Discussion in 'Shaders' started by Sirithang, Apr 26, 2011.

  1. Sirithang

    Sirithang

    Joined:
    Nov 29, 2010
    Posts:
    28
    Hi!

    I've seen the probleme once or twice here and on Unity Answer, but I feel the need to ask one more time, and try to get some help before trying some things.

    I try to give a shader at runtime as a meterial. This Shader is either a string or read from a file. What i do with Unity is not a game, it's an interactive app, and so it is usefull to be able to modify the shader on the fly, tweaking parameter on the site where the app is installed.

    I understand that since the Cg compiler is part of Unity Editor and not the player, you can't just give your shader to Material and have it ok.

    But I remark that if you give to material the "Compile-xxx.shader", it work. So is there (but i doubt) an offline Unity shader compiler somwhere that allow to pass a "unity" shader throught and give the compiled version.

    Second thing : does giving glsl (or hlsl) shader to Material will work? I could try to pass my Cg shader throught the Cg compiler to get some HLSL/GLSL but before installing all the toolchain and start messing with that, i would like to know if someone have already tried this...


    Thanks for your answers.
     
    SamFernGamer4k likes this.
  2. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Aras would have to tell you the specifics, but at the moment I don't think there is a complete version of Unity's shader compilation pipeline that runs outside the editor. This means that Cg, GLSL and HLSL cannot be compiled by a running Unity player. You might be able to reproduce the part of the pipeline that you need by running the Nvidia Cg compiler with the right options, but that's not something I've ever tried.
     
    SamFernGamer4k likes this.
  3. ScriptGeek

    ScriptGeek

    Joined:
    Mar 4, 2011
    Posts:
    45
    I've been looking for an answer on how to do this same thing myself for the past couple days. I found something that might suggest that it is possible to create a material at runtime, along with its shader. I haven't yet tried this, but I'm about to start tinkering with it: http://unity3d.com/support/documentation/ScriptReference/Material.Material.html

    I need to be able to create the shader at runtime as well as make runtime changes. Finding a solution on this seems nebulous. If anyone knows how to do this, I'd greatly appreciate an explanation.

    Cheers!
     
  4. redcool007

    redcool007

    Joined:
    May 28, 2011
    Posts:
    2
    unity cannt compile shader at runtime,but can use the shader string generate Material at runtime.
    and the shader's string can download.
     
  5. Sirithang

    Sirithang

    Joined:
    Nov 29, 2010
    Posts:
    28
    Strange to revive this topic to respond to someone with more than 7 month :p

    But just so you know, I have found the mean to do this after long search in my previous company. This is how I proceed :

    In Editor\Data\Tools, you'll find something called CgBatch. It's an exe that work something like this (I trying to remember I've done that like 7 month ago):

    CgBatch "pathtoyourfile" "pathToInclude" "pathToOutput"

    now pathToInclude are CgInclude, it's : Editor\Data\CGIncludes

    Then the file you created in pathToOutput, you just open it, read its content in a string, and give to material. ET VOILA! you can compile Unity Shader at runtime.

    BEWARE : this is totaly undocumented, I've found this by playing around Unity for some time, so it may be subject to totally change etc...
     
  6. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    223
    Thanks for this tip - just saved me a metric assload of work figuring it out.
     
  7. Julian89

    Julian89

    Joined:
    Jan 21, 2013
    Posts:
    2
    Great tip, working on this atm. However it seems there is an additional parameter now (path, so it is now input, path, includes, output). I can't seem to figure out what is supposed to go here. The program is running correctly but it's not doing any compiling (so input = output), so I imagine. Has anyone figured this out yet?
     
  8. FuzzyQuills

    FuzzyQuills

    Joined:
    Jun 8, 2013
    Posts:
    2,871
    Last edited: Mar 3, 2014
  9. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    From that page:
    So yes, you can make shaders, but not programmable pipeline ones. To be honest, I'm not sure how well this even works since fixed function shaders are converted to programmable pipeline ones whenever possible nowadays.
     
  10. ilya_ca

    ilya_ca

    Joined:
    Nov 19, 2011
    Posts:
    274
    A little note on the correct parameters to supply CgBatch with (acquired by running SysInternals Process Monitor on CgBatch.exe when compiling a shader).

    Here's how Unity's query to it looks like:
    "C:\Program Files (x86)\Unity\Editor\Data\Tools\CgBatch.exe" "Temp/CgBatchInput.shader" "Assets/ShaderDirectory" "C:/Program Files (x86)/Unity/Editor/Data/CGIncludes" "Temp/CgBatchOutput.shader" "-xbox360" "-ps3" "-d3d11_9x"

    So the correct parameters are:
    CgBatch.exe "SHADER_PATH" "SHADER_DIR" "CGINCLUDES_DIR" "OUTPUT_DIR"

    It seems that the additional SHADER_DIR argument is used to look for additional *.cginc files that the shader may depend on.

    Good article on running cgbatch: http://kylehalladay.com/all/blog/2014/01/12/Runtime-Shader-Compilation-Unity.html


    Hope this may help someone.
     
    Last edited: May 16, 2014
    highlyinteractive likes this.
  11. PrisedRabbit

    PrisedRabbit

    Joined:
    Aug 14, 2012
    Posts:
    62
    Good article! But how to use new compiler UnityShaderCompiler?
     
  12. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Before Unity 4.5, shader compilation was done by this separate standalone "CgBatch.exe" tool. Shader in, compiled shader out. However, starting with 4.5, that has changed (see blog post). UnityShaderCompiler.exe is more like a "service process" - Unity starts it, and then sends compilation commands to it via a local socket/pipe.

    So it's not as simple as just calling it from command line anymore, I'm afraid.
     
  13. PrisedRabbit

    PrisedRabbit

    Joined:
    Aug 14, 2012
    Posts:
    62
    Thank you for respond. I found that CgBatch from previous Unity works (executes) in Unity 4.5, but have't tried yet to compile some shaders.
     
  14. [RV]CWolf

    [RV]CWolf

    Joined:
    Jan 24, 2013
    Posts:
    33
    Is there any chance of UnityShaderCompiler.exe supporting the functionality of passing it an uncompiled shader? We are exploring allowing modders to create their own full flavour shaders (not the basic ones that can be passed into Material by string).

    If not, would it be possible for a third party to make a tool that fed in compilation commands to the .exe? If so, any chance of some documentation of what the .exe is expecting? We'd be tempted to play around with it.

    EDIT:
    Otherwise, would the above solution of using an older CgBatch.exe work with later / future version of Unity?
     
  15. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Would asking modders to install the free version of Unity be too much? :)

    I think that would be the most robust solution. Since we can change how the shader compiler works at any point (just like we did in 4.5).
     
    SamFernGamer4k likes this.
  16. [RV]CWolf

    [RV]CWolf

    Joined:
    Jan 24, 2013
    Posts:
    33
    That's a fair enough idea and I think it's not a completely unreasonable prereq for anyone who wants to mod shaders since that would be a low level mod workflow in itself =). Didn't even think about getting them to install Unity. Doh!

    Thanks!
     
    SamFernGamer4k likes this.
  17. Ippokratis

    Ippokratis

    Joined:
    Oct 13, 2008
    Posts:
    1,521
    No free version of unity in ipad yet.
    Plus, fixed shaders are about to die in some months after Unity 5.0 is out.
    edit - No, they will not die, my bad.
     
    Last edited: Aug 24, 2014
  18. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Not sure I follow. Unity iOS & Android basic addons became free over a year ago.
    The discussion here has nothing to do with fixed function shaders. Also, I'm not aware of any plans to remove them either.
     
  19. Ippokratis

    Ippokratis

    Joined:
    Oct 13, 2008
    Posts:
    1,521
    Hi Aras,
    The purpose of this thread - as I understand it- is to find a way to compile shaders on device, in a unity produced app, pretty much producing something like shaderToy on steroids.

    On Desktop, perhaps it is not so important, since the user can install Unity Free as you said. But, in the case of an ipad "Would asking modders to install the free version of Unity be too much? :)", is not possible because there is not a Unity Editor version for the ipad.

    The alternative proposed by FuzzyQuills above, is using http://docs.unity3d.com/ScriptReference/Material-ctor.html , since it can be used in the produced app. The downside is that it can only produce fixed function shaders.

    I misread the release notes, indeed the fixed function shaders are not going to die. Mixing partially fixed function & partially programmable shaders will not be possible anymore - which is not something relevant. -closed 5.0 beta-

    For me personally the ability to compile shaders on device is not a top priority - it is nice to have. The potential of writing and testing shaders in a unity graphics pipeline context on an ipad - android tablet is not going to make massive sales but will make some people who write and test shaders very very happy - myself included.
     
  20. DBN

    DBN

    Joined:
    Feb 12, 2014
    Posts:
    1
    Would you mind giving more details? Copy pasting compiled shaders one at a time from Unity to an Audiosurf 2 mod folder was incredibly painful and I don't want to go back to that.
     
  21. GroenBoer

    GroenBoer

    Joined:
    Nov 30, 2014
    Posts:
    46
    Sorry to revive an old thread but is anyone still interested in this?
    I have created a service that provides Cg shader compilation service and if anyone is interested I can make it available as an asset on the asset store but it won't be cheap as it is server intensive.
     
    SamFernGamer4k and Primerist like this.
  22. Primerist

    Primerist

    Joined:
    Sep 19, 2016
    Posts:
    5
    I'm interested in seeing this asset package.
     
    SamFernGamer4k likes this.
  23. alexandre-chambriat

    alexandre-chambriat

    Joined:
    May 5, 2015
    Posts:
    16
    Any update on this ?
     
  24. Deleted User

    Deleted User

    Guest

    Here's how you can run the shader compiler programmatically:

    Code (CSharp):
    1.  
    2.         public void CompileAllPasses()
    3.         {
    4.                 var guids = AssetDatabase.FindAssets("t:shader");
    5.                 foreach (var guid in guids)
    6.                 {
    7.                     var path = AssetDatabase.GUIDToAssetPath(guid);
    8.                     var shader = AssetDatabase.LoadAssetAtPath<Shader>(path);
    9.                     Assert.IsTrue(shader, "Could not load shader at '{0}'.", path);
    10.  
    11.                     var hasError = ShaderUtil.ShaderHasError(shader);
    12.                     Assert.IsFalse(hasError, "Shader '{0}' has errors.", path);
    13.  
    14.                     var material = new Material(shader);
    15.                     int passCount = material.passCount;
    16.                     for (int i = 0; i < passCount; i++)
    17.                     {
    18.                         ShaderUtil.CompilePass(material, i, true);
    19.                     }
    20.                     Object.DestroyImmediate(material);
    21.                 }
    22.         }
     
  25. Przemyslaw_Zaworski

    Przemyslaw_Zaworski

    Joined:
    Jun 9, 2017
    Posts:
    327
    AFrisby and Invertex like this.