Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Debugging shaders in Visual Studio

Discussion in 'Shaders' started by Elecman, Apr 29, 2015.

  1. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
  2. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
    I got it to work, but the Unity manual is both incomplete and the formatting could use some work also.

    Follow this guide and you should be able to get it to work:

    -------------------------------------------------------------------------------------

    Update software:
    -Download and install the DirectX SDK (June 2010).
    -Install the latest Windows Update.
    -Install Visual Studio 2015 (or higher) with the latest update. This is important because the updates often fix graphics debugging related bugs.
    -Install Visual C++ for VS2015. Note that in Visual Studio 2015, Visual C++ is not installed by default. When installing, be sure to choose Custom installation and then choose the C++ components you require. Or, if Visual Studio is already installed, choose File | New | Project | C++ and you will be prompted to install the necessary components, or run the installer again. More information:
    http://blogs.msdn.com/b/vcblog/arch...isual-studio-2015-affecting-c-developers.aspx

    Create a new project in Unity.

    Go to Hierarchy->Create->3D Object->Cube.

    In the Project window, right click and select Create->Shader->Standard Surface Shader. Leave the name as NewSurfaceShader.

    Open the NewSurfaceShader.shader file and add this line after "CGPROGRAM":
    Code (CSharp):
    1. #pragma enable_d3d11_debug_symbols
    In the Project window, right click and select Create->Material.

    Select the new material and go to Inspector->Shader->Select Custom/NewSurfaceShader.

    Drag the new material onto the cube.

    Go to File->Build Settings->Select PC Standalone, then click Switch Platform (if not already).

    Go to Edit->Project Settings->Quality->Inspector->Disable anti aliasing for all quality levels.



    Save the project.

    Go to File->Build Settings->Tick the box "Development Build" and "Script Debugging".



    On the Build Settings window, select Player Settings->Inspector->Resolution and Presentation->Display Resolution Dialog->Disabled.

    Select Player Settings->Inspector->Other Settings. Untick "Auto Graphics API for Windows" and check that Direct3D11 is at the top of the list. Also check if the Unity Editor title bar states "<DX11>".



    Select Build, name the file test.exe, and click Save. Note the path to the executable.

    Create a new project in Visual Studio 2015.
    File->New->Project->Installed->Templates->Other Languages->Visual C++->Empty Project.
    Note: if Visual C++ is not available, follow the instructions at the "Update Software" section above.



    Select a name and location for the project and click Ok.

    Go to View->Solution Explorer. Select the project (the project, not the solution).

    Go to Project->Properties->Configuration Properties->Debugging.
    Note: if Debugging is not available in the Configuration Properties, you selected the solution instead of the project in the Solution Explorer, or you created a C# instead of a C++ project.



    In the Command field, replace "$(TargetPath)" with the path to the Windows executable you build earlier.

    In the Command Arguments field, add "-force-d3d11".



    Click OK.

    Select Build->Build Solution.

    Go to Debug->Graphics->Start Diagnostics

    Install the Graphics Tools if a popup shows up.

    The standalone build should fire up and this message should appear in the top left: "Frames captured: 0. Use Print Screen key to capture a frame".



    Press the Print Screen key.

    Alt-Tab out of the standalone build and go to Visual Studio.
    Note: to close the Unity test app, select ALT-F4.

    You should see a frame capture in Visual Studio. Double click the captured frame and a new Visual Studio window should open.



    On the new Visual Studio window, Place the crosshair on the pixel of interest and left click. You can zoom in and out of the frame using the mouse scroll wheel.



    Go to the Graphics Event List window, select the first (top) occurrence of "obj:x DrawIndexed".
    Note: If you get a "No source available" warning later on, try one of the entries in the Camera.Render tree instead.



    Go to Graphics Pixel History window->obj:x DrawIndexed->Triangle->Vertex Shader->click the green arrow next to it.



    Now you can single step through your shader.



    Trouble shooting

    *If it displays the message "Source not available", then either:
    -You didn't add this line to the shader: #pragma enable_d3d11_debug_symbols
    -You didn't install the latest Visual Studio update.
    -You selected the wrong item in the Graphics Event List window. If the DrawIndexed item does not work, try one of the items in the Camera.Render tree.

    *Many variables display a NaN value due to a bug in VS. Variables only show the correct value if the line of code has just been executed. If you want to view the value of variables elsewhere in the shader, you have to step over them and note them down manually.

    *If the game is not displayed correctly after pressing Debug->Graphics->Start Diagnostics, then force use the integrated graphics instead of the high performance video card.

    *If Visual C++ is not available, follow the instructions at the "Update Software" section above.

    *If VS crashes, try running it as Administrator and make sure the latest VS update is installed.

    Further reading
    https://msdn.microsoft.com/en-us/library/hh972447.aspx

    Edit:
    -Updated to VS2015
    -Updated to Unity 5.2
     
    Last edited: Oct 20, 2015
    iSinner, Noisecrime, ATate and 15 others like this.
  3. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Are the steps reasonably close in VS2015? I cannot seem to find an empty project in the visual C++ section.

    Update:

    It appears I failed to install the C++ components when I setup 2015. :)
     
    Last edited: Oct 20, 2015
  4. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
  5. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    You are welcome. After installing the VC++, it worked exactly as described.
     
  6. ChiuanWei

    ChiuanWei

    Joined:
    Jan 29, 2012
    Posts:
    131
    cool tips,thanks
     
  7. Newcomma

    Newcomma

    Joined:
    Feb 10, 2015
    Posts:
    89
    Hi @Elecman I tried to follow these steps but when I hit "Start Diagnostics", even though my target for the project properties is set to run the unity editor executable, visual studio errors with

    'Unable to start program users/documents/testshader/debug/testshader.exe'

    I.e. it's still trying to start the VC++ project that was created rather than trying to run the executable, have I missed something?
     
  8. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
    Did you change the target path?

    Search for this line:
     
  9. Newcomma

    Newcomma

    Joined:
    Feb 10, 2015
    Posts:
    89
    Yeah I set the target correctly but the error still occurs. I've now gotten around this by opening the executable as a project and running from within visual studio which works ok.

    My new issue is that when I go to hit the green triangle debug button from within the graphics analyzer for the object with the shader set on it. Visual studio throws up a prompt asking to find the hlsl file (temp.hlsl.377_1.hlsl) which implies that it didn't manage to find the hlsl file.

    I can't seem to find this file anywhere in my project folder for where it might be built.
     
  10. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
    Yeah, the missing hlsl file thing is a bit of an issue. It can have many causes and sometimes even a restart will fix it. I have never gotten to the bottom of that.

    You can check these, but if that doesn't work, I don't have any more tricks to try.

    To be honest, the whole process is quite buggy, but it's the best thing we've got.
     
  11. egavi

    egavi

    Joined:
    Mar 15, 2016
    Posts:
    1
    Thank you for sharing! That was a great tutorial, and it worked for me.
    As you mentioned, debugging shaders with VS2015 is not a great experience.
    Could you please consider sharing tips on how to setup Nsight Visual Studio Edition 5.0 to debug dx11 shaders on VS2015?
     
  12. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
    Sorry, I have no clue about that one.
     
  13. cyberpunk

    cyberpunk

    Joined:
    Mar 20, 2013
    Posts:
    226
    Anyone know how to debug a compute shader?
     
  14. xophiix

    xophiix

    Joined:
    Dec 9, 2014
    Posts:
    3
    RenderDoc will automatically grab the temp hlsl file and all the cginc files.
    renderdoc_debug.png
    You can have a try on it.
    I think It' s easier to use with almost equivalent features as Visual Studio Graphic tool.
     
    Last edited: Apr 20, 2016
  15. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
    Thanks for pointing that out. I didn't even know that.
     
  16. Aku911

    Aku911

    Joined:
    May 18, 2016
    Posts:
    2
    cyberpunk likes this.
  17. Aku911

    Aku911

    Joined:
    May 18, 2016
    Posts:
    2
    Instead of creating an empty C++ project, you can just open the unity app as a project. Go to File | Open Project in Visual Studio and pick the compiled unity test app. Running ALT+F5 in VS will then let you capture that unity project.
     
    Last edited: May 18, 2016
    bellicapax likes this.
  18. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,369
    Thanks for the tip!
     
  19. cyberpunk

    cyberpunk

    Joined:
    Mar 20, 2013
    Posts:
    226
  20. S_P

    S_P

    Joined:
    Sep 20, 2014
    Posts:
    55
    Has anyone got "access denied" error? It appears when I try to start graphics debugging in VS 2015
    access denied.png

    EDIT : In case anyone face the issue, I've solved the issue.

    You have to mention the executable file name with extension as well with the Command directory in Properties > Debugging > Command.
     
    Last edited: Jun 7, 2017
  21. NikkiC5

    NikkiC5

    Joined:
    Jun 11, 2016
    Posts:
    11
    I've researched how to debug shaders in VS from multiple sources. I just can't seem to avoid the "Source not available". I've tried using VS 2015 and 2017. I've gone over different instructions many times. I'm not creating a C++ project, but opening directly from a compiled Unity project. I'll have to figure out another way of outputting shader data.:(

    edit: RenderDoc appears to prove more fruitful in terms of source debugging, using the same Unity executable.
     
    Last edited: Jul 25, 2017
  22. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
    I realize it's been a year since the last post but one thing that's changed since the tutorial above was posted, is that there's a separate Copy pdbs checkbox. Even if you check Development Build and Script Debugging, pdbs will not be copied over to the build path if Copy pdbs is not checked.No pdbs beside the executable, no debug symbols to be loaded into VS debug session. So VS can't really give you any info aside from "Source not available" and the like.

    On another note, I've debugged shaders in the past using this method, but for some reason I'm getting an access violation exception on app launch. Does anyone else get the same error?
     
  23. asperatology

    asperatology

    Joined:
    Mar 10, 2015
    Posts:
    981
    Oh my swirls.... This is new information. I hoped OP adds this to the #2 post.


    Also, just to add to this long thread:

    If you're only applying Compute Shaders for rendering, and you don't have Vertex Shaders and/or Pixel Shaders, everything you see in this thread does not apply to you.

    Visual Studio Graphics Debugging Tool cannot inspect Compute Shaders. Or I tried, but it only can inspect Vertex and Pixel Shaders.



    Those 2 green arrows are what you need to click on to enable shader source debugging. And you can see Compute Shaders are missing from there, because they don't do it, and rather, you would have to read up the Disassembly to figure out what the Compute Shaders are doing.

    I wasted 12 hours on this, for me to come to this conclusion, so you don't have to. Unless you can tell me how to debug Compute Shaders..., 'cause that would be swirly.
     
    Last edited: Apr 24, 2019
  24. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
    Are you sure that's the correct draw call?
    In this article, at Figure 7, it shows there is support for CS shaders(granted it's using VS2017) so look for a draw call that actually has a CS to run.
    There's also this walkthrough from MS docs.

    PS: I have written CS in Unity but it's been some time since and it was for a short time, so I don't remember if I used VS diagnostics. I might have used RenderDoc.
     
  25. asperatology

    asperatology

    Joined:
    Mar 10, 2015
    Posts:
    981
    You're right, it is not.

    After I cleared my mind and decided to jump back into the shader code, then went and debug the compute shader, it still doesn't exist in the Draw() call. It was then, I noticed that compute shaders are done by "dispatching" the threads to kernels to do the calculations, before the shaders are drawn.

    Navigating around the Timeline in the Event List, I found my Compute Shader.

    But stepping through the compute shader code, the execution jumps across multiple lines and returned early on, so I don't know too much about whether that's actually default behavior for compute shader debugging.
     
  26. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
    It's not surprising. Shaders by default get optimized and the pragma for exposing the dx11 debug symbols apparently does just that and does not disable shader optimizations. It seems there's no way to do that through Unity.
     
  27. asperatology

    asperatology

    Joined:
    Mar 10, 2015
    Posts:
    981
    I see. Shame, but thanks for the information.

    Sounds like I would need to conjure up a new Direct3D project in Visual Studio, and then debug the HLSL shader through there, before copying the fixed code back into Unity.
     
  28. lclemens

    lclemens

    Joined:
    Feb 15, 2020
    Posts:
    760
    Visual Studio 2019 + Unity 2020.2 + #pragma enable_d3d11_debug_symbols + selected on everything in the left Event List (including obj:4->DrawIndexed()). ==>

    upload_2020-12-23_20-18-11.png

    Maybe it doesn't work with Unity's .shader files?
     
  29. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
    Can't answer for sure, but I doubt the extension matters. Can you try RenderDoc? It's integrated in Unity, so you can do captures from inside the Editor.

    Btw, have you checked whether there's an update to Unity's tools for VS? Renderdoc shouldn't require latest VS version.
     
  30. lclemens

    lclemens

    Joined:
    Feb 15, 2020
    Posts:
    760
    I tried Renderdoc, but when I followed the tutorial the shader info was there, but there was no way to step through the vertex shader's code.
     
  31. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
  32. lclemens

    lclemens

    Joined:
    Feb 15, 2020
    Posts:
    760
    Thanks for the pointer! I took another crack at RenderDoc and it's working! The trick was finding the appropriate event in the Event Browser. It was buried under UIR.DrawChain/UIR.ImmediteRender/UniversalRenderPipeline.RenderSingleCamera: MainCamera/ScriptableRenderer.Execute: ForwardRenderer/DrawOpaqueObjects/RenderLoopNewBatcher.Draw/DrawIndexedInstanced.

    That took me a long time to find! Anytime I start a new capture I drill through 9 levels of hierarchy again. Is there an easier way to find the event that draws the mesh/texture I'm interested in?
     

    Attached Files:

  33. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
    Glad it worked for you.

    It depends. If we are talking about non-fixed target, then I don't know of a way. If you want to debug a single draw in-between captures, I'd use search in Event Browser(binoculars icon) and lookup e.g. DrawOpaqueObjects or RenderLoopNewBatcher, to quickly jump to a known location you know is close to the draw you're interested in.
     
    lclemens likes this.
  34. lclemens

    lclemens

    Joined:
    Feb 15, 2020
    Posts:
    760
    That works perfectly! It even saves the last search, so I can easily just hit the search button and then hit enter to go straight to it. Thanks again npatch, you rock!!
     
  35. npatch

    npatch

    Joined:
    Jun 26, 2015
    Posts:
    247
    No prob. Happy hunting!
     
    lclemens likes this.