Get just the object's rotation matrix in shader (without view or projection included)

I'm making a shader for the rings around planets and I've ALMOST got it working.

I want the planet to cast shadows on the rings (but I think the distances I'm using are too big to get Unity's in-built shadow system working - I also think it might just be cheaper to code shadows, in the few cases where I need them, into my shaders myself).

I've got the rings calculating a shadow depending on how near the current fragment is to the line that points from the sun through the planet. But I can't seem to get that calculation factoring in the rotation of the rings. At the moment, the shadow falls as straight lines behind the planet but, if the rings are rotated steeply this won't look right. This is because the shader isn't taking into account the rings' rotation.

I need to multiply the current vertex position by, effectively, the quaternion rotation of the rings. I have tried all of the matrix rotations on this page...
https://docs.unity3d.com/410/Documentation/Components/SL-BuiltinStateInPrograms.html
...and also unity_ObjectToWorld and unity_WorldToObject.

Here is the code from my shader:

1. Shader "Materials/rings alpha blended" {
   Properties {
   _TintColor ("Tint Colour", Color) = (0.5,0.5,0.5,0.5)
   _MainTex ("Particle Texture", 2D) = "white" {}
   _InvFade ("Soft Particles Factor", Range(0.01,3.0)) = 1.0
   _sunDirection ("Sun Direction", Vector) = (0, 0, 0, 0) // a Vector3 from Unity that describes the position of the sun relative to the planet these rings orbit
   }
2. Category {
   Tags { "Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent" "PreviewType"="Plane" }
   Blend SrcAlpha OneMinusSrcAlpha
   ColorMask RGB
   Cull Off Lighting Off ZWrite Off
3. SubShader {
   Pass {
   
   CGPROGRAM
   #pragma vertex vert
   #pragma fragment frag
   #pragma target 2.0
   #pragma multi_compile_particles
   #pragma multi_compile_fog
   
   #include "UnityCG.cginc"
4. sampler2D _MainTex;
   fixed4 _TintColor;
   float4 _sunDirection;
   
   struct appdata_t {
   float4 vertex : POSITION;
   fixed4 color : COLOR;
   float2 texcoord : TEXCOORD0;
   };
5. struct v2f {
   float4 vertex : SV_POSITION;
   float3 worldPos : TEXCOORD1;
   float3 vertexFromCentre : TEXCOORD3;
   float3 rotatedVertexFromCentre : TEXCOORD4;
   fixed4 color : COLOR;
   float2 texcoord : TEXCOORD0;
   UNITY_FOG_COORDS(1)
   #ifdef SOFTPARTICLES_ON
   float4 projPos : TEXCOORD2;
   #endif
   };
   
   float4 _MainTex_ST;
6. v2f vert (appdata_t v)
   {
   v2f o;
   o.vertex = UnityObjectToClipPos(v.vertex);
   o.worldPos = mul(unity_ObjectToWorld, v.vertex).xyz;
   o.vertexFromCentre = v.vertex;
7. o.rotatedVertexFromCentre = mul(UNITY_MATRIX_IT_MV, v.vertex).xyz;
8. #ifdef SOFTPARTICLES_ON
   o.projPos = ComputeScreenPos (o.vertex);
   COMPUTE_EYEDEPTH(o.projPos.z);
   #endif
   o.color = v.color;
   o.texcoord = TRANSFORM_TEX(v.texcoord,_MainTex);
   UNITY_TRANSFER_FOG(o,o.vertex);
   return o;
   }
9. sampler2D_float _CameraDepthTexture;
   float _InvFade;
   
   fixed4 frag (v2f i) : SV_Target
   {
   #ifdef SOFTPARTICLES_ON
   float sceneZ = LinearEyeDepth (SAMPLE_DEPTH_TEXTURE_PROJ(_CameraDepthTexture, UNITY_PROJ_COORD(i.projPos)));
   float partZ = i.projPos.z;
   float fade = saturate (_InvFade * (sceneZ-partZ));
   i.color.a *= fade;
   #endif
   
   fixed4 col = 2.0f * i.color * _TintColor * tex2D(_MainTex, i.texcoord);
   UNITY_APPLY_FOG(i.fogCoord, col);
10. float depthValue = sqrt(length(i.worldPos.xyz - _WorldSpaceCameraPos) * 0.25f);
    col.a = col.a * saturate(depthValue) * i.vertex.z;
11. if (dot(normalize(_sunDirection), normalize(i.vertexFromCentre))<0) {
    float distanceToLine = length( cross( _sunDirection, _sunDirection + i.vertexFromCentre ) ) / length(_sunDirection);
    float sunlight = saturate((saturate(distanceToLine-1.9f) * 3) + 0.5f);
    col = col * sunlight;
    }
12. return col;
    }
    ENDCG
    }
    }
    }
    }

On the line that defines 'distanceToLine' I have tried using i.rotatedVertexFromCentre instead of i.vertexFromCentre and changing how rotatedVertexFromCentre is multiplied in the v2f vert function, using each of the matrix rotations on that page I linked to, but none works correctly. Anyone care to try this shader out and see if you can help?
That would be great!

 

I might have made this seem more complicated than I needed to, sorry; What I'd like to know is how to rotate a vertex by the object's rotation. I have tried all of the rotations listed on this page...
https://docs.unity3d.com/462/Documentation/Manual/SL-BuiltinValues.html
...(with the Object2World and World2Object updated to unity_ObjectToWorld and unity_WorldToObject for Unity 5). Any other ideas?

 

Sounds like what you need is a basic primer on transform matrices.

Quick overview of the included variables:
unity_ObjectToWorld - Transforms the mesh vertices from their local mesh space to Unity world space. This is the same world space as your scene. Fairly straightforward.
unity_WorldToObject - Transforms world space into local mesh space.
One thing that is important to understand here is this is not necessarily local gameObject space, this is local mesh space, and these can be different. Most commonly when your model's import settings have a scale factor. If you're using skeletal meshes it's the local position after being transformed by skeletal animation.

Then there's the other UNITY_MATRIX_* values:
UNITY_MATRIX_MVP - This is the most important one, this single matrix does all the transforms from the initial mesh space position into projection space (aka clip space). This matrix is the product of UNITY_MATRIX_M, UNITY_MATRIX_V, and UNITY_MATRIX_P together.
UNITY_MATRIX_M - This is identical to unity_ObjectToWorld, also called the Model transform.
UNITY_MATRIX_V - This is the transform from world space to local View space. This is similar to if you had an gameObject as a child of a camera gameObject, but without any scale applied. This means the positions are all in world space distances, but with the camera at 0,0,0 and rotated to match the camera's orientation.
UNITY_MATRIX_P - This is the transform from view space to Projection space. Projection space, or clip space, can be thought of as the position on screen, with anything on the far left edge of the screen, regardless of how far away, has an x of "-1", and on the right "1". It's actually going to be negative and positive "w", but we'll skip that for now.

There also exist combinations of these 3 main matrices, likes UNITY_MATRIX_MV, or UNITY_MATRIX_VP, which do exactly what you might expect. UNITY_MATRIX_MV transforms from model into view space, and UNITY_MATRIX_VP from world space into projection space.

There are also UNITY_MATRIX_T_MV, UNITY_MATRIX_IT_MV, UNITY_MATRIX_I_V and many other matrices that don't have the UNITY_MATRIX_* prefix, some of which are duplicates (like unity_ObjectToWorld and UNITY_MATRIX_M). We'll ignore those for now.

So that breaks down the different matrices that are available, but doesn't answer your question. For that you need to understand the basics of a transform matrix. A float4x4 transform matrix stores the scale, rotation, and translation with the first 3x3 being the scale and rotation and the last row float3 being the translation. The remaining float4 column we'll ignore as it doesn't really matter for what you're trying to do.

The short version is if you want an object transformed by the rotation and scale but not moved you only want to apply that float3x3 section of the matrix to the vertex positions. There are two main ways to do this:
mul((float3x3)unity_ObjectToWorld, v.vertex.xyz);
or
mul(unity_ObjectToWorld, float4(v.vertex.xyz, 0.0));

These are pretty much identical, though the first method returns a float3 and the second returns a float4.

If you want only rotation and not scale, that's harder and depends on exactly how you want to use the data. Most of the time if you don't care about scale you're dealing with directions, like surface normals. In that case you apply the rotation and scale matrix and normalize the result. Unity has a number of built in functions for this, almost none of which are listed in the documentation. Of you really need rotation with our scale and the vector unnormalized then you have to do a bunch more work.

 

Thanks for your more detailed explanation.
I think the problem is, in the context of my game, I don't know where converting to 'world space' moves the vertex. Ultimately, the star is at 0, 0, 0, but in-game I actually keep the player at 0, 0, 0 and move everthing around it.
So, then, my follow-on question is: If I convert the vertex to 'world space' where does it put the vertex position relative to the star's position?
Do I NOT need to include '_sunDistance' in the right-hand side of the distanceToLine calculation? (I've just tried removing it, and it doesn't make any difference, so I'm still at a loss).

 

World position in the shader is the same as world position in script. If you're moving everything around to keep the player at 0,0,0 in world space, that means everything else is moving in world space and unless the star and the player are in the same location it is no longer at 0,0,0.

Using the term world space is a bit confusing here too, since when dealing with solar systems you have actual "worlds", we had this problem working on Planetary Annihilation too since we had local-to-each-planet "world" space, as well as local-to-the-solar-system "world" space, and local-to-the-camera "world" space. However when talking about world space with Unity I mean what Unity refers to as world space when using any of the script side transform functions and properties or the object to world shader matrix.

Now to do a distance to line test you need to know three points. The start of or a point on the line, the end of or direction of the line, and the point to test against. Right now you're using a vertex position transformed by one of the shader matrices, and the position of or direction to your sun in some space, and assuming that 0,0,0 is the center of the planet / ring mesh for the third point. Obviously the vertex position and sun position need to be in the same space, but you can't also be sure that the local position center of the mesh is going to where you expect it to be because of Unity's dynamic mesh batching.

My suggestion is to pass the world position of the planet and the sun (as gotten from the script side transform.position). Then in the shader use the full unity_ObjectToWorld matrix to transform the vertex positions and do your line distance math with those.

 

I understand the concept of world space - and I'm not confusing it with each of the 'worlds' in the game. I understand how you can have a local vertex position relative to 0, 0, 0 of the OBJECT and a world position for each vertex that includes the object's position. I just don't understand, having rotated each vertex by ALL of the options available (including unity_ObjectToWorld) why nothing works. Can you see anything in my shader that's wrong?

 

o.rotatedVertexFromCentre = mul(unity_ObjectToWorld, v.vertex).xyz;
Doesn't just rotate into world space. It does the full transform including translation. You can exclude the translation part by using:
o.rotatedVertexFromCentre = mul((float3x3)unity_ObjectToWorld, v.vertex.xyz);
That's actually pretty much like bgolus already said.

 

Thanks, that seems to have worked! I wasn't sure how many things I had wrong, so it was hard to know which suggestions to include. Turns out I also need to scale the amount I subtract from distanceToLine (which makes sense, given that I'm rotating, scaling and now NOT translating each vertex position. I needed to undo the matrix scaling. Thanks again.

 

Hi, I've been making a volumetric galaxy using raymarching, and was looking for a technique to scale it vertically as well as rotate it so this looks perfect, only problem is that once applying it, it seems to do the opposite of what i want it to. If i rotate the object clockwise, the shader will rotate the galaxy counter clockwise. If i scale it in y, nothing happens. If i scale it in x or z, they scale in the opposite direction.

Could i please have some advice?

Edit: I've managed to fix some strangeness, the irregularity of it. However it is all still reversed

Edit the second: For anyone with the same problem, You just need to use unity_WorldToObject instead