Rendering GL to a Texture2D immediately in Unity4

Dear forum

I need to render stuff using the GL class to a Texture2D immediately in Unity4. To make it happen I do the following:

1) Get a temporary RenderTexture and set up a camera to reference it
2) Draw stuff using the GL class
3) Force the camera to render
4) Grab the pixels from the RenderTexture by calling ReadPixels() on a Texture2D
5) Clean up.

I get an error saying that "ReadPixels was called to read pixels from system frame buffer, while not inside drawing frame." and nothing in the Texture2D.

To recreate the example put the script below on a camera in an empty scene.

I had this working in an earlier version of Unity3. What is the best practise of doing this now?

Carl Emil

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class GL2TextureExample : MonoBehaviour {
5.     Texture2D texture;
6.     Material material;
7.    
8.     void Start(){
9.         // create material for GL rendering //
10.         material = new Material( Shader.Find( "GUI/Text Shader" ) );
11.         material.hideFlags = HideFlags.HideAndDontSave;
12.         material.shader.hideFlags = HideFlags.HideAndDontSave;
13.     }
14.    
15.    
16.     void Update(){
17.         // render GL to texture when space is pressed //
18.         if( Input.GetKeyDown( KeyCode.Space ) ) texture = RenderGLToTexture();
19.     }
20.    
21.     Texture2D RenderGLToTexture(){
22.         // create necessary objects //
23.         Camera camera = new GameObject().AddComponent<Camera>();
24.         RenderTexture renderTexture = RenderTexture.GetTemporary( Screen.width, Screen.height, 0 );
25.         camera.targetTexture = renderTexture;
26.        
27.         // make sure we have a clear background //
28.         camera.backgroundColor = Color.clear;
29.         GL.Clear( true, true, Color.clear );
30.         camera.Render();
31.        
32.         // add stuff to the GL buffer //
33.         RenderGLStuff();
34.        
35.         // render to render texture //
36.         camera.Render();
37.        
38.         // read the current rendertexture into a new Texture2D texture //
39.         Texture2D newTexture = new Texture2D( Screen.width, Screen.height );
40.         newTexture.ReadPixels( new Rect( 0, 0, Screen.width, Screen.height ), 0, 0 );
41.            
42.         // apply pixels and compress //
43.         bool applyMipsmaps = false;
44.         newTexture.Apply( applyMipsmaps );
45.         bool highQuality = true;
46.         newTexture.Compress( highQuality );
47.        
48.         // clean up after the party //
49.         GL.Clear( true, true, Color.clear );
50.         Destroy( camera.gameObject );
51.         RenderTexture.ReleaseTemporary( renderTexture );
52.         Destroy( renderTexture );
53.        
54.         // return the goods //
55.         return newTexture;
56.     }
57.    
58.     void RenderGLStuff(){
59.         material.SetPass( 0 );
60.         GL.Begin( GL.LINES );
61.         GL.Color( new Color( 1, 1, 1, 0.2f ) );
62.         for( int i=0; i<100; i++ ){
63.             Vector3 position = Random.insideUnitSphere * 10;
64.             GL.Vertex3( position.x, position.y, position.z );
65.         }
66.         GL.End();
67.     }
68.    
69.     // put this script on a camera and uncomment the followng line to test how it should look like
70.     //void OnPostRender() { RenderGLStuff(); }
71.    
72.     void OnGUI(){
73.         GUILayout.Label( "Press space to test" );
74.         if( texture == null ) return;
75.         GUI.DrawTexture( Camera.mainCamera.pixelRect, texture );
76.     }
77. }
78.  

 

No ideas?

 

Three hours of pain. That's it! I give up! It's 'just' a performance optimisation anyway.

Say I have a complex piece of graphics rendered using the GL class that does not need to change over time. It would be LOVELY to render this graphics to a Texture2D once and be done with it. But Unity 4 does not seem to allow this, neither immediately nor at the end of the frame.

If someone have this working, please, please post an example.

X-|

 

It works! =D

A camera is not needed after all. I just had to draw directly to the active RenderTexture. I hope someone will find the example below useful. If you can figure out how to anti-aliase the Texture2D cheaply then please do share the solution.

Carl Emil

Code (csharp):

1.  
2. using UnityEngine;
3.  
4. public class GL2TextureExample2 : MonoBehaviour
5. {
6.     Texture2D texture;
7.     Material material;
8.    
9.    
10.     void Start(){
11.         // create material for GL rendering //
12.         material = new Material( Shader.Find( "GUI/Text Shader" ) );
13.         material.hideFlags = HideFlags.HideAndDontSave;
14.         material.shader.hideFlags = HideFlags.HideAndDontSave;
15.     }
16.    
17.    
18.     void Update()
19.     {
20.         if( Input.GetKeyDown( KeyCode.Space ) ){
21.             Destroy( texture );
22.             texture = RenderGLToTexture( 800, 400, material );
23.         }
24.     }
25.    
26.  
27.     static Texture2D RenderGLToTexture( int width, int height, Material material )
28.     {
29.         // get a temporary RenderTexture //
30.         RenderTexture renderTexture = RenderTexture.GetTemporary( width, height );
31.        
32.         // set the RenderTexture as global target (that means GL too)
33.         RenderTexture.active = renderTexture;
34.        
35.         // clear GL //
36.         GL.Clear( false, true, Color.black );
37.        
38.         // render GL immediately to the active render texture //
39.         RenderGLStuff( width, height, material );
40.  
41.         // read the active RenderTexture into a new Texture2D //
42.         Texture2D newTexture = new Texture2D( width, height );
43.         newTexture.ReadPixels( new Rect( 0, 0, width, height ), 0, 0 );
44.  
45.         // apply pixels and compress //
46.         bool applyMipsmaps = false;
47.         newTexture.Apply( applyMipsmaps );
48.         bool highQuality = true;
49.         newTexture.Compress( highQuality );
50.        
51.         // clean up after the party //
52.         RenderTexture.active = null;
53.         RenderTexture.ReleaseTemporary( renderTexture );
54.        
55.         // return the goods //
56.         return newTexture;
57.     }
58.  
59.    
60.     static void RenderGLStuff( int width, int height, Material material )
61.     {
62.         material.SetPass( 0 );
63.         GL.PushMatrix();
64.         GL.LoadPixelMatrix( 0, width, height, 0 );
65.         GL.Begin( GL.LINES );
66.         GL.Color( new Color( 1, 0, 0, 0.5f ) );
67.         for( int i=0; i<500; i++ ) GL.Vertex3( Random.value * width, Random.value * height, 0 );
68.         GL.End();
69.         GL.PopMatrix();
70.     }
71.    
72.  
73.     void OnGUI()
74.     {
75.         GUILayout.Label( "Press <SPACE> to render GL commands to a Texture2D" );
76.         if( texture == null ) return;
77.         GUI.DrawTexture( new Rect( Screen.width*0.5f-texture.width*0.5f, Screen.height*0.5f-texture.height*0.5f, texture.width, texture.height ), texture );
78.     }
79.  
80. }
81.  

 

Afaik its still deferred to the end of the unity frame, when unity will draw everything depending on the order the cameras are set. But OnPostRender function attached to a camera will get called the moment it's actually rendered if you need to do things in between.

 

You can attach the AA as Post Effect script to the camera you're using to render this image. If that somehow isn't an option, you could modify the AA as Post Effect script to work on a RenderTexture. You could also implement a form of MSAA where you'd simply render the scene to your render texture at 2 or 4 times the resolution of the final texture then apply a reconstruction filter to create your final anti-aliased image, but that would probably hurt your performance too much.

 

Thanks hippocoder and DethRaid.

There is no camera. When I add a camera to the soup then I can only render either 1) in a coroutine after WaitForEndOfFrame or 2) in the camera's OnPostRender method. I need to create the texture(s) immediately.

I guess I can use Graphics.Blit() with the AA Post Effect shader...

Right now I'm just rendering a larger size than what is needed and scaling it down (like hippocoder suggests) and this works fine for now.

 

Are you sure about this, can't you just use a disabled camera and call render() on it? Or does that get deferred to the end of the frame too?

Don't bother, this was my first thought, and I actually implemented it earlier. The results were less than stella, in fact with your dummy code they were downright ugly. Mind you it was using the luminance method, since no depthmap was available, perhaps that would give better results, but I doubt it. I suspect the post-AA simply doesn't work very well with just line graphics.

 

Ok, I see that I was a bit too quick to conclude things after being really confused over this problem. Yes, you can call Render on a disabled Camera whenever you like, I don't know if it will get deferred. However, let me correct:

– If you want to ReadPixels from the default RenderTexture.active, then you have to do so either after WaitForEndOfFrame or inside OnPostRender. I guess this is because the default active RenderTexture (the screen output texture) needs to be rendered first.

– If you create a RenderTexture and set it as RenderTexture.active, then you can ReadPixels whenever you like.

– (it seems to me) if you want to draw GL stuff to a RenderTexture you've created and assigned to a disabled Camera by calling Camera.Render, then you are in bad luck. I can't get this to work.

So instead of using a camera, I end up using the method in the latest script posted above.

Good to know =)

 

I know this is an old thread but I found it one of the few around the topic. I want just add that a Texture2D is not necessary as you can use the render texture directly especially if you don't use the temporary allocation but you create the rendertexture at the Start. Second, and important, you can not use the material the unity doc suggests: http://docs.unity3d.com/Documentation/ScriptReference/GL.html
It does not work with this example (I do not know why) so the code:

Code (csharp):

1.   material = new Material( Shader.Find( "GUI/Text Shader" ) );

is quite important (I lost several hours before discovering it)
Finally let us remember that after a setting of RenderTexture.active GUI Group is lost...