large screenshots for print

Hi.

I'm trying to figure out how to output some large screenshots for print. They need to be bigger than my maximum screen size and I'd like to get some input on how to aproach that to see if it's something I should dive into.

1. First thought is using a large rendertexture and saving that to disk, but I'm thinking the step from rendertexture to 'file on harddisk' is not very straightforward? Also, would there be any limitation to the size of the rendertexture?

2. Another thought is to use the built-in CaptureScreenshot function somehow. Maybe a script that pauses the game, creates a camera that moves in a grid and takes multiple screenshots that can be stitched together outside unity?

3. Should this be done with a plug-in?

Any input appreciated,

thanks.

/Patrik

 

Right, I've wandered off to these parts of the woods before and got completely lost :-D . Not sure I'll get much out of it, but is there some good resources for learning abt projection matrices? ( I'll google it of course ;-) )

Doing it orthogonally would work I guess, but that's not good enough in most cases.

Thanks, Patrik

 

Note that doing this thing won't work proper with most image filters (e.g. glow/blur type will overflow between your images).

Nintendo had a paper on a solution to this: render the same image multiple times, but use the projection matrix to do subpixel offsets. Then stitch them together on a pixel-by-pixel basis.

say, you screenshot 4 times:
at base pos
basePos + (.5,0) pixels
basePos + (.5,.5) pixels
basePos + (0,.5) pixels

then you put the pixels next to each other - so your screenshot order becomes:

0 1 0 1
3 2 3 2
0 1 0 1
3 2 3 2

this will still make glow be a bit wrong, but most likely you can't see it - I've noticed these effects in a lot of hi-res prints from PS2 games.

If you want antialiasing, you just render more and blend some of them on top of each other....

the paper is in Game Programming Gems 4, written by Steve Rabin, and is called "Poster Quality Screenshots". You're welcome to drop by our office and borrow the book.

 

In Director you did this by rendering to an offset screen and saving the information from the rendering, may it by plugin or by pixel.

 

Things are so much simpler in my head...

Thanks I'm pretty sure it's way over my head, but I'll come by this week and take a look at it.

/Patrik

 

Wow, this question touched on a problem I was just thinking about: how to render just 2/3 of the width/heigh of the camera's view,--the bottom-right 2/3--but have that fill the screen.

My goal is to have the vanishing point of the screen be in the top left. You'd never see the top or left of the image, just the vanishing point, some space around it, the right side, and the bottom. Why? So that HUD stuff can overlay the bottom and right portions, leaving the vanishing point properly centered in the remaining open screen.

My first thought is to do this using Matrix4x4.TRS:

Code (csharp):

1.  
2. camera.projectionMatrix = camera.projectionMatrix * Matrix4x4.TRS( Vector3(HOffset,VOffset,0) , Quaternion.identity , Vector3.one );
3.  

And for purposes of printing/stitching, you'd need to zoom in, so you'd use a scale smaller than one:

Code (csharp):

1.  
2. camera.projectionMatrix = camera.projectionMatrix * Matrix4x4.TRS( Vector3(HOffset,VOffset,0) , Quaternion.identity , Vector3(ImageScale,ImageScale,1) );
3.  

I'll report back.

 

Nope. Doesn't work.

That offsets the camera itself, not the rendered rectangle--and thus perspective is not preserved.

I'll start a new thread to inquire on that subject rather than risk pulling this one off track

 

Thanks to the little side-track started by Morgan, and the matrix manipulation code by Aras i now have a small script to create large screenshots. It actually generates a series of screenshots that need to be manually stitched together afterwards, but it works.

Like Nicholas mentioned, this approach doesn't work well with glow and other Image Effects.

So, the thing I want to get working now is some kind of automatic stitching of the individual screenshots. I'm thinking this needs to be done in a plug-in where I can create a large image file on the HD and paste the smaller ones into that. Anybody up for it?

The code to generate the smaller screens look like this. To take a screenshot simply set takeScreenshot=true. the scale variable determines how big the resulting image is compared to the original, but it's exponential so a value of 2 gives an image 4 times larger. It should be attached to the camera

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. [RequireComponent (typeof (Camera))]
5. public class LargeScreenshot : MonoBehaviour {
6.    
7.     public int scale = 2;
8.     public bool takeScreenshot = false;
9.    
10.     private float left,right,top,bottom,w,h = 0;
11.     private float timeScaleReset = 1.0f;
12.     private bool takingScreenshot = false;
13.    
14.     void Update () {
15.         if(takeScreenshot){
16.             if(!takingScreenshot)
17.             StartCoroutine("TakeLargeScreenshot");
18.             takeScreenshot = false;
19.         }
20.     }
21.    
22.     void LimitsFromCameraProjectionMatrix(){
23.         Matrix4x4 m = camera.projectionMatrix;
24.         w = 2*camera.nearClipPlane/m.m00;
25.         h = 2*camera.nearClipPlane/m.m11;
26.     }
27.    
28.     IEnumerator TakeLargeScreenshot(){
29.         takingScreenshot=true;
30.         LimitsFromCameraProjectionMatrix();
31.        
32.         Camera cam = camera;
33.         //stop time
34.         timeScaleReset = Time.timeScale;
35.         Time.timeScale = 0;
36.        
37.         //Take partial screenshots
38.         int x = 0;
39.         int y = 0;
40.         int i = 0;
41.         int n = scale*scale;
42.         while(i<n){
43.             //Set up projection matrix
44.             left = x*w/scale-w/2;
45.             right = left+w/scale;
46.             bottom = y*h/scale-h/2;
47.             top = bottom+h/scale;
48.             Matrix4x4 m = PerspectiveOffCenter(left, right, bottom, top, cam.nearClipPlane, cam.farClipPlane );
49.             cam.projectionMatrix = m;
50.             //Capture screenshot
51.             Application.CaptureScreenshot("screenshot_"+x+"_"+y+".png");
52.             x++;
53.             if(x>=scale){
54.                 x=0;
55.                 y++;
56.             }
57.             i++;
58.             yield return 0;
59.         }
60.         //reset camera
61.         cam.ResetProjectionMatrix();
62.         //reset time
63.         Time.timeScale = timeScaleReset;
64.         takingScreenshot=false;
65.         yield return 0;
66.     }
67.     Matrix4x4 PerspectiveOffCenter(float left, float right, float bottom, float top, float near, float far){        
68.         float x =  (2.0f * near)       / (right - left);
69.         float y =  (2.0f * near)       / (top - bottom);
70.         float a =  (right + left)     / (right - left);
71.         float b =  (top + bottom)     / (top - bottom);
72.         float c = -(far + near)       / (far - near);
73.         float d = -(2.0f * far * near) / (far - near);
74.         float e = -1.0f;
75.  
76.         Matrix4x4 m = Matrix4x4.zero;
77.         m[0,0] =   x;  m[0,1] = 0.0f;  m[0,2] = a;   m[0,3] = 0.0f;
78.         m[1,0] = 0.0f;  m[1,1] =   y;  m[1,2] = b;   m[1,3] = 0.0f;
79.         m[2,0] = 0.0f;  m[2,1] = 0.0f;  m[2,2] = c;   m[2,3] =   d;
80.         m[3,0] = 0.0f;  m[3,1] = 0.0f;  m[3,2] = e;   m[3,3] = 0.0f;
81.         return m;
82.     }
83. }

/Patrik

 

Good work Patrik. Looks like Wiki material to me. I will have to try this out.
AC

 

Cool!

And maybe the assembly automation could be done another way, like in Photoshop itself.

 

Something similar to this stitching was brought up at cheetah3d.com/forum and the solution was attributed to graphic converter. the example discussed was image sequence as filmstrip.

See:

http://www.cheetah3d.de/forum/showthread.php?t=2002

I havent used it but it might be worth the effort invested to try it out.
AC

 

Credit should go to Aras, it's completely dependent on his matrix manipulation code.

Using Photoshop or GraphicConverter is possible, but for simplicity it would be nice to have an 'all-in-one' function where you wouldn't need any manual post processing. Also, doing a version using the technique Nicholas described would only be possible with some custom program/plug-in.

I've been looking at this article abt merging images in .NET using C#. I have no clue of how to do that in a plug-in, but if we can get that in place maybe it would be possible to start looking into the 'Glow-safe' method aswell

/Patrik

 

Hello!

Now that 2.0 is out do we have this "save to disk". If not what would be the way to save render texture to disk?

 

Thanks Aras,

In documentation there is something about active RenderTexture. How can I set specific RenderTexture active or determine which MRT is active?

 

Thanks again,

Missed that one. This works great. Here's the code if anyone's interested:

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. using System.IO;
4.  
5. public class TakeHires : MonoBehaviour {
6.    
7.     private Texture2D ScreenShot;
8.     private RenderTexture rt;
9.     int res = 4096;
10.    
11.     // Use this for initialization
12.     void Start () {
13.         rt = new RenderTexture(res,res,24);
14.         ScreenShot=new Texture2D(res,res);
15.         camera.targetTexture=rt;
16.         StartCoroutine("take");
17.     }
18.    
19.     IEnumerator take() {
20.         yield return new WaitForEndOfFrame();
21.         RenderTexture.active = rt;
22.         ScreenShot.ReadPixels( new Rect(0, 0, res, res), 0, 0 );
23.         ScreenShot.Apply();
24.        
25.         byte[] bytes = ScreenShot.EncodeToPNG();
26.         Destroy( ScreenShot );
27.         File.WriteAllBytes(Application.dataPath + "/../SavedScreen.png", bytes);
28.     }
29. }
30.  

 

Thanks for doing this phantom, Dr Jones just pointed me here. I am not a coder so please excuse the ignorance, but I'm not quite getting the image I expected:
I set a save path in the script, attach the script to a camera and press play. The image saves to disc but as this:



I've added a black background to see it easier but basically only the house renders at full transparency. and the background terrain and tree bark textures are missing, only a black silhouette in their place.
Finally would it be easy to convert this to JavaScript? I'm sort of starting to understand that a wee bit hehe.
TIA
Boxy

 

Yep:

Code (csharp):

1. import System.IO;
2.  
3. private var screenShot : Texture2D;
4. private var rt : RenderTexture;
5. var res : int = 4096;
6.    
7. function Start () {
8.     rt = new RenderTexture(res, res, 24);
9.     screenShot = new Texture2D(res, res);
10.     camera.targetTexture = rt;
11.     Take();
12. }
13.  
14. function Take () {
15.     yield WaitForEndOfFrame();
16.     RenderTexture.active = rt;
17.     screenShot.ReadPixels( new Rect(0, 0, res, res), 0, 0 );
18.     screenShot.Apply();
19.      
20.     var bytes = screenShot.EncodeToPNG();
21.     Destroy( screenShot );
22.     File.WriteAllBytes(Application.dataPath + "/../SavedScreen.png", bytes);
23. }

But yeah, it seems to not like taking pictures of the terrain for some reason...haven't looked into that.

--Eric

 

Great thanks Eric.
Thought I'd try and input some standard dimensions I'm using for illustrations 5000 x 2724 px. It worked but my mac actually crashed, I got the screen of death, wee. I think this is the first one I ever had!
If I duplicate the layer enough times in Photoshop I get full transparency back but as you can see the background and tree trunks aren't being rendered at all and anything in front of them is being culled also, except the house and fps prefab?
I think this is going to be a really useful illustration aid when I can get it to work because setting up terrains in Unity is so much easier than in C4D
Cheers
Boxy

 

camera.Render() didn't help in this case (though it's a good idea...I just sort of translated the script without actually reading it ). Turns out the problem was that the texture doing ReadPixels needed to be RGB24. Kind of strange that the terrain gets rendered to transparent otherwise (a shader thing?), but whatever works....

Code (csharp):

1. var resWidth : int = 2048;
2. var resHeight : int = 1152;
3.  
4. function Start () {
5.     var rt = new RenderTexture(resWidth, resHeight, 24);     
6.     camera.targetTexture = rt;
7.     var screenShot = new Texture2D(resWidth, resHeight, TextureFormat.RGB24, false);
8.    
9.     camera.Render();
10.     screenShot.ReadPixels(Rect(0, 0, resWidth, resHeight), 0, 0);
11.     var bytes = screenShot.EncodeToPNG();
12.     System.IO.File.WriteAllBytes(Application.dataPath + "/../SavedScreen.png", bytes); 
13. }

@boxy: I'm a bit surprised that resolution worked at all (even with a crash), since there are some hardware limitations involved. I know my card won't do textures past 2048x2048. I believe yours is supposed to do 4096x4096 though.

--Eric

 

Oh yes yes! Thank you Eric, that works great. I am also surprised the first one worked but I won't be trying it again
Thanks again
Boxy

 

Oh I'm back on this again
I wanted to grab a screenshot only when pressing a key rather than at the start, so I modified it and attached it to the fps camera:

Code (csharp):

1. var resWidth : int = 4096;
2. var resHeight : int = 2232;
3.  
4. function Update() {
5.     if (Input.GetKeyDown ("k")) {
6.    var rt = new RenderTexture(resWidth, resHeight, 24);  
7.    camera.targetTexture = rt;
8.    var screenShot = new Texture2D(resWidth, resHeight, TextureFormat.RGB24, false);
9.    
10.    camera.Render();
11.    screenShot.ReadPixels(Rect(0, 0, resWidth, resHeight), 0, 0);
12.    var bytes = screenShot.EncodeToPNG();
13.    System.IO.File.WriteAllBytes(Application.dataPath + "/screenshots/screen3.png", bytes);
14.     }  
15. }

It works, but realtime shadows don't show up on my buildings. Any ideas?
Much appreciated as always!
Thanks
Boxy

 

Strange...I did a quick test with that and realtime shadows are showing up here.

--Eric

 

Hmm thanks Eric, it may be something to do with that building or its shader setup then, I'll try some other objects.
Cheers
Boxy

 

Sussed it, I changed my editor quality settings>shadow cascades to four and they now show up.
Cheers
Boxy

 

I've been playing with the sub-pixel method mentioned by Nicholas and have a few issues. I think I'm doing something wrong with the sub-pixel offset as the edges in my images get a wavy look.


(don't mind the dots in the blue area...)

For setting the projectionMatrix I'm using this function from the docs:

Code (csharp):

1. static function PerspectiveOffCenter(
2.     left : float, right : float,
3.     bottom : float, top : float,
4.     near : float, far : float ) : Matrix4x4
5. {        
6.     var x =  (2.0 * near) / (right - left);
7.     var y =  (2.0 * near) / (top - bottom);
8.     var a =  (right + left) / (right - left);
9.     var b =  (top + bottom) / (top - bottom);
10.     var c = -(far + near) / (far - near);
11.     var d = -(2.0 * far * near) / (far - near);
12.     var e = -1.0;
13.  
14.     var m : Matrix4x4;
15.     m[0,0] = x;  m[0,1] = 0;  m[0,2] = a;  m[0,3] = 0;
16.     m[1,0] = 0;  m[1,1] = y;  m[1,2] = b;  m[1,3] = 0;
17.     m[2,0] = 0;  m[2,1] = 0;  m[2,2] = c;  m[2,3] = d;
18.     m[3,0] = 0;  m[3,1] = 0;  m[3,2] = e;  m[3,3] = 0;
19.     return m;
20. }

And, some simplified code for rendering the offset images:

Code (csharp):

1.  
2. //SUB PIXEL OFFSET (spo)
3. mWidth = camera.nearClipPlane/camera.projectionMatrix.m00;
4. mHeight = camera.nearClipPlane/camera.projectionMatrix.m11;
5. spo.x = mWidth/(Screen.width*scale*1.0);
6. spo.y = mHeight/(Screen.height*scale*1.0);
7.  
8. //RENDERS THE OFFSET IMAGES. SCALE=2 -> 4 IMAGES
9. for(var y=0;y<scale;y++){
10.      for(var x=0;x<scale;x++){
11.           var m:Matrix4x4 = PerspectiveOffCenter(left+spo.x*x,right+spo.x*x,bottom+spo.y*y,top+spo.y*y,camera.nearClipPlane,camera.farClipPlane);
12.           camera.projectionMatrix = m;
13.          //RENDER BUFFER TEXTURE
14.      }
15. }

My guess is that either

1. The sub pixel offset is wrong (spo var)? Seems to be good though, as changing it gives even stranger results. Someone who actually knows how the projectionMatrix works, please fill in
2. The PerspectiveOffCenter function is not what I should use in this case? Again, my projectionMatrix knowledge is limited so perhaps in this case the offset should be done differently.

Second issue I have is the dots in the blue area of the image above. I'm stopping time with Time.timeScale(), and it works fine in most cases. The only thing unique about the backround is that I'm using a vertex shader to extrude a mesh and it seems to stop moving a few frames later than everything else? Any reason this should happen for that case?

Any help appreciated.

/Patrik

 

Did anyone ever try this on a windows build? I've not had any luck getting offset projection matrixes working in unity on the pc. I'd like to include an option for users to generate high res images but it has to work cross-platform.
I don't want to use the rendertexture method as the output isn't anti-aliased.

 

Hardware Raster Operators are not working as our thinking... I tried some other ways

1. move the camera rect
2. move the camera's local position

to achieve sub-pixel re-Sampler. But all fail ... each 8*8 block result nearly same color ...

here's my code

Code (csharp):

1.  
2.     public IEnumerator UpScaleScreenshot()
3.     {
4.         //stop time
5.         timeScaleReset = Time.timeScale;
6.         Time.timeScale = 0;
7.  
8.         yield return new WaitForEndOfFrame();
9.         takingScreenshot = true;
10.  
11.         // save camera state
12.         Camera cam = camera;
13.         Vector3 cam_position = cam.transform.localPosition;
14.         Set_Texture_Mips(-2.0f);
15.  
16.         // Screen size in world space
17.         float screen_world_height = 2.0f * cam.nearClipPlane * Mathf.Tan(cam.fieldOfView / 2.0f * Mathf.Deg2Rad);
18.         float screen_world_width = cam.aspect * screen_world_height;
19.  
20.         // Screen size pixels
21.         int screen_width = Screen.width;
22.         int screen_height = Screen.height;
23.  
24.         // pixels size in world space
25.         float pixel_width = screen_world_width / screen_width;
26.         float pixel_height = screen_world_height / screen_height;
27.  
28.         Texture2D base_scale_buffer = new Texture2D(screen_width, screen_height, TextureFormat.RGB24, false);
29.         Texture2D ram_cache = new Texture2D(scale * screen_width, scale * screen_height, TextureFormat.RGB24, false);
30.  
31.         //int step = 1;
32.         for (int step = 0; step < scale * scale; step++)
33.         {
34.             int offset_w = (step % scale);
35.             int offset_h = (step / scale);
36.             Vector3 sampler_offset = new Vector3(offset_w * pixel_width, offset_h * pixel_height, 0.0f);
37.             //cam.transform.localPosition = cam_position + sampler_offset;
38.             Rect offseted_rect = camera.rect;
39.             offseted_rect.x = offseted_rect.x + sampler_offset.x / screen_width;
40.             offseted_rect.y = offseted_rect.y + sampler_offset.y / screen_height;
41.             offseted_rect.xMin = offseted_rect.xMin + sampler_offset.x / screen_width;
42.             offseted_rect.yMin = offseted_rect.yMin + sampler_offset.y / screen_height;
43.             offseted_rect.xMax = offseted_rect.xMax + sampler_offset.x / screen_width;
44.             offseted_rect.yMax = offseted_rect.yMax + sampler_offset.y / screen_height;
45.             cam.rect = offseted_rect;
46.             print("Camera displays from " + offseted_rect.xMin + " to " + offseted_rect.xMax + " pixel");
47.             cam.Render();
48.             base_scale_buffer.ReadPixels(new Rect(0, 0, screen_width, screen_height), 0, 0);
49.             base_scale_buffer.Apply();
50.             for (int w = 0; w < screen_width; w++)
51.             {
52.                 for (int h = 0; h < screen_height; h++)
53.                 {
54.                     ram_cache.SetPixel(offset_w + w * scale, offset_h + h * scale, base_scale_buffer.GetPixel(w, h));
55.                 }
56.             }
57.         }
58.  
59.         //Encode texture into PNG
60.         ram_cache.Apply();
61.         byte[] bytes = ram_cache.EncodeToPNG();
62.         string filename = Application.dataPath + "/../screenshot_upscale.png";
63.         System.IO.File.WriteAllBytes(filename, bytes);
64.  
65.         //reset camera
66.         Set_Texture_Mips(+2.0f);
67.         cam.transform.localPosition = cam_position;
68.         cam.targetTexture = null;
69.         RenderTexture.active = null;
70.  
71.         yield return 0;
72.         //reset time
73.         takingScreenshot = false;
74.         Time.timeScale = timeScaleReset;
75.     }
76.  

 

Did anyone create a functional script to take tiled screen shots and stitch them into one a large screen shot so far? It seems that there is no full solution in this thread although it is about four years old.

~ce

 

I would also be interested in a working script as I need screenshots for printing soon.

 

Megagrab

 

This script does not work with skyboxes, right?
Each screenshot fragment has the whole sky as background.