Sprite Loading and Batching at Runtime

Hello all.

So I've been discussing with @JoeStrout and @superpig how we might be able to use the current API to load Sprite assets placed in folders outside the build (for me it's the StreamingAssets/ folder) and still benefit from the batched draw calls that make Sprites so wonderful.

Thanks to superpig's help, I was able to make this handy little script that picks takes any .png images placed in StreamingAssets/Data/Textures/Icons/Items/ and converts them into a single Sprite atlas that can be accessed from code just like those created inside the Unity editor. I wanted to share this code so that anyone who is looking for how to do this can see how. Also, I'm sure this code is riddled with efficiency issues so please let me know if you think there is a way to optimize it in a meaningful way.

Right, so on to the code. I have two functions to handle this.

Note: Usings

Code (CSharp):

1. using UnityEngine;
2. using System.Collections.Generic;
3. using System.IO;

1. Grab the image files from the necessary folder (StreamingAssets/Data/Textures/Icons/Items/) convert them to Texture2D objects and put them into a Texture2D array.

Code (CSharp):

1.        
2. GlobalVars.Icons_Items = new List<Sprite>(); //I store these sprites here, but you can use any Array/List to store them that you like.
3. Texture2D[] texGrp; //Texture2D array to create the packed texture
4. Sprite[] sptGrp; //Array of sprites created from the packed texture
5.  
6. //First, we grab all the files in the directory. I'm using the StreamingAssets path here, but you can go with whatever you like.
7.  
8. string[] arTex = Directory.GetFiles(Application.streamingAssetsPath + "/Data/Textures/Icons/Items/");
9. List<Texture2D> texList = new List<Texture2D>();
10.  
11.         for (int i = 0; i < arTex.Length; ++i)
12.         {
13.             if (arTex[i].EndsWith(".png"))
14.             {
15.                 byte[] iBytes = File.ReadAllBytes(arTex[i]); //Read the bytes of the image
16.                 Texture2D rTex = new Texture2D(1, 1);
17.                 rTex.LoadImage(iBytes); //Use the bytes to make a new Texture2D
18.                 texList.Add(rTex);
19.             }
20.  
21.         }
22.        
23. texGrp = new Texture2D[texList.Count];
24. texList.CopyTo(texGrp); //Copy the Texture2D list to an array
25.  
26. sptGrp = SpriteLoader(texGrp); //This function is shown below
27.  
28.         if (sptGrp != null)
29.         {
30.             for (int i = 0; i < sptGrp.Length; ++i)
31.             {
32.                 //Last part is to put the loaded sprites somewhere we can access later
33.                 GlobalVars.Icons_Items.Add(sptGrp[i]);
34.             }
35.         }
36.  

Code (CSharp):

1.     public static Sprite[] SpriteLoader(Texture2D[] texes)
2.     {
3.         Texture2D spAtlas = new Texture2D(4096, 4096); //Texture atlas we will be using for the packed textures
4.         Rect[] spRefs; //Rects returned from the image packing function.
5.         Sprite[] arSprites = new Sprite[texes.Length]; //The sprite array we are returning in this function.
6.         spRefs = spAtlas.PackTextures(texes, 1, 8192);
7.  
8.         //This next part is a bit weird. The rects returned by the PackTextures function are actually percentages of the texture's pixels in double form. So they need to be converted to actual pixel space before moving forward.
9.  
10.         for(int i = 0; i < spRefs.Length; ++i)
11.         {
12.             Rect nRect = new Rect(spRefs[i].x * spAtlas.width, spRefs[i].y * spAtlas.height, spRefs[i].width * spAtlas.width, spRefs[i].height * spAtlas.height);
13.             spRefs[i] = nRect;
14.         }
15.  
16.         for(int i = 0; i < spRefs.Length; ++i)
17.         {
18.             //Almost there, just need to create the sprites and assign the necessary Rect
19.             Sprite sp = Sprite.Create(spAtlas, spRefs[i], new Vector2(spRefs[i].x, spRefs[i].y));
20.             arSprites[i] = sp;
21.         }
22.         //Send back the sprite array to be added to the list in the function above.
23.         return arSprites;
24.     }

So that's pretty much it. I tested this in a scene by having the sprite array populate a Canvas > Panel > Grid Layout Group and the result was a single draw call!

Hope this is useful to someone.

 

Hmmm....
How many textures have you tried this with? And what are the size of the textures? I could see this being an issue with many large textures that are ranging between 2048 or 4048 in size.

 

A very good point. I've tried this with around 300 textures. As the path in the function above suggest, mostly icons. So I've been using stuff that ranges 128x128 to 300 x 150. The real reason I wanted to do this was because I'm trying to make my game as mod-friendly as possible, allowing people to supply their own icons for custom made items or portraits for custom made characters is a pretty big need and this seems (knock on wood) to do the trick.

The main issue I had before using this method was that I was pretty much going to have to sacrifice a lot of draw calls by using RawImage components and just dealing with standard Texture2D files. This allows me to batch those all by category, so all item icons are one draw call, with an inventory that's got a lot of functionality, that really is an important feature. With this method I'm able to keep that modding ability and still benefit from the reduced draw calls that the new Unity UI provides.

I would say it probably wouldn't be useful for certain texture sizes, particularly ones in the 2048/4096 sizes. But that all depends on how you set it up really and what you're looking to achieve. I would be interested to find out what it would look like if someone did try merging a large number of big textures.

Thanks for chiming in!

 

I see. Great concept. I'm going to experiment with this. I work for texture sizes of 2048 and 4048 on the daily and have about 20 different atlases at max capacity with over 100 different fullsize non compressed images. . Be interesting to see what this does.

 

Thanks for giving it a try!

Like I said, I'm sure there's some optimization that can be done in both functions I posted, so there's always that to consider. I would hope this method has application possibilities beyond what I'm using it for, so I'll keep my fingers crossed for your success!

 

I also using this method, packted a atlas at runtime, and create sprites from the atlas, like this code:

Code (CSharp):

1. Rect[] rects = atlas.PackTextures(_txt2d, 2, 1024);
2.                     for (int i = 0; i < rects.Length; i++)
3.                     {
4.                         int x = Mathf.FloorToInt(rects[i].x * atlas.width);
5.                         int y = Mathf.FloorToInt(rects[i].y * atlas.height);
6.                         int blockx = Mathf.FloorToInt(rects[i].width * atlas.width);
7.                         int blocky = Mathf.FloorToInt(rects[i].height * atlas.height);
8.                         Rect _rect = new Rect(x, y, blockx, blocky);
9.                         Sprite sprite =  Sprite.Create(ArenaResourceLoader.atlas, _rect, new Vector2(_rect.x, _rect.y));
10.                         if (!string.IsNullOrEmpty(iconUrls[i]))
11.                             atlasRectDict.Add(iconUrls[i], sprite);
12.                     }

Then I get the sprite from the dic, but the draw call no reduced, why?

Code (CSharp):

1. [code=CSharp]if (atlasRectDict.TryGetValue(iconUrls, out _sprite))
2.             {
3.                     GetComponent<Image>().sprite = _sprite;
4.             }

[/code]

 

I'm actually noticing that my draw calls aren't being reduced anymore as well. This is very strange since I know for a fact that when I tested this back in March it worked as intended. I wonder if something changed in recent releases that would make this no longer work?

Hopefully we can get @superpig to give us some info on what's going on here. Has there been a change to the API that makes this solution not viable?

Any info would be greatly appreciated.

 

@superpig I hate to bug you but I'm not really sure if this is something I can wrap up in a bug report as I'm not sure if there's a new way we're supposed to be doing this to get sprites created at runtime to batch properly.

Any ideas as to why this isn't working to reduce draw calls anymore?

 

Thanks for the heads up superpig! I took a look at things in the Frame Debugger and it does appear that the sprites are part of an atlas, though I'm not able to access the atlas as a whole, just the portion that the sprite uses. Still drawing each as an individual mesh though and not batching it with the others.

I'm not sure if it's something to do with the code attached to the icons (they are interactive, not sure if that's an issue) or if it is an issue with the shader I'm using. Next thing I'm going to try is to recreate the issue in a fresh project and see what happens.

Can you recommend someone on the dev team who might know what to do?

I really appreciate the help!

 

I figured that might be the case. I'll take a stab at doing that then. Thanks for the help!

I'll make sure to update this thread with whatever conclusion we come to.