Texture Compression Times Really Slow (ETC1/2)

Is anyone experiencing excruciatingly slow texture compression times for ETC1/ETC2?

I'm on Unity 5 with patch 0p3 now. Compression on the Android side used to be really quite fast (way faster than my pvrtc compressions on iOS), but now the times are much slower. It appears pvrtextool is running single threaded and its taking a 2048 texture over 30 minutes to compress on best quality (on a 3.4Ghz i7 5K iMac and longer on my older '08 Mac Pro).

Any suggestions or ideas whats going on considering how significantly longer its become? Scott

 

Doesn't it use etctool or something like that?

 

To be honest I can't recall ever checking until now but I was surprised to see pvrtextool being launched for each texture in the import process. I had to even check to make sure that it even supported etc compression, but according to their website; it does.

So I'm not sure if something is falling back to pvrtextool, this is a recent change, or what. I just know so far that the performance is incredibly bad. Scott

 

Maybe it's because you are running on MacOS. I'm in Unity 4.6 right now on Windows 8.1, and it pops up an ETC tool exe. So maybe I'm not the best help, but maybe it's something changed for Unity 5, or because it's using pvrtextool on a Mac.

I'm sure if it was taking everyone 30min there would be a lot more complaining going on though!!

 

This was the case for Unity 4 also. Superslow compression on my ETC2 textures. I really hate updating the textures now. But ETC2 gives very nice result on my Android app I am working on. Unity with the PVRTexTool running as a process spent over 24 hours on converting my project from Unity 4 to Unity 5

I am on PC

 

I did contact Imagination Technologies and this is what they said
"Unfortunately PVRTexTool doesn't have multithreaded ETC1 compression capability. It's a good suggestion and we will be investigating it for a future release."

so only options for now are:
* Use different tool
or
* Make Unity do threaded texture importing, when multiple textures are imported

 

Setting compression quality to normal takes just a couple of minutes on a 4K texture, but setting it to best takes over an hour.. strange

 

If it's based on the standard ETC compression code that Strom et al released, then, IIRC, there are three quality settings Fast, Medium, & Slow, and (according to their code) the latter two increase the search space relative to Fast by factors of around 27x and 125x respectively. They might also try test other modes which may increase the search time further.

 

I just installed Unity 5.0.1f1

Setting GLES 2.0 or GLES 3.0 compression on the Android build for example, a unitypackage import can take 30 mins per PNG / EXR file, absolutely killing development.

Any tips on how this can be improved, avoided? Do I only set compression at night before I go to sleep and process 8-10 pngs per day? =/

 

Unity's PRVTexTool is still taking forever to compress.
I did a non-unity project on iOS a few years ago and PRVTexTool was much faster.

 

I know this doesn't help make the compression times run faster but figured I'd share what I do.

1) installed asset server cache
2) turned off "compress assets on import" in Unity preferences
3) I also installed Unity on a second machine configured to share the same instance of the asset server cache but left the compress preference on.

When updating images or modifying atlases, the compression gets skipped -- until a build is performed making development quicker. To help cut down build times, I periodically will do one of two things:

1) when shutting down for the night, I'll turn on "compress assets on import" and let it run over night. The compressed textures will be updated in the cache for later.

or

2) I'll load the project up on my second machine letting me continue working while it compresses and updates the textures in the shared cache.

This is obviously for builds when I'm not using Unity's cloud build. It's not perfect but it at least keeps me from wanting to run in traffic Scott

 

I'm sorry, but it's unacceptable how this slow conversion process is single threaded. 15 hours ago I opened my Unity 4 project for the first time in Unity 5. I still don't have it open because it's still importing textures and only about 70% done. My task manager consistenly reads 8% CPU usage. Big fat waste of time, this is really annoying. It can't be that hard to launch multiple threads.

Is there at least a way to do this conversion in a multi threaded manner via some script?

 

That is great news, thank you in advance!

 

Hey florian, i just looked at 5.2 changelog and i couldn't find anything about multithreaded ETC compression.

Was it moved to a 5.2.x or later release?

Thanks

 

5.2.0 does have multi-threaded PVRTexTool. You can see that in your process monitor when compressing job is running.

 

The import does still seem to be incredibly slow compared to how it used to be. It only says ~1 sec on Normal, and up to 20 mins for best on a 2k texture using a quad core processor .

 

Well, PVRTexTool is a 3rd party software so I don't think you should address that to UT.

 

Is there any way to map "Best" back to "normal" quality for compression, it takes almost an hour to convert my project from 5.3 beta to 5.3 release... it reimported everything! which would be fine if i could find out which of those textures are not set to normal, without having to stare at the screen and write them all down =/

 

this tool is killing me.... still.

 

If you have only POT textures2D, then you can choose "Texture" type in Project window search bar, select all found assets, select Texture 2D in Inspector and set compression to Normal, this will set it to normal for all of them.
This won't work though if any of them is non-POT.

Oh, I forgot to mention that your textures should be of advanced type for compression option to show for all them selected.

 

Hi TimGS, some are advanced, some are not, some are True Color, some are compressed, some are POT, some are not.

 

Manual routine then.

 

yes , PVRTexTool.exe

 

This is killing me! I don't even wanna think about how slow this must have been before multithreading..
Any update on how to speed this up, or set quality to normal??

 

This new tool would speed up ETC2 tremendously, I'm hoping they integrate it into Unity sooner than later: http://forum.unity3d.com/threads/fast-etc2-encoder-etc2comp-unity-integration-please.431815/

 

It's been hours crunching textures, and it's still at 15%!

God, I only selected 10 textures with the Best setting. Is the difference perceivable and worth it?

I believe this is going to take about 24 hours, and there's one PVRTexTool active for each processor on my machine.

I killed the unity process but now I'm stuck on relaunch, it's importing everything in ETC1+alpha Best Quality.

I'd like to try to kill the processes but... what if I had to start over?

Isn't there a way to cancel the conversion process without doing a complete rollback of the project or deleting the pictures?

 

I had to go through every single texture and set its quality to Normal, instead of Best, that took a long time. Apparently at some point "Best" wasn't actually doing anything and it was instead falling through to "Normal" behind the scenes. Then they made Best work and it increased import by hours. There is no quick solution at the moment. And i couldnt even tell you if the quality is worth the wait.

 

It's definitely worth it in case of lightmaps. Everything else — its a waste of time imo.

 

I'm experiencing this in Unity 2017 android- I innocently applied 'best' quality to a 2k texture, thinking maybe it would take 2-4 times the usual time, but now 35mins later I'm still waiting.... There should at the very least be a 'Cancel' button so you can stop this and get on with your work. Also I am guessing when you upgrade Unity, or switch platforms this will happen all over again, which if you have set even a few textures to 'Best' could mean it will be an 'over night' job as opposed to an 'over lunch' job. I STRONGLY feel that Unity needs to do something to make 'Best' a) cancel-able and b) optional when you are switching platforms or opening a project after updating- this crap could easily cost you a day of work time if you or someone else had set 20 or so textures to 'best' without realizing the consequences...

 

Haha, and while I was typing that it just finished, and made virtually zero difference as far as I can see!

 

this issue killing me...

 

I bought an asset on the asset store. Imported it. It's now 12 hours in and only 50% done. ( Unity 2017.3.1f1 )

 

Because of slow compression, I'd recommend under Unity > Preferences (settings), disable "Compress assets on import."

 

I'm experiencing this as well.

It's been 4hrs now, and it's still at 80% completion.

 

Im wondering how unity wants to reduce this time to 500 milliseconds !
that would be amazing.

 

For those who are still having this problem, you could try using this awesome wrapper: https://github.com/fxgames/pvrtextool_wrapper (Install details are in the link. Don't forget to also read the precautions before installing)

After using this, it reduced up to 90% of compression time for me

 

The problem is not only that ETC texture compression time is very slow, it is ALWAYS (for each texture) possible to obtain a compressed texture with better visual quality than having top PSNR or SSIM - image quality metrics used in all compression tools nowadays. I have made some research on this topic and have a proposal of new image quality metrics DPSNR: https://bitbucket.org/eugene_eugene_123/dpsnr/downloads/ (presentation in Spanish, tools to compress to ETC1 and example images)

 

If anyone is stuck on relaunch you can stop it compressing by lunching unity hub and changing the platform. then in the build settings click the platform you want to switch back to and change compression option back. then when you switch platform you should be all good.

 

Doesn't seem to apply now (Unity 2018.4.x). I delete pvrtextool but no error is thrown. I replace it for log output but nothing comes.

 

We need GPU texture compression

 

I just want to bypass it but I cannot find the exe.

 

Same issue here, my project takes me one hour to reimport all, but takes me at least 12hours to switch to android platform and it seems won't finish forever on i7 8700 12 cores cpu

 

1 Disable compress textures on import ( in settings )

2 Unity is compressing all textures in the project folder when building, it would be faster if only textures that are actually used in the build are compressed

 

2019.4.16f did some speedups for ETC/ETC2/EAC compression

• Editor: Optimized mobile ETC/ETC2/EAC texture compression (about 1.4x speedup).

https://unity3d.com/unity/whats-new/2019.4.16

 

Yes thats great, but most time is spend compressing textures that are not in the build in my project.
If those can be excluded somehow that would save the most time.

I can do it manually but thats not practical during development because I imported assets from the asset store to create levels an only when they are finished do i know what textures I can manually delete from the project.

 

Thanks for your solution, I turn off compress to let me could work on my machine. My project just has so many textures(around 10,000 or more) so it really need seems forever to compress them. So I'm asking my company could give me a super power CPU machine to do the build.

 

Unity Accelerator might be a less expensive solution.