Sunday 9 June 2013

Fastest ETC compressor on the planet: etcpak

[edit: new version is available]

My new (mobile) game has quite a lot of assets. Four different resolution sets, more than 23000 source images in each of them. After packing everything into atlases, the resulting data set is about 1106 Mpixels big. Since each pixel occupies 4 bytes (RGB + alpha channel), the raw data would roughly fit on a DVD disc.

Of course, 3D hardware supports texture compression basically since the beginning of 3D revolution, which is quite some time. In the mobile space there are two major texture compression formats. The first one (and the only one supported on iOS) is PVRTC. The second one is ETC (Ericsson Texture Compression) and it's supported by virtually all OpenGL ES devices. It's also here to stay, as ETC compression support is mandatory in OpenGL ES 3.0 and OpenGL 4.3.

Now, the problem is that compression takes time. A looong time. On a dedicated i7 CPU it takes about 3 hours to compress all my atlases to PVRTC format, using ImgTec's PVRTexTool utility. ETC is better with about half an hour, but that's still unacceptable for a quick, iterative development. There are various quality settings, you can choose between perceptual and non-perceptual processing, but even in the fastest mode the compression is still unbearably slow.

There are other compression utilities available, but they fare no better. I am aware of the following ones:
I have a test image, which is a real-life data 4096x4096 RGBA texture atlas filled up to about 87%. Since some tools load PNG files and other require PPM input, which is basically streaming raw image data from the disk, I have measured the load time of the PNG test image on my i3 540 to be 0.67 second. Every utility loading PNG image will have that time deducted from total time, even if the program reports it took longer (for example, crunch says it loads the texture in 1.029s).

ToolCommand lineTime
PVRTexToolCL 3.40PVRTexTool.exe -i atlas-base1.png -o pvr.pvr -f ETC1 -q etcfast24.71 s
ericsson ETCPACK 1.06etcpack.exe -s fast -e nonperceptual atlas-base1.ppm etc.ktx23.86 s
mali etcpack 4.0.1etcpack.exe atlas-base1.ppm . -s fast -e nonperceptual -c etc119.20 s
crunch (rg-etc1) 1.04crunch_x64 -ETC1 -fileformat KTX -mipMode none -uniformMetrics -dxtQuality superfast -file atlas-base1.png4.41 s

So, crunch is really fast, isn't it? Well, I didn't know that before I set out to write my own compression utility. And it runs circles around crunch. The compression time is 0.45 s. That's not a typo, it's 10x as fast as the fastest utility previously available. It's 50x as fast as PVRTexTool. And it has a special mode for processing alpha channel textures. Creating two ETC textures, one with RGB data and a second one with alpha channel takes 0.69 s. That's the time it takes to decompress the PNG image. And it's so fast you will be limited by HDD I/O wait.

As for the resulting image quality, my tool was never intended for production usage. And for testing during development it doesn't look that bad. Take a look.

OriginalCompressed

You can download the Windows executables (both 32 and 64 bit, but use 64 one, as it's a lot faster) from https://bitbucket.org/wolfpld/etcpak/downloads. As usual, MSVC redist is required.

Source code can be found at bitbucket.