iOS, Mac OS X, and more

iOS

Building a Better Cache

For my app Portfolio, image loading performance is arguably the most important consideration, especially given that the typical image a user loads is in the 2048-4096 pixel range. These aren’t small images from Instagram. With the upcoming 4.1.0 update I have been putting significant work into maximizing the loading performance to get the most responsiveness and best framerate possible.

Portfolio pre-caches a couple different sizes when images are first loaded in, but due to the various ways someone might interact with the interface, it can have the need to generate new sizes on the fly to conserve memory (e.g., pinching to zoom in).

Existing Caching Options

Some time ago I had added SDWebImage as an intermediate layer to cache these ad-hoc representations to help improve performance – as long as there is disk space available, why not use some of it to prevent unnecessary image resizing? It worked, but it has always shown up when doing performance profiling in Instruments. While that’s not necessarily indicative of anything, it provided a good place to start looking for improvements.

SDWebImage saves cached images using UIImagePNGRepresentation. This is sensible for a basic, general purpose cache implementation, but I don’t need lossless caching in Portfolio so the performance hit from doing so is unnecessary.

I have looked at most of the notable iOS caching options:

While each would work to some degree, they’re not exactly ideal for the size of images Portfolio deals with: SDWebImage is generally I/O bounds, FastImageCache works best when all images are the same size, TMCache uses too much memory, and Haneke is also I/O bound. Based on that and my own testing of the image reading and writing options in iOS, I decided the best solution was to build a cache more suitable for Portfolio.

Building a Caching Solution

With Portfolio, image loading performance is far more important than image saving. Saving can be deferred and performed on a background thread, but image loading needs to be done as fast as possible when requested.

For some time Portfolio has used CGImageSourceCreateThumbnailAtIndex to generate the correct-sized image from the source, but this has proved only barely fast enough. It does this to generate non-full-resolution images that conserve on memory usage since iOS devices have always been severely constrained on memory, particularly when trying to work with large images. The warning issued by the system about using too much memory is generally too late to do anything useful, which then leads to being killed by the memory watchdog process and looking identical to a crash by the user.

I have been considering this problem since long before the version I decided to finally tackle it. Over that time I have been building on an initial idea of only loosely matching items in the cache. When Portfolio needs to load a full screen image, it doesn’t necessarily need that exact size – it could be a bit larger and still work fine as long as it isn’t so large to create memory problems. The approach I ultimately settled on still uses CGImageSourceCreateThumbnailAtIndex – it’s faster and of better image quality than resizing by drawing into a smaller bitmap context – but it’s now used only as a last resort. When Portfolio queries the cache it gives it a threshold value (right now this defaults to 20%). If there’s an existing image size within 20% of the size requested, it returns that instead of generating a new size.

When a cache request is made, the cache first checks its in-memory cache (just a standard NSCache instance) for an image within the threshold. Since an NSCache instance has no public method to return the keys that currently exist in it, a separate record is kept for all keys added to the cache (some of these might have been purged but it doesn’t matter since checking is relatively cheap operation).

If no image is found within the in-memory cache, it then checks the disk cache. The cache controller uses GCD to monitor the cache location for any changes and keeps the current file list in memory. This means that checking for any possible matches is not I/O bound at all – it only accesses the disk when attempting to load in the image.

On-disk images are stored as raw BGRA data (full credit to FastImageCache for this idea). These take up significantly more space than JPEGs or PNGs but make up for it by loading significantly faster. They also have the benefit of already being byte-aligned for Core Animation, which Portfolio uses extensively for displaying.

If the cache has not found a match in either the in-memory or disk caches Portfolio uses up its final optimization: it checks to see if the desired size is within 20% of the size of its pre-generated sizes (currently a thumbnail size and a full-screen size). If that matches it loads and uses that file. If not it creates the size it needs. Either way it adds that to the cache for future loading operations.

Conclusions

Caches are not a one-size-fits-all optimization. Most of the caching options available on Github are built around the more common use case of lots and lots of unique small images (e.g., social media thumbnails) rather than multiple versions of the same file. Portfolio’s image browsing performance is now back in line with how it was on the second generation iPad before it had to cope with four times the image data in the same timeframe. When performance matters enough, and there’s a way to leverage the unique needs of an app to improve performance, it’s worthwhile spending the time to do it.

Benchmarks

iOS Image Reading and Writing Performance

In looking for some performance improvements to make in my app Portfolio, profiling in Instruments pointed at the parts responsible for image reading and writing. For this I started by building a test app to benchmark a few different approaches.

Writing

iOS provides a variety of methods for reading and writing images. When used in moderation there’s usually little need to be concerned about which one(s) you choose, but when used heavily performance becomes an important consideration.

Performance tests of four methods of saving images in iOS

This benchmark shows the average of the time required to save a large-ish image (4096×1612) for three trials of 100 iterations. As you can see, saving the image as a moderately compressed JPEG is is over 800% faster than saving a PNG (saving a single PNG of that size averages 1.41 seconds), and saving direct BGRA data is still 200% faster.

An interesting observation I also made is that using the ImageIO framework to save a JPEG is effectively the same as using UIImageJPEGRepresentation but doing the same approach with PNG is slower than using UIImagePNGRepresentation.

Reading

The benchmarks I ran for image reading performance showed it was a significantly less expensive operation than writing, even in the worst case. These are also using the same size image and are the average for three trials of 100 iterations.

Performance tests of three methods of loading images in iOS

For most developers the first approach is likely to be the most common, and, for smaller image files, it’s probably sufficient. As shown by the third approach though, if you can afford the time and storage space to pre-cache the raw BGRA pixel data of an image, its speed increase is substantial. It also has the benefit of being already in a format Core Animation likes and won’t cause a background memory copy.

The middle option is one currently used by Portfolio to minimize memory usage – it loads only the size it needs rather than bringing the entire file into memory. Over its five year life more crashes have resulted from iOS suddenly deciding it was using too much memory than to any actual bug, which made the loading speed tradeoff worthwhile.

Conclusions

When you’re using small images there will likely be much more worthwhile performance gains to be had by optimizing other areas of your app. Large images are where image reading and writing optimizations really start to be important. Since writing an image can often be deferred to a background thread without any significant impact to the responsiveness of your app, writing out the BGRA data directly from a bitmap CGContextRef is usually worth the increase in time over saving a JPEG. In cases where you need lossless saving, it’s definitely an improvement over saving a PNG.

Reading an image from disk is a point where speed matters immensely because it’s often triggered by an immediate need to display the image. The longer it takes to load, the lower your framerate will be, and your interface will feel sluggish. For this reason it’s almost essential for you to pre-cache the image size(s) you’ll need to ensure fast reading later.