For my app Portfolio, image loading performance is arguably the most important consideration, especially given that the typical image a user loads is in the 2048-4096 pixel range. These aren’t small images from Instagram. With the upcoming 4.1.0 update I have been putting significant work into maximizing the loading performance to get the most responsiveness and best framerate possible.
Portfolio pre-caches a couple different sizes when images are first loaded in, but due to the various ways someone might interact with the interface, it can have the need to generate new sizes on the fly to conserve memory (e.g., pinching to zoom in).
Existing Caching Options
Some time ago I had added SDWebImage as an intermediate layer to cache these ad-hoc representations to help improve performance – as long as there is disk space available, why not use some of it to prevent unnecessary image resizing? It worked, but it has always shown up when doing performance profiling in Instruments. While that’s not necessarily indicative of anything, it provided a good place to start looking for improvements.
SDWebImage saves cached images using UIImagePNGRepresentation. This is sensible for a basic, general purpose cache implementation, but I don’t need lossless caching in Portfolio so the performance hit from doing so is unnecessary.
I have looked at most of the notable iOS caching options:
While each would work to some degree, they’re not exactly ideal for the size of images Portfolio deals with: SDWebImage is generally I/O bounds, FastImageCache works best when all images are the same size, TMCache uses too much memory, and Haneke is also I/O bound. Based on that and my own testing of the image reading and writing options in iOS, I decided the best solution was to build a cache more suitable for Portfolio.
Building a Caching Solution
With Portfolio, image loading performance is far more important than image saving. Saving can be deferred and performed on a background thread, but image loading needs to be done as fast as possible when requested.
For some time Portfolio has used CGImageSourceCreateThumbnailAtIndex to generate the correct-sized image from the source, but this has proved only barely fast enough. It does this to generate non-full-resolution images that conserve on memory usage since iOS devices have always been severely constrained on memory, particularly when trying to work with large images. The warning issued by the system about using too much memory is generally too late to do anything useful, which then leads to being killed by the memory watchdog process and looking identical to a crash by the user.
I have been considering this problem since long before the version I decided to finally tackle it. Over that time I have been building on an initial idea of only loosely matching items in the cache. When Portfolio needs to load a full screen image, it doesn’t necessarily need that exact size – it could be a bit larger and still work fine as long as it isn’t so large to create memory problems. The approach I ultimately settled on still uses CGImageSourceCreateThumbnailAtIndex – it’s faster and of better image quality than resizing by drawing into a smaller bitmap context – but it’s now used only as a last resort. When Portfolio queries the cache it gives it a threshold value (right now this defaults to 20%). If there’s an existing image size within 20% of the size requested, it returns that instead of generating a new size.
When a cache request is made, the cache first checks its in-memory cache (just a standard NSCache instance) for an image within the threshold. Since an NSCache instance has no public method to return the keys that currently exist in it, a separate record is kept for all keys added to the cache (some of these might have been purged but it doesn’t matter since checking is relatively cheap operation).
If no image is found within the in-memory cache, it then checks the disk cache. The cache controller uses GCD to monitor the cache location for any changes and keeps the current file list in memory. This means that checking for any possible matches is not I/O bound at all – it only accesses the disk when attempting to load in the image.
On-disk images are stored as raw BGRA data (full credit to FastImageCache for this idea). These take up significantly more space than JPEGs or PNGs but make up for it by loading significantly faster. They also have the benefit of already being byte-aligned for Core Animation, which Portfolio uses extensively for displaying.
If the cache has not found a match in either the in-memory or disk caches Portfolio uses up its final optimization: it checks to see if the desired size is within 20% of the size of its pre-generated sizes (currently a thumbnail size and a full-screen size). If that matches it loads and uses that file. If not it creates the size it needs. Either way it adds that to the cache for future loading operations.
Caches are not a one-size-fits-all optimization. Most of the caching options available on Github are built around the more common use case of lots and lots of unique small images (e.g., social media thumbnails) rather than multiple versions of the same file. Portfolio’s image browsing performance is now back in line with how it was on the second generation iPad before it had to cope with four times the image data in the same timeframe. When performance matters enough, and there’s a way to leverage the unique needs of an app to improve performance, it’s worthwhile spending the time to do it.