Your Facebook News Feed is filled with photos of your friends, family, and loved ones — photos you may want to view on your phone. We are always looking for ways to make things better and faster for mobile. So, our team took a look at how we can make photos faster on iOS and we found a way to reduce the data used by Facebook for iOS by about 10% and show a good image 15% faster than before. Here's how we did it.
Up until now, Facebook for iOS app has loaded your photos in News Feed as follows:
Progressive JPEG (PJPEG) is an image format that stores multiple, individual “scans” of a photo, each with an increasing level of detail. When put together, the scans create a full-quality image. The first scan gives a very low-quality representation of the image, and each following scan further increases the level of detail and quality. When images are downloaded using PJPEG, we can render the image as soon as we have the the first scan. As later scans come through, we update the image and re-render it at higher and higher quality.
Support for PJPEG became popular in browsers in 2010, and we've been serving photos as PJPEGs for a while now. However, mobile apps haven't really caught up yet. For example, there is currently no out-of-the-box support on iOS for rendering images progressively, so we had to build our own for the Facebook app.
Rendering images progressively in the Facebook app has some advantages:
There is a downside to PJPEG: Decoding and rendering the image multiple times at varying scan levels uses more CPU. Decoding images can be moved to background threads, but the process is still heavy on CPU. The real challenge for us was to find the right balance between data usage, network latency, and CPU utilization. For instance, we considered using WebP since it is more optimal in file size than JPEG in some cases, but the format does not support progressive rendering.
The following diagram shows how we used to download photos in Facebook for iOS. Each bar indicates an image download, and “Wait Time” is the period of time between viewing a photo placeholder to viewing a photo that is clear enough to enjoy. Even when the smaller image appeared, many people ended up waiting for the full image:
Throwing PJPEG into the mix changes the picture:
We render three different scans of each photo:
The result is that people see a good photo sooner!
To determine what “good” means, we tried several different scan-levels and found the levels at which people interacted with photos the most. We also looked at the relative similarity between each of the scans and the final image. Our comparison function takes two images and returns a number between 0 and 1. A score of 0 means completely different, while 1 means exactly the same. Here are the results:
To measure the impact of choosing different scan levels, we ran an A/B test and then examined the data.
At Facebook, we continually work hard to reduce the amount of time you spend waiting, and this is just one of our many efforts. While PJPEG has helped make photos load faster, we know there is always room for improvement.
Many people worked on this effort; credit should go to Linji Yang, Miguel Cohnen, Kun Chen, Kirill Pugin, Edward Kandrot, Marty Greenia, Brian Cabral and Tomer Bar.