Twitter’s new AI to help Auto-Crop your Photos better.


Twitter announced that they’ve created a neural network which automatically crops your photos to produce much better results on the social networking service.

Twitter is been on the image sharing side since 2011. But cropping  images for neat previews wasn’t something they had been thinking off seriously. One strategy previously, though, was to employ face detection and crop around the most prominent face in each photo.

But, there are many images which consist more than faces and can often get “awkwardly cropped preview images.”

For its latest attempt at improving the photo cropping system, Twitter is teaching its AI new tricks. Instead of looking for faces, the AI will be hunting for “salient” regions of photos — the areas that people are most likely to look when freely gazing at the picture.

“Academics have studied and measured saliency by using eye trackers, which record the pixels people fixated with their eyes,” Twitter says. “In general, people tend to pay more attention to faces, text, animals, but also other objects and regions of high contrast.

“This data can be used to train neural networks and other algorithms to predict what people might want to look at. The basic idea is to use these predictions to center a crop around the most interesting region.”

Here are some before and after examples that show us how twitter has done a great job with the “AI” thing (and also how bad it was before..)


twitter_ml_crops (3)

Technically speaking, Neural networks that deal with saliency are usually too slow to be used for cropping massive volumes of photos in real-time, but Twitter developed some optimizations that help its system perform 10 times faster than standard methods. The result is that the AI can do intelligent cropping on all photos as soon as they’re uploaded.

This feature is now being rolled out to Twitter users across the Web and on Twitter’s iOS and Android smartphone apps.