Source – bgr.in
Twitter is now using machine learning to automatically crop picture previews to their most interesting part. The use of machine learning and neural networks for image recognition as become common with both Google and Facebook heavily deploying the technology on their platforms. Now, Twitter is also joining with its own implementation with a single aim of improving user experience.
Twitter explains that it has been working on this technology for a while now and describes the methods in a detailed blog post. Machine Learning Researcher Lucas Theis and Software Engineer Zehan Wang explain that the micro-blogging platform previously used face detection to focus the image on the most prominent face it could find. However, since all the images don’t have faces, the company found the approach didn’t work with all kinds of pictures especially objects and cats.
Twitter has now tweaked its algorithm with what it calls as cropping using saliency. The platform now crops images based on whatever is interesting and not just limited to faces. Twitter defines the most interesting part as a region having high saliency where a person is likely to look at when freely viewing the image.
“In general, people tend to pay more attention to faces, text, animals, but also other objects and regions of high contrast.”
Twitter now uses this data to train its neural networks and other algorithms to predict what people might want to look at rather than limiting it to prominent face. Engineers at Twitter note that recent advances in machine learning has made saliency prediction a lot better. The major challenge for the company being the sllow processing speed of neural networks deployed to predict saliency, making them not suitable to run in production.
However, Twitter optimized the neural network’s implementation by using two techniques that help reduce the size and hence the computational requirements. Twitter explains using a technique called knowledge distillation to train a similar network in order to imitate the slower but more powerful network. The second is a pruning technique to remove feature maps of the neural network which were costly to compute but contributed in little to performance.
In raw performance terms, Twitter claims to have made its neural networks 10x faster than a vanilla implementation. The update is in the process of being rolled out to everyone accessing the service on web, iOS and Android apps. So, next time if you see a cat prominently on your Twitter feed then it is powered by these very neural networks.