How to find information on Google Search using your phone camera or a screenshot

Google can now understand what you’re looking at and help you discover visually similar images and related content, gathering results from all over the internet.

Through Google Lens, the newly improved search box can now process what you see and using a photo, your camera, a screenshot or almost any image, generate helpful results that identify, translate and even help you shop for the item on the image.

Optimised machine learning models have enabled the platform to now utilize visual search and text translation in over 100 languages to complete any task in just 100 milliseconds — shorter than the blink of an eye.

“Earlier this year, we made visual search even more natural with the introduction of multisearch. You can take a picture or use a screenshot and then add text to it similar to the way you might naturally point at something and ask a question about it.

"Multisearch is available in English globally and will be coming to over 70 languages in the next few months,” said Cathy Edwards, Google Vice President and General Manager of Search.

Google Lens helps identify what you see Pulse Live Kenya

How Google Lens works

To experience this improvement on Search, find the camera icon beside the voice button and proceed to either point the camera at the object you'd like to identify or upload the screenshot, image or picture you would like to identify.

Lens compares objects in your picture to other images and ranks those images based on their similarity and relevance to the objects in the original picture.

Lens also uses its understanding of objects in your picture to find other relevant results from the web.

A second improvement feature includes a translation update on Lens which will take effect later in the year.

You can point a phone camera at a poster in another language, for example, and you will see translated text realistically overlaid onto the pictures underneath.

This process uses generative adversarial networks (also known as GAN models), which help power the technology behind the Magic Eraser on Pixel.

Google Lens currently supports English, Spanish, French, German, Italian, Portuguese, Korean, Chinese, Danish, Dutch, Hindi, Indonesian, Japanese, Polish, Russian, Swedish, Thai, Turkish, and Vietnamese.

SearchON 2022: Google announces news ways of exploring information on Search Pulse Live Kenya

Starting October 3, 2022, Google has added shortcuts right under the search bar on its Google app for iOS enabling users to shop via screenshots, translate text with their cameras, hum to search, search into a phone, solve homework with camera and identify songs by listening.

In the coming months, Google will be rolling out an even faster way to find what you need. When you begin to type in a question, Google will provide relevant content, keyword or topic options to help you craft the question.