Google Lens Enhances AR Capabilities with Moving Object Queries

Google has updated its augmented reality application, Lens, to include responses to queries about moving environments.

Over the years, we’ve expanded Search so you can ask a question in many ways — from typing a query to humming a tune.

Today, we’re introducing even more new AI-powered ways to search what you see and hear. Learn more ↓ https://t.co/MHZCqo1ujX

— Google (@Google) October 3, 2024

Users on Android and iOS can record videos and ask questions about objects of interest. Lens employs the AI model Gemini to understand context. 

“Suppose you want to learn more about an interesting fish. Lens will prepare an overview explaining why they swim in circles and provide additional resources and useful information,” noted Lou Wang, Product Director at the company.

The experimental feature is available in English. To access it, users must register for the Google Search Labs program and activate “AI Overviews and more.” 

Artificial intelligence is used to determine which objects in the frame are most interesting, significant, and relevant to the given question. 

In addition to video, information can be searched using images. To do this, users need to take a picture and ask a question verbally. 

Lens has also been enhanced in the area of e-commerce. The application can recognize a product in a photo and provide information about it: price, brand, reviews, and stock availability.

“Suppose you see a backpack you like. You can use Lens to identify this product and instantly see details that interest you,” noted Wang. 

Meanwhile, a project leader for video generation Sora has left OpenAI to join DeepMind, Google’s AI division. 

I will be joining @GoogleDeepMind to work on video generation and world simulators! Can’t wait to collaborate with such a talented team.

I had an amazing two years at OpenAI making Sora. Thank you to all the passionate and kind people I worked with. Excited for the next chapter!

— Tim Brooks (@_tim_brooks) October 3, 2024

In recent months, several key employees have departed from the developer of ChatGPT. This comes amid reports of Sam Altman’s desire to change the company’s corporate structure.

Reports have emerged about Google’s work on a “thinking” artificial intelligence.

In mid-September, AI startup OpenAI unveiled a new large language model, o1, trained using reinforcement learning for complex reasoning tasks.