xAI Unveils First Multimodal Version of Grok-1.5V

Elon Musk’s company xAI has unveiled a new version of its chatbot, Grok, capable of processing requests in various formats.

The presentation came just weeks after the release of the previous version.

“Grok-1.5V competes with existing multimodal models in several areas: from interdisciplinary reasoning to understanding scientific diagrams, graphs, and screenshots,” the blog states.

The developers provided several examples in the press release demonstrating the chatbot’s new capabilities:

• converting a flowchart sketch into Python code;
• generating a bedtime story from a child’s drawing;
• explaining memes;
• converting a table into CSV file format.

After testing counterparts like GPT-4V, Claude 3Sonnet, Claude 3 Opus, and Gemini Pro 1.5, xAI claims its multimodal model leads in many parameters.

Company representatives emphasized that Grok-1.5V surpasses its competitors in the RealWorldQA benchmark—a new metric designed to assess real-world spatial understanding.

To pass the test, the AI model was trained on over 700 images, each accompanied by a question and answer for each element. xAI has made RealWorldQA publicly available under a Creative Commons license.

Grok-1.5V was launched less than a month after xAI released the model’s open-source code.

According to the developers, “significant” updates to the chatbot’s ability to understand and generate multimodal signals are expected in the coming months.

Early testers and current users will gain access to Grok-1.5V shortly.

Back in December 2023, xAI representatives notified the SEC of plans to raise $1 billion through a private sale of equity securities.