Nvidia has released an open-source multimodal AI model, and it’s as good as GPT-4

Nvidia introduced a new family of large multimodal language models, NVLM 1.0, including the NVLM-D-72B, trained on 72 billion parameters. The models demonstrate high performance in a wide range of tasks, such as computer vision, code generation, image analysis, mathematical problem solving and text generation. It looks like industry leaders led by OpenAI and Google will have to make room.

Image source: freepik.com

«”We’re introducing NVLM 1.0, a family of cutting-edge multimodal large language models that achieve state-of-the-art results in vision and language tasks, rivaling leading proprietary models (such as GPT-4o) and open-source models,” Nvidia said. According to them, open source gives researchers and developers unprecedented access to cutting-edge technologies.

The flagship model NVLM-D-72B demonstrates adaptability when processing complex visual and text input data. Researchers highlight the model’s ability to interpret memes, analyze images, and solve mathematical problems step by step. The developers also noted that the NVLM-D-72B improves its performance on word problems after multimodal training, unlike most similar models.

The NVLM project also introduces innovative architectural solutions, including a hybrid approach that combines different multimodal processing techniques. Third-party researchers say the NVLM-D-72B is “on par with the Llama 3.1 405B in math, coding, and vision.”

The release of Nvidia NVLM 1.0 marks a turning point in AI development. By open-sourcing a model that competes with proprietary giants, Nvidia isn’t just sharing code—it’s challenging the very fabric of the AI ​​industry. Thanks to Nvidia, many smaller organizations and independent researchers will be able to make greater contributions to the development of AI, which could usher in an era of unprecedented collaboration and innovation in AI.

This move could set off a chain reaction that would force other AI leaders to open up their research as well, potentially speeding up AI progress across the board.

It should be noted that the release of NVLM 1.0 is not without risks. As powerful AI becomes more accessible, so too do concerns about its misuse and possible ethical implications. The AI ​​community is already faced with the need to use new technologies responsibly.

One thing is certain: Nvidia’s AI policies will affect the entire industry. The only question is how radical its impact will be, and whether competitors can adapt quickly enough to thrive in this new world of open AI.