China’s artificial intelligence (AI) lab DeepSeek has unveiled an open source version of its R1 artificial intelligence model with reasoning capabilities that it claims outperforms OpenAI’s o1 AI model in some benchmarks across a number of key metrics.
Image source: Alexander Sinn / Unsplash
As reported by TechCrunch, the R1 model outperforms OpenAI o1 in benchmarks such as AIME, MATH-500 and SWE-bench Verified. These tests cover a variety of areas including math problems, programming, and overall model performance. AIME evaluates model performance against other AI models, MATH-500 is a set of math problems, and SWE-bench Verified focuses on programming.
A special feature of the R1 is its ability to self-test, which helps eliminate errors typical of other models. However, this approach requires more time to obtain a response – from several seconds to several minutes. However, such models are considered more reliable in complex fields such as mathematics, physics and various sciences.
DeepSeek said R1 contains 671 billion parameters, making it one of the largest models in the world. For the convenience of users, the company also released simplified versions of R1 with a volume of parameters from 1.5 to 70 billion. At the same time, the most compact version can work even on a regular laptop, while the full version requires more powerful hardware. However, R1 is available through the company’s API at a price that is said to be 90-95% lower than OpenAI o1.
It is noted that the R1 also has some disadvantages. Since the model was developed in China, it is subject to mandatory verification by regulators for compliance with “core socialist values.” For example, R1 refuses to answer questions about the events in Tiananmen Square or about the independence of Taiwan, which is typical of many Chinese AI models that avoid discussing sensitive political topics in the Middle Kingdom.
It is worth saying that DeepSeek became the first Chinese laboratory to enter into real competition with OpenAI o1, presenting a preliminary version of R1 back in November. However, other players followed suit, including Alibaba and Moonshot AI-owned Kimi. According to George Mason University artificial intelligence researcher Dean Ball, all this suggests that Chinese AI companies continue to be “fast followers” in the technology race. “The success of lightweight versions of DeepSeek […] shows that powerful reasoning models will become increasingly accessible and can even run on local hardware,” Ball said in his X post.