Apple Study Shows AI Models Don’t Think, They Just Simulate Thinking

Apple researchers have found that large language models such as ChatGPT are incapable of logical thinking and are easily confused by adding irrelevant details to the task at hand, TechCrunch reports.

Image Source: Dkoi/Unsplash

The published paper, “Understanding the Limits of Mathematical Reasoning in Large Language Models,” raises questions about the logical reasoning capabilities of artificial intelligence. The study found that large language models (LLMs) can solve simple math problems, but adding irrelevant information leads to errors.

For example, the model may well solve the following problem: “Oliver picked 44 kiwis on Friday. He then picked 58 kiwis on Saturday. On Sunday he collected twice as many kiwis as on Friday. How many kiwis does Oliver have? However, if you add the phrase “On Sunday, 5 of these kiwis were slightly smaller than average,” the model will likely subtract these 5 kiwis from the total, despite the fact that the size of the kiwis does not affect their number.

Image source: Copilot

Mehrdad Farajtabar, one of the study’s co-authors, explains that such errors indicate that LLMs do not understand the essence of the task and are simply reproducing patterns from the training data. “We hypothesize that this decline [in efficiency] is due to the fact that modern LLMs are incapable of true logical reasoning; instead, they try to reproduce the reasoning steps observed in their training data,” the paper states.

Another OpenAI specialist countered that correct results can be obtained using prompt engineering. However, Farajtabar noted that complex tasks may require exponentially more contextual data to neutralize distractions that, for example, a child would easily ignore.

Does this mean that LLMs cannot reason? Maybe. No one has yet given an exact answer, since there is no clear understanding of what is happening. LLMs may be “reasoning,” but in a way we don’t yet recognize or can’t control. In any case, this topic opens up exciting prospects for further research.

admin

Share
Published by
admin

Recent Posts

Threads gets ‘long overdue improvements’ to search and trends

Meta✴ Platforms, the owner of the social network Threads, announced “long overdue improvements” for its…

2 minutes ago

Ubisoft spoke about the capabilities and innovations of stealth mechanics in Assassin’s Creed Shadows – new gameplay

Image source: Ubisoft Let us remind you that the events of Assassin’s Creed Shadows will…

52 minutes ago

Assembly of the second NASA SLS rocket has started – in a year it will send people on a flight around the Moon

NASA announced that assembly of the second lunar rocket, SLS (Space Launch System), has begun…

52 minutes ago

The creators of Black Myth: Wukong will surprise players before the end of the year – teaser from the head of Game Science

Co-founder and CEO of the Chinese studio Game Science, Feng Ji, hinted that some surprises…

3 hours ago

Nvidia stock is no longer the best performer – MicroStrategy soars 500% in a year thanks to Bitcoin

Last Wednesday, trading volume in MicroStrategy shares exceeded that of Nvidia and Tesla. The company,…

3 hours ago

Tired of waiting: sales of S.T.A.L.K.E.R. 2: Heart of Chornobyl exceeded one million copies within two days of release

The post-apocalyptic open-world shooter S.T.A.L.K.E.R. 2: Heart of Chornobyl from the developers from the GSC…

4 hours ago