Apple Study Shows AI Models Don’t Think, They Just Simulate Thinking

Apple researchers have found that large language models such as ChatGPT are incapable of logical thinking and are easily confused by adding irrelevant details to the task at hand, TechCrunch reports.

Image Source: Dkoi/Unsplash

The published paper, “Understanding the Limits of Mathematical Reasoning in Large Language Models,” raises questions about the logical reasoning capabilities of artificial intelligence. The study found that large language models (LLMs) can solve simple math problems, but adding irrelevant information leads to errors.

For example, the model may well solve the following problem: “Oliver picked 44 kiwis on Friday. He then picked 58 kiwis on Saturday. On Sunday he collected twice as many kiwis as on Friday. How many kiwis does Oliver have? However, if you add the phrase “On Sunday, 5 of these kiwis were slightly smaller than average,” the model will likely subtract these 5 kiwis from the total, despite the fact that the size of the kiwis does not affect their number.

Image source: Copilot

Mehrdad Farajtabar, one of the study’s co-authors, explains that such errors indicate that LLMs do not understand the essence of the task and are simply reproducing patterns from the training data. “We hypothesize that this decline [in efficiency] is due to the fact that modern LLMs are incapable of true logical reasoning; instead, they try to reproduce the reasoning steps observed in their training data,” the paper states.

Another OpenAI specialist countered that correct results can be obtained using prompt engineering. However, Farajtabar noted that complex tasks may require exponentially more contextual data to neutralize distractions that, for example, a child would easily ignore.

Does this mean that LLMs cannot reason? Maybe. No one has yet given an exact answer, since there is no clear understanding of what is happening. LLMs may be “reasoning,” but in a way we don’t yet recognize or can’t control. In any case, this topic opens up exciting prospects for further research.

admin

Share
Published by
admin

Recent Posts

IOS 18.2 will only learn to estimate iPhone charging time by December

The next iOS 18.2 update may introduce a long-awaited feature that will allow you to…

4 mins ago

Mozilla decided to lay off a third of its employees and close its advocacy department

The Mozilla Foundation, the owner of the Mozilla Corporation subsidiary that develops the Firefox browser,…

14 mins ago

Apple fears it won’t be able to create anything better than the iPhone

Apple has warned investors that future products may never be as profitable as its iPhone…

4 hours ago

Warner Bros. will link Hogwarts Legacy 2 with the HBO Harry Potter series, and sales of the first part have reached a new peak

Variety, citing data from Warner Bros. revealed the current sales of Hogwarts Legacy, and the…

5 hours ago

Ryzen 7 9800X3D showed a significant increase in minimum FPS in games compared to Ryzen 7 9700X

The publication of the first independent reviews of the new AMD Ryzen 7 9800X3D gaming…

5 hours ago

Two heroes, the return of Umbrella and the goat man: the blogger revealed possible details of Resident Evil 9

Blogger BiohazardDeclassified, who in the past shared truthful information about the then-mysterious Resident Evil Village,…

7 hours ago