Apple researchers have found that large language models such as ChatGPT are incapable of logical thinking and are easily confused by adding irrelevant details to the task at hand, TechCrunch reports.
The published paper, “Understanding the Limits of Mathematical Reasoning in Large Language Models,” raises questions about the logical reasoning capabilities of artificial intelligence. The study found that large language models (LLMs) can solve simple math problems, but adding irrelevant information leads to errors.
For example, the model may well solve the following problem: “Oliver picked 44 kiwis on Friday. He then picked 58 kiwis on Saturday. On Sunday he collected twice as many kiwis as on Friday. How many kiwis does Oliver have? However, if you add the phrase “On Sunday, 5 of these kiwis were slightly smaller than average,” the model will likely subtract these 5 kiwis from the total, despite the fact that the size of the kiwis does not affect their number.
Mehrdad Farajtabar, one of the study’s co-authors, explains that such errors indicate that LLMs do not understand the essence of the task and are simply reproducing patterns from the training data. “We hypothesize that this decline [in efficiency] is due to the fact that modern LLMs are incapable of true logical reasoning; instead, they try to reproduce the reasoning steps observed in their training data,” the paper states.
Another OpenAI specialist countered that correct results can be obtained using prompt engineering. However, Farajtabar noted that complex tasks may require exponentially more contextual data to neutralize distractions that, for example, a child would easily ignore.
Does this mean that LLMs cannot reason? Maybe. No one has yet given an exact answer, since there is no clear understanding of what is happening. LLMs may be “reasoning,” but in a way we don’t yet recognize or can’t control. In any case, this topic opens up exciting prospects for further research.
Meta✴ Platforms, the owner of the social network Threads, announced “long overdue improvements” for its…
Image source: Ubisoft Let us remind you that the events of Assassin’s Creed Shadows will…
NASA announced that assembly of the second lunar rocket, SLS (Space Launch System), has begun…
Co-founder and CEO of the Chinese studio Game Science, Feng Ji, hinted that some surprises…
Last Wednesday, trading volume in MicroStrategy shares exceeded that of Nvidia and Tesla. The company,…
The post-apocalyptic open-world shooter S.T.A.L.K.E.R. 2: Heart of Chornobyl from the developers from the GSC…