More than 14 major news publications from the top 1000 closed access to their materials for OpenAI’s SearchGPT AI search engine a week after its launch. The decision could impact the completeness and accuracy of SearchGPT’s search results, and also raises questions about OpenAI’s credibility and data practices.
Image source: Growtika / unsplash.com
About a week ago, OpenAI launched the AI search engine SearchGPT, but already 14 of the 1,000 largest news sites, including The New York Times, Wired, The New Yorker, Vogue, Vanity Fair and GQ, have already blocked its search robot. The OAI-SearchBot indexes information so that SearchGPT can return relevant results to users.
According to Originality.ai, a company that monitors such blockings, this decision was unexpected. Jon Gillham, CEO of Originality.ai, expressed bewilderment at this situation: “I don’t understand why publishers are blocking him. They need that traffic.”
OpenAI stressed that OAI-SearchBot does not collect data to train AI models such as GPT-5, and recommended that site owners allow the bot to ensure sites are visible in search results. According to OpenAI, this will help resources remain in search results and attract more traffic. However, the lack of access to some websites may make SearchGPT’s search results less comprehensive than Google’s.
Another OpenAI search bot, GPTbot, which collects data to train AI models, has already been blocked by hundreds of sites. This solution is more understandable because publishers want to get traffic from search engines, but do not want to share content for AI training. OpenAI has spent years collecting data online without permission, so mistrust of the company may be one reason for the ban.
Gillam suggests that publishers may not trust OpenAI’s claim that its new bot does not collect data to train AI models. Another reason may be to avoid a situation where new AI search engines, in order to retain users, show them a summary of the content rather than redirecting them to the original sites, which reduces traffic and revenue for publishers.
Gillam also noted that OpenAI has been actively pursuing deals with publishers this year to use their archived materials. This may look like a smart strategy for OpenAI to first build relationships with publishers through partnerships and then announce the launch of SearchGPT.
The New York Times has become the main opponent of OpenAI among publishers, filing a lawsuit against it and Microsoft. The publisher claims that the two technology companies are illegally using its materials to create competing products. As Charlie Stadtlander, a spokesman for The New York Times, said: “The Times does not permit the use of its materials for generative search or AI training without a written agreement, whether or not we block any bot from accessing our content.” “
In its complaint against OpenAI and Microsoft, The New York Times emphasized that AI search engines could potentially siphon traffic away from publishers, hurting their finances: “By providing content to The Times without permission, defendants’ tools undermine The Times’ relationship with readers and deprive them of revenue from subscriptions, licensing, advertising and affiliate programs.”
«”Defendants also use Microsoft’s Bing search engine, which copies and categorizes The Times’s online content to generate responses with verbatim excerpts and detailed summaries of The Times articles that are much longer and more detailed than those produced by traditional search engines,” the publisher’s complaint states. .
The emergence of SearchGPT and the reaction of publishers shows the complexity of the relationship between AI technologies and traditional media. Mistrust of OpenAI and a desire to protect their revenues are pushing publishers to take precautions. In the future, we will have to see how these contradictions will be resolved and what impact this will have on the availability and quality of information on the Internet.