Reddit's CEO, Steve Huffman, is taking a strong stance against tech giants like Microsoft, Anthropic, and Perplexity for scraping Reddit's data without permission. After securing deals with Google and OpenAI, Huffman insists that other companies need to pay if they want to use Reddit's data.
In a recent interview, Huffman highlighted the challenges Reddit faces in blocking these companies. "It has been a real pain in the ass to block these companies," he said, specifically calling out Microsoft for using Reddit's data to train its AI and summarizing content in Bing results without informing Reddit. Reddit's recent updates to its robots.txt file now block web crawlers without agreements, leading to Reddit results being visible only on Google, not Bing.
Huffman criticized Microsoft AI CEO Mustafa Suleyman's comment that public internet data is "freeware." He argued that the traditional value exchange from search engines has changed, with search, summarization, and training merging. This shift muddles the value exchange of crawling for traffic.
Reddit's move aligns with traditional media publishers seeking payment for content feeding generative AI. Anthropic and Microsoft responded to the changes, with Anthropic stating they haven't added Reddit URLs since mid-May, and Microsoft affirming respect for blocking web crawling.
As the landscape evolves, Reddit is pushing for fair compensation, setting a precedent for other platforms and content creators.