Reddit AI Search Favors Popular Comments, Audit Finds

Reddit’s AI search algorithm systematically prioritizes formal, upvoted commentary over personal anecdotes, according to a new preprint from researchers at the University of Illinois Urbana-Champaign. The study reveals that the platform’s artificial intelligence is significantly more likely to incorporate comments that read like objective advice rather than those rich in individual experience. This bias effectively strips first-person narratives from the final outputs, transforming intimate testimony into generalized guidance.

The research team processed 10,000 distinct questions through the AI search feature three separate times, analyzing a total of 30,000 generated answers. To trace the origins of these responses, they mapped them back across 14.68 million comments. The dataset was drawn exclusively from 20 subreddits focused on advice and support, meaning the findings specifically reflect that segment of the site. It is important to note that this paper has not yet undergone peer review.

Formality Drives Selection Odds

To understand what drives the AI’s choices, the researchers compared comments that were quoted or listed as sources against other comments within the same discussion threads that the system ignored. They discovered that formality was the strongest linguistic predictor of selection. When a comment scored one standard deviation higher in formality, it had a 49% higher odds ratio (1.488) of being selected. This metric was determined using a text classifier. Similarly, comments containing directive language such as “should” or “must” saw increased selection odds with an odds ratio of 1.070.

Conversely, comments exhibiting markers of personal experience faced lower odds of selection, with an odds ratio of 0.789. Supportive language also suffered a slight disadvantage, recording an odds ratio of 0.924. The authors define “experiential voice” through features like the use of first-person pronouns and past tense verbs. While formality correlated with higher community voting and experiential voice with lower voting, the statistical associations weakened after controlling for a comment’s score, position, and age. The final adjusted odds ratios stood at 1.213 for formality and 0.860 for experiential voice.

The authors explicitly caution against interpreting these results causally. They write, “Our study is observational and should not be interpreted causally.” This means that while formal writing is associated with higher selection rates, the study does not prove that writing formally causes a comment to be chosen.

Visibility and Positioning Advantages

Beyond language style, a comment’s existing visibility played a massive role in its inclusion. A comment’s vote ranking within its thread showed the largest association in the selection model. A one-standard-deviation increase in ranking multiplied the odds of being chosen by 2.88. The median selected comment sat at the 91st percentile for score in its thread, whereas non-selected comments averaged only the 45th percentile.

Structural factors also heavily influenced outcomes. 92% of selected comments were direct replies to the original post, even though such replies accounted for only 53% of all comments in the dataset. Furthermore, selected comments appeared much faster, with a median time of 1.2 hours after the initial post, compared to 5.9 hours for non-selected comments.

Other technical metrics also favored certain types of content. A one-standard-deviation increase in comment length resulted in an odds ratio of 1.79. Comments containing an external link saw an odds ratio of 2.25. Notably, low-quality content was rarely selected; comments with a score of zero or below comprised just 0.53% of selected comments, compared to 5.1% of all collected comments.

Erasure of First-Person Narrative

The audit found that the written answers produced by Reddit’s AI drastically reduced the presence of first-person language. In an analysis of 1,000 queries, researchers observed that words like “I” and “my” dropped from 3.3% in the quoted source comments to a mere 0.06% in the final AI-generated responses.

To contextualize this finding, the team ran the same queries through GPT-4o-mini and GPT-5 via OpenAI’s API with web search enabled. Both models used less first-person language than the original Reddit comments, but Reddit’s own responses demonstrated the most significant reduction. Across 4,932 citations analyzed, only 18 referenced Reddit across both models and search configurations. This data reflects API calls for advice queries, which differs from the specific setup used in the ChatGPT application.

Methodology and Feature Evolution

The researchers utilized a large language model to rewrite real posts from the 20 subreddits into short search queries. The sample included ten large communities, such as r/personalfinance and r/AskDocs, and ten smaller ones, including r/UKJobs and r/AusLegal. The three experimental runs were spaced approximately five hours apart to ensure that differences in results reflected the AI system’s behavior rather than changes in user activity on Reddit. When two runs pulled from the same source posts, the responses were nearly identical, confirming that variations stemmed primarily from retrieval choices.

A typical answer drew from about seven different subreddits. The subreddit where the question originated contributed 17.7% of retrieved posts in the large-community set and 15.7% in the small-community set. The authors noted that they do not view a higher share from the originating subreddit as inherently better.

The paper refers to the technology as Reddit Answers, the name it held when the feature launched in December 2024. However, Reddit’s announcement page now displays an update from May 26, 2026, stating, “Reddit Answers is now merged into Reddit search for one unified search experience.” The questions in the study were created from posts dated through July 2026, placing the runs after this integration. Currently, Reddit’s Help page describes the functionality as AI search, accessible via an Ask button in the search bar.

Implications for Authentic Voice

This audit highlights a potential conflict between Reddit’s core value proposition and its AI implementation. Reddit’s appeal for audience research often lies in users describing their direct experiences with products or problems. In these 20 communities, comments rich in such experiential markers were less likely to reach the AI answer, and when they did, their personal tone was largely erased.

Reddit CEO Steve Huffman previously told investors in February that the platform excels at answering questions where “the answer actually is multiple perspectives from lots of people.” This audit measures exactly which of those diverse perspectives make it into the AI-generated summary. Users are encouraged to review the source threads listed beneath each answer before incorporating the AI summary into any report.

The authors acknowledge that these selection patterns may not apply to other types of communities or different AI search systems. The story will be updated if a revised or peer-reviewed version of the paper alters these findings.