User preferences for large language model refusals: implications for moderation and market structure

How much should AI models say “no”? A new unpublished study by London School of Economics researchers Pavel Kireyev and Maria Ana Vitorino argues that refusal behavior—when a large language model (LLM) declines to answer a prompt—doesn’t just shape safety outcomes; it also steers user choices and the competitive balance between proprietary and open-source models.

What the researchers did

Using data from LMArena, a community platform where users compare paired LLM responses and vote for their preferred answers, the authors estimate a discrete choice model that captures two simultaneous decisions: whether users opt to make “risky” prompts (more likely to trigger refusals) and which LLM they believe delivers the best response quality given the chance of a refusal.

That framework allows them to simulate how different moderation policies—stricter or looser refusal thresholds—shift user behavior and market shares across the LLM landscape.

Key findings

• Proprietary LLMs generally deliver higher-quality responses and hold larger market shares—when users expect their prompts to be answered.
• These models also tend to enforce stricter moderation and refuse more often, particularly on risky prompts.
• Open-source LLMs typically run looser moderation, refuse less, and attract users who anticipate refusals from proprietary systems—especially for borderline or sensitive queries.
• Counterintuitively, strict moderation by proprietary providers can reduce overall market concentration by leaving room for lower-quality, less restrictive open-source competitors to win the risky-prompt segment.
• If regulators mandated uniform moderation across all models, the authors suggest market concentration could increase, likely benefiting proprietary leaders and dampening competition.

Why this matters

Moderation policies are often framed as a tension between user safety and model capability. This research adds a third axis: competition. Refusals don’t just filter content—they re-route demand. When top-tier proprietary models refuse more, they create an opening that less restrictive rivals can exploit, even if those rivals lag on raw quality. In practice, that means moderation settings can be as consequential as scaling laws when it comes to market structure.

Inside the model

The discrete choice approach treats users as weighing expected response quality against the likelihood of a refusal. A risky prompt has higher refusal probability under stricter policies, so users strategically select both prompt type and model. The LMArena pairwise votes provide revealed-preference signals about which responses users prefer, conditional on whether the model answers at all. By fitting this behavior, the authors can simulate policy counterfactuals—loosening or tightening moderation, or harmonizing policies across models—and project resulting shifts in usage and market share.

The policy twist

Well-intentioned rules that harmonize moderation—say, requiring all LLMs to refuse at similar thresholds—could have an anticompetitive side effect. If proprietary leaders already dominate on quality, leveling the refusal playing field may remove the niche that allows open-source models to compete via lower refusal rates. The study’s takeaway: uniform moderation could inadvertently increase market concentration and reduce competitive pressure.

Product strategy implications

• Proprietary providers: Stricter moderation can be a deliberate positioning choice, improving safety and brand trust while ceding some risky-prompt demand to others—potentially a price worth paying if it builds long-term legitimacy.
• Open-source and challengers: Carefully managed flexibility on refusals can be a differentiator, drawing users who prioritize fewer blocks—provided safety standards remain defensible.
• Multi-model platforms: Offering model choice (and transparent moderation profiles) can match users to the right safety/utility mix, improving overall satisfaction.

Balancing safety and competition

The authors outline an “efficient frontier” of moderation policies—combinations that trade off safety (fewer risky outputs) against market concentration (more competition). The frontier perspective reframes debates that often pit safety against freedom: the goal isn’t zero refusals or maximal blocking, but a policy set that delivers robust safety without eliminating competitive dynamics that benefit users.

Caveats and context

• The study is unpublished; details may evolve with peer feedback and broader replication.
• LMArena data reflect user votes in paired comparisons, which emphasize perceived response quality; real-world deployment constraints and enterprise compliance needs may shift preferences.
• “Risky” is contextual: content categories, local regulations, and platform incentives vary by market.

The bottom line

Refusals are not just a safety lever; they’re a market lever. According to this LSE analysis, stricter moderation by proprietary leaders simultaneously elevates safety and decentralizes demand by enabling open-source competitors to specialize in higher-risk prompts. Flattening moderation across the board could reverse that pattern, concentrating power back with high-quality incumbents. For policymakers and product leaders alike, the message is clear: moderation choices shape not only what AI says, but who gets heard in the market.

Study referenced: Pavel Kireyev and Maria Ana Vitorino, “User preferences for large language model refusals: implications for moderation and market structure,” London School of Economics and Political Science (2025), unpublished.