arXiv (2026) — PriceBench for LLM hotel booking preferences

Summary:

This academic paper looks at what AI booking agents choose when several hotel options all fit the rules: the cheaper one, the better-reviewed one, or a certain brand. It was written by Pavel Kireyev at the London School of Economics and Political Science and published on arXiv. The goal is a benchmark, PriceBench, that measures an LLM’s hidden buying preferences from its hotel choices.

The author built 3,600 hotel choice tasks from 179 real New York City hotels and tested 28 LLMs from 8 providers. Each task showed 2 or 3 options and was repeated with the order swapped. A logit choice model then estimated how each LLM reacts to price, review score, star rating, brand and other features. The evidence is the models’ observed choices, not surveys.

Key findings:
• Stronger models were more consistent, but not more alike in what they preferred.
• 5 of the 28 models mostly picked the first-listed option, so whoever controls the ranking could control most of their bookings.
• Price sensitivity varied by more than 10x across the other 23 models.
• On the same tasks, average booked nightly price ranged from $247 to $393 depending on the model.
• Some models acted as bargain hunters; others paid much more for better reviews, and this split even close models from the same family.
• Brand preferences differed widely and did not line up by provider.
• Results held across prompt formats, sampled decoding and larger choice sets, though exact dollar values shifted with format.

Margin notes:
This is a narrow lab-style benchmark: hotel bookings only, one city, mostly single-turn prompts without tools. The tasks are synthetic, built from real listings, so they don’t reflect live market behaviour or real spending. Some robustness tests covered only 6 of the 28 models, and the comparison with humans uses numbers from past studies rather than human choices on the same tasks.