Skip to content

On the same hotel tasks, the average nightly price an AI model booked ranged from $247 to $393, depending on the model

Strategy review & optimization


$146

spread in mean booked nightly price across models on the same 3,600 binary hotel tasks

Reader Note:
Mean booked nightly price per model on an identical task set built from 179 real NYC hotels. GPT-5.4 booked the cheapest on average and Phi-4 Mini the most expensive. The tasks use synthetic prompts and money, so this is not real consumer spend.
Evidence:
Experimental
Vendor interest:
None
Sample:
28 LLMs from 8 providers tested on 3,600 hotel choice tasks built from 179 real New York City hotel properties; each task was scored in both original and swapped orderings.

Good for

Backing budget and pricing decisions in similar situations.

Not for

  • Situations far from the one tested, without checking again.
  • This was measured in one market. It may not hold where you are.
  • The source never says when this data was collected. It could be considerably older than the publication date suggests.
  • The two groups were not split at random — they picked their own side. The difference may have been there before anything was measured.

Coming soon

This feature isn’t available yet.