Why Short AI Answers Can Still Be Hard to Read
Every word in a one-sentence AI code explanation is familiar, yet you read it twice. What did researchers find about the way AI packs ordinary English together?
- A study of generated English finds that several constructions associated with comprehension difficulty appear more often in AI output than in human writing or rephrased versions.
- Training a separate small model to favor shorter responses changed its grammatical preferences while allowing it to keep answering correctly.
- The findings concern English and vary by model and task. They do not establish an AI-text detector or prove how every commercial model acquired its style.
Compression can leave the reader to reconstruct the relationships
A short code explanation can still require a second reading. A string of nouns may save words while leaving their relationships implicit. An unusual combination of familiar words can demand more interpretation than a longer, ordinary phrase.
That is the problem examined in a paper posted on October 21 by Thomas Hikaru Clark, Matias B. Martin and Kyle Mahowald. The researchers investigate difficult linguistic features in generated English, then test whether pressure to pack meaning into fewer tokens could help produce such preferences.
Consider an illustrative phrase, customer request review process. Expanding it to how the team reviews requests from customers adds words but identifies an actor and the relationship between the requests and customers. This is an explanatory example, not a measured item from the study. The distinction is between reducing output length and reducing the reader’s interpretive work.
The comparison went beyond spotting unusual vocabulary
The collection covered 14 models in total, with different participation by task: 12 in chat, nine in coding and 14 in essays. The researchers compared the outputs with roughly matched human material, including subtitles, StackExchange and student essays.
They also translated generated sentences into Japanese and back into English. This supplied paired alternatives intended to preserve meaning while changing the wording. Comparing those pairs helps separate the content of an answer from a model’s preferred way of expressing it.
| Feature examined | Work it can leave to the reader |
|---|---|
| Unusual word combinations | Interpret a combination that is uncommon in the reference material. |
| Stacks of three or more modifying nouns | Determine which noun modifies which. |
| Long grammatical dependencies | Keep earlier material available while connecting related words. |
| Inanimate subjects with typically animate verbs | Recover a metaphorical or implied meaning from context. |
All nine frontier or near-frontier models used more low-frequency word combinations and the studied inanimate-subject constructions than human writing. Other effects varied by task and model. Several changes under round-trip translation were smaller in essays than in chat and coding.
“Unattested” combinations were absent from a reference corpus; the researchers were not searching the models’ entire training sets. Likewise, the study counted features linked by prior research to processing difficulty. It did not run a new human reading-time test for every generated answer.
A correct answer can acquire a less humanlike grammatical preference
To investigate a possible mechanism, the team separately trained Qwen2.5-1.5B-Instruct to turn structured event descriptions into English. A length penalty made longer outputs less rewarding. The task allowed alternative constructions that expressed the same event with different token counts.
As the penalty increased, the model increasingly favored the shorter construction. Some of that preference carried over to similar constructions it had not seen in training. Transfer to more distant grammatical alternatives was limited.
Crucially, either wording could still be a correct task answer. The model could therefore succeed while adopting preferences that differed from how people choose expressions in context. The authors present this as a proof of concept for a mechanism, rather than a causal reconstruction of the training of every frontier model.
Judge clarity separately from brevity
The analysis is restricted to English, a subset of models and a limited set of starting prompts. Its results cannot simply be transferred to Korean or another language. Nor does any one noun stack establish that an author used AI: people use these constructions too.
For someone reviewing an AI-generated technical explanation, the practical implication is to assess the relationships the reader must recover. A shorter sentence is useful when the action and its participants remain clear. If cutting words obscures those relationships, restoring a few words may better serve the explanation.
Passing a coding task and explaining the resulting change are separate achievements. The study gives a concrete reason to evaluate both, even when an answer looks concise and grammatically fluent.