What we won't pretend AI can do.

Every AI vendor's marketing implies the model can do more than it can. That includes ours, honestly — every "AI assistant that helps you write, code, and think" tagline glosses over the failure modes.

This post is the specific list of things Claude Sonnet (the model LADLE runs on) is bad at, from the perspective of a vendor that would financially benefit from you not knowing them.

**Arithmetic and math beyond the trivial.** Sonnet handles addition and small multiplication reliably. It fails on multi-step math with confident-sounding wrong answers. If your task involves numbers where precision matters, either use extended thinking (which helps meaningfully) or double-check the arithmetic in a calculator. Do not paste a spreadsheet and ask "what's the total" without verifying.

**Citations and references.** Sonnet will produce plausibly-formatted citations that don't exist. This includes: fake academic papers, fake court cases, fake URLs, fake book titles. Web search (when enabled) reduces this to near-zero for anything the search finds. Without search, treat every specific reference as unverified until you check.

**Recent events.** The training data has a cutoff. Anything after that cutoff, the model doesn't know unless web search is on. If you ask about something recent and search is off, you'll get old information or a plausible guess.

**Multi-step reasoning with tight verification requirements.** Complex logic puzzles, multi-step math proofs, long agentic workflows where each step depends on the previous — Sonnet is competent but not reliable. If the answer needs to be right, verify each step. Extended thinking helps but doesn't eliminate.

**Long-context recall of specific facts.** With a 200K context window, Sonnet CAN see everything you gave it. It doesn't always ATTEND to the middle of long inputs as strongly as the beginning and end (the "lost in the middle" phenomenon that affects every LLM). Put critical facts near the top or bottom, or ask about them specifically.

**Following complex multi-step instructions perfectly.** Give Sonnet 15 rules to apply to a document. It will get 12-14 right and quietly ignore or misinterpret 1-3. Verify against your rules; don't trust the "here's the output" reply blindly.

**Producing consistent output at high volume.** If you ask for "20 variations of X," you'll get 20 things, but they'll cluster in structure and phrasing. LLM diversity is narrower than the count suggests. If you need actual variety, generate in smaller batches with different prompts.

**Legal advice on your specific situation.** LLM legal analysis is confidently wrong 30-40% of the time on specific factual scenarios. Use it for concepts and drafts; get actual lawyers for actual advice.

**Medical diagnosis or treatment recommendations.** Same failure mode. Sonnet will produce a confident-sounding differential that's a poor substitute for a doctor who has your history. Don't rely on it for anything material.

**Handling ambiguity gracefully.** Faced with an ambiguous question, Sonnet often picks an interpretation and runs with it rather than asking. The pick is sometimes right, sometimes wrong. Being explicit in your prompts reduces this.

**Generating text that meets specific length constraints.** "Write exactly 100 words" is not reliably produced. You'll get 80-140. If precise length matters, generate and trim.

**Judging its own confidence.** Sonnet doesn't reliably know when it's guessing. It says "I'm confident" and "I'm not sure" but these are more style choices than calibrated probability. Verify anything the confidence-language suggests you should trust.

**What we're NOT saying:**

We're not saying LADLE is bad. It's a genuinely useful daily assistant for a wide range of tasks. It's the failure modes above that determine when it's the right tool and when it isn't.

We're not saying "AI is overhyped" as a broad claim. Some of the hype IS overhype; some is real. The right response is calibrated use, not blanket cynicism.

We're not saying "wait for GPT-5 or Claude 5." The successor models will be better on some of these axes and still fail on others (probably subtler ones). The list gets shorter over time; it never becomes empty.

**Why we're publishing this:**

Because a product's honesty about its limits is a form of respect for its users. If we didn't tell you, someone would eventually publish the list for us, and the reveal would be worse than the transparency.

Use LADLE for what it's good at. Verify what needs verification. Get humans for what humans do better. That's the real user manual.