# How to check its work It writes wrong answers in the same confident tone as right ones. Three ways to tell them apart. **Time:** 10 minutes **What you'll have when you're done:** Three checks you can run on any answer, and a way to decide how much checking a given piece of work actually needs. **Before you start:** - Any AI chat tool you're allowed to use - An answer it gave you recently that you accepted without checking ---The first guide in this track made the point. The tone is identical whether it's right or wrong. There's no wobble, no hedge, no tell. That's worth sitting with, because every instinct you have for spotting a bluffing human is useless here. People who are unsure sound unsure. This doesn't. So checking can't be a feeling. It has to be a procedure. Here are three, cheapest first. --- ## The question that does most of the work Before the three methods, one habit worth more than any of them: ``` Which part of what I gave you says that? ``` Ask it about any specific claim. A number, a name, a date, a deadline, a quote. If the claim came from your source, you get a pointer to the line. If it didn't, you get one of two things: a retraction, or a confident restatement with no actual location in it. Both tell you what you needed. This works because it changes the task. "Summarise this" invites it to produce something summary-shaped. "Show me where this came from" can only be answered by the document or not at all. --- ## Check 1: trace every specific back to a source The fastest sort you can do on any answer is to split it in two. | Came from you | Came from it | |---|---| | Facts in the document you pasted | Anything supplied from memory | | Numbers you provided | Numbers it calculated or recalled | | Names, dates, quotes in your source | Names, dates, quotes it produced | | Your constraints, restated | Context it added for you | Everything in the right column is unverified. Not wrong. Unverified. That's a different claim and a more useful one. In practice this takes about a minute. Read the answer and underline every specific. Then ask, for each one, whether you supplied it. What's left underlined is your checking list. Most of the time that list is short, which is the point. You're not redoing the work. You're checking three things instead of twenty. --- ## Check 2: ask again, cold Invented details are unstable. Real ones aren't. Open a **new conversation.** This part matters. Ask again in the same thread and it will agree with itself, because your earlier exchange is sitting right there shaping the answer. Agreement inside one conversation tells you nothing at all. In the fresh window, ask the same question in different words. Then compare the specifics. - **The specifics match.** Good sign. Not proof, but the answer is at least stable. - **The specifics drift.** A number moves, a name changes, a date shifts by a year. That detail was generated rather than recalled, and you've found it. This is most useful on the things you'd struggle to check directly. Which release added a feature, whether a rule has an exception, what a limit actually is. It costs about ninety seconds and it catches the confident inventions that reading alone won't. --- ## Check 3: spot-check what's cheapest to verify You can't verify everything. So verify the parts where being wrong is cheapest to detect, and treat the result as a signal about the rest. Good spot-checks, in order of speed: - **Search for a quote.** If it quoted your document, use ctrl-F on the actual words. A quote that isn't findable is not a quote. - **Add up the numbers.** If it produced a table with a total, check the total. Arithmetic is where carelessness shows first. - **Count the things.** You asked for five and it says "the five main points". Count them. - **Follow one link.** If it gave you sources, open one. Not all of them. One. Here's the useful part. A spot-check isn't only about the thing you checked. If the quote is fabricated or the total is wrong, that tells you the whole answer was produced without care, and the parts you didn't check deserve the same suspicion. One failed spot-check should send you back to the top, not only to that line. --- ## How much checking is enough Not everything needs all three. Match the effort to what happens if it's wrong. | If it's wrong, then | Do this | |---|---| | You notice immediately and fix it | Read it. That's the check | | Someone else sees it before you do | Trace the specifics, check 1 | | It goes into a document that gets approved | Checks 1 and 3 | | A decision gets made on it | All three, and cite your sources yourself | | It touches money, people, legal, or medical | Check it the way you would if a stranger had written it, because one did | The last row is the honest position on all of this. The output arrived in your voice, in your document, in a format you recognise. None of that means it came from anywhere. --- ## When it breaks **It insists, confidently, that it's right.** Confidence is not evidence, and pushing back harder tends to produce a more confident version rather than a more accurate one. Go to check 2 instead. A fresh conversation settles it faster than an argument does. **It apologises and changes the answer immediately.** Equally uninformative. It will often fold the moment you express doubt, whether or not it was wrong. A retraction under pressure is not a correction, so verify the new answer the way you'd have verified the old one. **It gives you a source that doesn't exist.** Fabricated citations look completely ordinary: real-sounding author, plausible title, well-formed link. This is why check 3 says open one. If a source doesn't resolve, treat every other source in that answer as unchecked. **Both conversations agree and both are wrong.** It happens, mostly on things widely repeated and widely wrong. Consistency means stable, not true. For anything that matters, one check has to be against the world rather than against the model. **It's right about the facts and wrong about what they mean.** The hardest one to catch, and no procedure here finds it. That's your job, and it's the part of the work that stays yours. --- ## What to take from this You don't need to check everything. You need to know which parts you haven't checked. The failure mode isn't trusting it. It's losing track of which sentences came from your document and which arrived from nowhere. Once those are separated, the rest is ordinary work.

Next