LESSON 01
WHAT THE MACHINE
IS ACTUALLY DOING.
You do not need the math. You do need an accurate mental model, because almost every expensive mistake owners make with these tools comes from believing they are something they are not.
It predicts language. It does not look things up.
A large language model is a system that has been trained on an enormous amount of written text and has learned, in extraordinary detail, which words tend to follow which other words in which situations. When you ask it something, it produces a response one piece at a time, each piece chosen because it fits what came before.
That is the whole mechanism, and it is worth sitting with for a moment, because the implication is not obvious. There is no lookup step. When you ask a question, the model is not consulting a stored record of the answer and reporting back. It is generating text that is shaped like a correct answer, using patterns learned from text where the answers usually were correct.
Most of the time that produces something accurate, because accurate writing is what most of the training text contained. But accuracy is a side effect of the process, not the goal of it. The goal is fit. Once you understand that, the tool's behavior stops being mysterious and starts being predictable.
Why it is confidently wrong sometimes
People call this hallucination. The word makes it sound like a malfunction, as though the system briefly broke. It did not. It did exactly what it always does.
Ask for the name of a specific county ordinance, a case citation, a statistic, or a source, and the model will produce something that has the right shape. A plausible ordinance number. A citation formatted correctly. A figure with a reasonable number of digits. The shape is what it learned. Whether that particular item exists is a separate question the mechanism never asked.
Here is the part that costs people money: the confidence of the output carries no information about its accuracy. In a person, hedging and hesitation are useful signals. You have spent your whole life reading them. With a model, the invented answer and the correct answer arrive in the same steady, well-organized prose. Your instinct for detecting uncertainty is not just weakened here. It is actively misleading.
Treat fluency as a formatting feature. It tells you nothing about truth.
This is one piece of a bigger picture; Ben Lovro's main site has the rest.
What 'context' means, and why it is the whole game
Context is everything the model can see at the moment it answers. Your question, the conversation so far, and any document or data you pasted or attached. That is its working memory for this response.
Two consequences follow, and both of them are practical.
First, information you did not supply is information it does not have. It does not know your market, your buy box, your pricing, your team, your last conversation with a seller, or that your contractor in Irmo is already three weeks behind. If you ask for advice without supplying those facts, it will produce generic advice, because generic is the only thing available.
Second, context has a limit, and conversations degrade as they get long. Twenty exchanges deep, earlier detail gets crowded out, and the model starts contradicting things it said at the top. The fix is unglamorous: for a new task, start a new conversation and restate the facts that matter.
Most bad output is a context problem rather than a model problem. The system answered the question you actually asked, with the information you actually gave it.
A reasoning-and-language tool, not a database
The single most useful reframe is this one. Sort every task you might hand it into two buckets.
| Plays to its strengths | Plays to its weaknesses |
|---|---|
| Summarize this call transcript | What is the current interest rate |
| Rewrite this email to be shorter and warmer | What did this property sell for |
| What am I not considering about this decision | What does South Carolina law require here |
| Turn these rough notes into a scope of work | Recite this statute accurately |
| Explain this document in plain English | Calculate this and I will not check it |
The left column has one thing in common: you supplied the material. The model is transforming, organizing, condensing or explaining information that is already in front of it. That is language work, and it is where these systems are genuinely strong.
The right column asks it to retrieve a specific fact from memory it does not have in the way you imagine. Some tools now search the web and cite sources, which helps considerably. It does not eliminate the problem, because the summary of what it found is still generated text, and you still have to open the source.
What to do with this on Monday
Three habits follow directly, and they are the difference between an owner who gets value from this and one who eventually gets burned.
- Bring the material. Paste the transcript, the notes, the document. Do not ask it to remember something you never told it.
- Verify anything specific. Names, numbers, dates, citations, rules, addresses. If it is a fact with a source, go look at the source.
- Read as an editor. You are looking for the error, not admiring the writing. The output is a first draft produced by a capable assistant who is occasionally and undetectably wrong.
None of that is difficult. It is just a different posture than the one most people bring, which is to ask a question and accept the answer because it sounded like it knew.
Frequently asked
Questions people actually ask
Does the AI know things, or is it guessing?
Neither word fits cleanly. It has absorbed patterns from an enormous body of text, which lets it produce accurate information on well-covered subjects far more often than chance. But it has no way to check itself, and no sense of the difference between recalling something and constructing something plausible.
Why does it get math wrong?
Because arithmetic is not a language pattern. It is a procedure with one right answer, and a system optimized for what text usually looks like is a poor fit for it. Some tools now run a calculator behind the scenes, which is a real improvement. Check anything that ends up in an offer regardless.
Does it remember our previous conversations?
Depends on the product, and you should find out which yours does. Some carry a memory across sessions, some do not. Even where memory exists it is partial. Never assume it retained a detail that matters.
Will it learn my business over time?
Not in the way people picture. It does not quietly improve at your specific work between sessions. What actually produces that effect is you building a reusable set of context and prompts you feed it each time. That is a library you maintain, not a skill it develops.
Is a bigger or newer model always better?
Newer models are generally more capable and still make the same category of error. Upgrading changes how often it is wrong. It does not change the fact that you have to check the things that matter.
Make your next move
A year from now, what will you be glad you started today?
You don't need another promise that everything will be easy. You need something useful to learn — and a next step you're willing to take.