You ask an AI to explain a hard idea. The answer is tidy, friendly and makes total sense. You nod along. You feel like you have got it.
Then the exam question arrives, and you cannot start.
A student in r/studytips described this: the AI "gives answer too fast with so-easy-go explanation that doesn't reach the actual understanding." Another wrote simply, "My test scores actually got worse." These are not isolated complaints, and there are good reasons behind them.
Reason 1: Fluency is not understanding
When text is clear and well organized, our brains read it as easy, and "easy" is easily mistaken for "learned." Psychologists call this the fluency effect. A great explainer, human or AI, can make you feel you understood before you can do anything with it.
Reason 2: You did none of the work
When a teacher explains something, you still have to catch it, connect it, and apply it. An AI can do all three for you: it reads your question, connects the ideas, and applies them to an example. Your part shrinks to reading.
In a randomised trial with nearly 1,000 high school maths students (PNAS), learners with an unrestricted GPT-4 helper did 48% better on practice problems but 17% worse on an unaided exam than peers who had no AI. The AI had been doing the thinking during practice.
Reason 3: Exams test transfer
Exams rarely ask the same thing the explanation answered. They ask a variation. That requires you to have built a flexible mental model, not just recognized a demonstration. Explanations show you the model. Only practice builds it.
Reason 4: Nothing flagged the gap
A human tutor notices when you say "yes, I get it" too quickly. Teachers online make this point: an AI "can't tell when a student is masking 'yup, I get it' when you can see they don't." A chatbot takes your "makes sense" at face value and moves on.
How to check whether you really understand
Try these before you trust the feeling:
- The blank page test. Close the chat and write the explanation from memory.
- The new example test. Make up an example the AI has not shown and solve it.
- The teach-back test. Explain it aloud to someone, or to the AI, and ask it to find mistakes.
- The delay test. Try again tomorrow. What survives is what you learned.
If you fail any of these, that is useful. It shows exactly where to work.
A worked example: understanding compound interest
Suppose you ask an AI to explain compound interest. You get a clear paragraph and a table, and it all makes sense.
Now try the tests:
- Blank page: write the formula and what each part means, with no notes. You may find you remember the idea but not the formula, or the reverse.
- New example: you invest 1,000 at 5% for three years. Work it out by hand, then check against the AI's method. Then change the rate and do it again.
- Teach-back: explain to a friend why the growth in year three is bigger than in year one.
If you can do all three, you have understood it. If the second one fails, you have found exactly what to practise, which the smooth explanation could never have shown you.
Why "I'll remember this" is unreliable
We are poor judges of our own learning. Right after reading, everything is fresh and it feels as if it will stay. A day later, much of it has gone. That gap between how well you feel you know something now and how well you will know it later is why delayed testing is so useful. It shows you the real picture, not the flattering one.
For teachers: designing around it
If you set AI-assisted tasks, build in a moment where students must produce something without the AI: a short in-class explanation, a few unaided questions, or a comparison of their attempt with the AI's answer. It does not have to be graded. What matters is that students find out for themselves whether the understanding is theirs.
A short habit to adopt
After any explanation, whether from AI, a video or a teacher, do one thing before moving on: close it and write one sentence saying what the main idea is and one example of it. If you cannot, you are not finished yet.
How to use AI explanations better
- Ask for a hint, not the whole explanation. "Give me the first step only."
- Answer before you read. Write your prediction first, then compare.
- Ask for a worked example with a step blanked out. Fill in the missing step yourself.
- Then ask for a harder variation. Explanations are the start of learning, not the end.
- Vary the setting. Practise with the same concept in a different context.
What a tutor does differently
A good tutor explains less than you would expect and asks more. It waits for your attempt, responds to the specific mistake, and gives just enough help to keep you moving. It also checks understanding with a fresh problem, not the same one.
Tutor AI follows that pattern. It starts with a question, gives hints in steps, and finishes with a check on something new. Explanations are still there when you need them, they just come after you have tried.
Takeaway
Feeling that you understand is a weak signal. Being able to produce the answer, from memory, on a new problem, is the strong one. Design your study around that second signal.