The Step-by-Step Thinking (Chain-of-Thought) Technique

What is chain of thought and where did it come from?

A chain-of-thought prompt is a method letting the model generate the intermediate stages of a complex task. In the classic variant, the prompt includes not only a question and an answer but also a few examples containing correct solution steps. The model imitates that stepping structure on the new question.

The technique became widely known through the 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." In the NeurIPS paper, the authors showed that intermediate-reasoning examples improved results on arithmetic and symbolic reasoning tasks in sufficiently large models. This is not a verdict that it "always works on every model and every task"; it's an experimental result tied to the models, examples and measured tasks.

In later models, thinking behaviour can be built into the product itself. That's why the old sentence "Let's think step by step" may create no special advantage on every modern model. The prompt's goal and acceptance criteria must still be written explicitly.

A hidden chain of thought and a verifiable explanation are not the same

The model's internal computation, the reasoning shown to the user and real evidence are three separate things. The internal process may not be visible in every product. The visible explanation can be tidy text generated after the fact. Evidence arises only from input data, sources, formulas, tool results and repeated checks.

In its explanation of reasoning models, OpenAI stated that the o1 family does not show the raw chain of thought to users, instead presenting a model-generated summary. That decision applies to a specific product; other models and open-weight systems can behave differently. The practical conclusion doesn't change: don't treat the visible text as "an X-ray of the model's brain."

In which tasks is chain of thought useful?

A chain of thought gains meaning only in tasks with several dependent stages. If reaching the answer feeds one step's result into the next, splitting the work is useful. If the task is a single conversion or a simple lookup, intermediate explanation can create extra cost and text.

• Maths problems: the data, formula, calculation and final unit are separated.
• Rule-based classification: each condition is checked in order and the first matching rule is recorded.
• Planning: dependencies, resources, risks and sequence become visible.
• Document comparison: the same clause is found separately in two sources and the contradiction is shown.
• Code diagnostics: symptoms, hypotheses, tests and observations are chained together.

For translation, simple summaries, headline variants and direct data extraction, try a short prompt first. Google's current prompt design documentation for the mobile Gemini Nano recommends splitting multi-step conditional work into more focused model calls instead of one long answer when it gets difficult. Sometimes the best "chain of thought" lives not inside the prompt but in the system's workflow.

6 chain-of-thought prompts without demanding hidden thinking

These examples do not require an internal monologue. They separate the intermediate results to verify from the final answer. Replace the brackets with your own information.

1. Mathematical calculation

Solve the problem. In the answer show only:
1) the given numbers and units;
2) the formula used;
3) the results of intermediate calculations;
4) the final answer and its unit.
If any input is missing, don't calculate; write which data is missing.

2. Comparing two proposals

Compare proposals A and B by price, scope, limitations and risk.
For each criterion, write the fact from the document first, then a short comment.
Don't invent an advantage not in the document.
Don't declare a winner at the end; show under which conditions each fits.

3. Routing a customer request

Check the request in this order: security risk, account access,
payment, technical error, general question. Pick the first matching category.
In the output give the category, one piece of evidence from the message and the next step.
If two categories fit equally, require human review.

4. Fact audit of an article

Extract the verifiable claims in the text.
For each claim, find the supporting passage in the given official source.
Mark the result as "supported / partial / not found".
Only then propose fixes; if there's no source, don't add facts.

5. Diagnosing a code error

From the symptoms and logs, write at most three hypotheses.
For each hypothesis give one safe test that would confirm it.
Don't rewrite the code before the test results arrive.
Final answer: hypothesis, test, observation, minimal fix.

6. Sequencing a project plan

Order the tasks by dependency.
For each stage write the input, the owner, the output and the acceptance criteria.
Show work that can run in parallel separately.
Mark unknown durations as estimates; don't present them as fact.

When a role and boundaries are also needed, you can add the role-assignment technique to this structure. But stacking every method into one prompt is not the goal. Keep only the part that fixes a measured problem.

How is a chain of thought's result verified?

Neat-looking intermediate steps don't prove the answer is correct. Google Research's study on finding logic errors showed the tested models struggled to reliably identify the mistake in a given reasoning trace. That older research can't be transplanted directly onto every model today, but it's a good warning to check visible reasoning separately.

1. For the same task, prepare a short baseline prompt and a staged variant.
2. Include easy, borderline, incomplete and deliberately contradictory samples.
3. Check not only the final answer but the source and calculation of each intermediate result.
4. Score by correct answers, critical errors, format and human editing.
5. If the staged variant delivers no real improvement, return to the shorter prompt.

As in a prompt engineer's evaluation approach, the test set must stay the same. Testing one variant on easy samples and the other on hard ones is not a comparison. When the model or version changes, rerun the test; don't treat the earlier result as a permanent property.

5 typical mistakes when using chain of thought

The most common mistake is turning the technique into the goal. The goal is not making the model write long thoughts; it's getting a result that is more correct, more verifiable and needs less human editing. If that isn't happening, return to the prompt's core parts: goal, context, input, sources, boundaries and output.