What is chain-of-thought reasoning?

Chain-of-thought (CoT) reasoning is when an LLM generates intermediate reasoning steps before its final answer. The original Wei et al. 2022 paper showed that prompting a 540B model with eight worked examples (or the trigger phrase "Let's think step by step") jumped GSM8K math accuracy past fine-tuned GPT-3 with a verifier. For a few years that was a prompting technique. Now it's baked into the models themselves.

Reasoning models hide the CoT

OpenAI o1 and o3, Claude with extended thinking, and DeepSeek R1 generate long internal reasoning traces before answering. o1 and o3 hide the raw chain from API consumers and return a summary; OpenAI explicitly forbids prompts trying to extract it. Claude 3.7 Sonnet's extended thinking shows the trace in raw form, with the caveat that displayed thoughts may not faithfully represent the model's actual decision-making.

In agents, CoT is the trace

Inside an agent loop, CoT shows up as the "thought" step of the ReAct pattern: the model writes a short reasoning block, then picks a tool, then reads the observation, then thinks again. That internal monologue between tool calls is what makes a run inspectable. Kill the thoughts and the trace becomes a list of opaque tool calls with no signal about why each one happened.