How do thinking and reasoning models work?

I'm Nikita Namjoshi, and I want to explain what it means when we say a large language model can "think" or "reason." These terms get tossed around a lot, but the practical question is simple: what changes under the hood when a model actually reasons through a problem rather than just spit out the most likely next word?

I'll walk through the core ideas that make modern thinking models effective: the role of scaling laws, why asking a model to "show its work" helps, how we can spend more compute at inference time, and how reinforcement learning shaped the models that generate long, useful chains of thought. I will also cover practical strategies you can expect to see when these models are deployed and give clear pointers for developers and curious users.

Table of Contents

• 🔬 Why scaling laws matter
• 🧠 Why "show your work" helps: chain-of-thought prompting
• ⚙️ Test time compute: spending compute when it matters
• 🔁 Best-of-n and why frequency can help
• 🏷️ Learned reward models: scoring candidate answers
• 🎯 Reinforcement learning: teaching models to think
• 🧩 Supervised fine-tuning and reinforcement learning: a practical combo
• 🔍 What's actually happening when the model "thinks"
• 🛠️ Practical strategies for developers and practitioners
• 🔭 Research landscape and where thinking models are heading
• 📈 Putting it all together: the architecture of a thinking system
• 🧭 A few misconceptions I clear up often
• 💡 Real examples that illustrate the mechanics
• 🧾 Practical checklist before deploying thinking models
• 🔗 Tools and references I recommend
• ❓ FAQ
• 🧾 Closing thoughts

🔬 Why scaling laws matter

One of the simplest but most important observations in recent years is captured by what people call scaling laws. In a nutshell: when you give transformer models more compute during training, more parameters, and more training data, their performance goes up in predictable ways.

This pattern explains a lot of the progress we've seen. Models trained with bigger compute budgets pick up more subtle linguistic patterns, handle rare constructions better, and generalize across tasks like coding, math, and reasoning. So scaling the model up is one pillar of getting better output.

But scaling during training is only part of the picture. There's an equally important lever you can pull after training: the compute used while the model generates a response, also known as test time compute. That second lever is essential for thinking models. Let me explain why.

🧠 Why "show your work" helps: chain-of-thought prompting

Models generate answers token by token. Each generated token is the result of a forward pass through the model. If you ask a model for a final numeric answer in one shot, it has to solve the whole reasoning problem inside a single forward pass to output the correct token.

That is often hard. But if you ask the model to produce intermediate steps, a surprising thing happens: accuracy improves. This technique is called chain-of-thought prompting. Instead of just asking "What is the final answer?" you prompt the model to write down the intermediate steps it used to reach that answer.

Example: Instead of "the answer is 11," a chain-of-thought response includes the arithmetic steps that lead to "the answer is 9."

Why does that work? Because generating intermediate steps uses more tokens and therefore more forward passes. Those extra forward passes let the model "spend" more compute thinking through the problem rather than trying to compress the whole solution into a single token generation step. In effect, you're increasing test time compute by prompting the model to reason step by step.

Two practical takeaways:

• When solving multi-step problems, ask the model to explain its steps. That alone often improves correctness.
• Chain-of-thought is not magic. It leverages the token-by-token nature of generation to give models more room to compute intermediate results.

⚙️ Test time compute: spending compute when it matters

Test time compute refers to the extra compute you provide the model at inference, after training is complete. You can think of it as cranking up the thinking time for particular queries. Several strategies let models use more compute at test time, and each has trade-offs in latency, cost, and reliability.

Here are common ways to use test time compute:

• Generate more tokens per reply. Asking for detailed chains of thought naturally increases token usage and gives the model more forward passes to reach a conclusion.
• Generate multiple candidate responses. Call the model many times with the same prompt to get diverse answers, then pick the best.
• Use a separate verification or reward model. Produce many candidates and evaluate them with another model that assigns scores for correctness or relevance.

Each technique buys you additional computation at inference. Let me take you deeper into two of these strategies: best-of-n and learned reward models.

🔁 Best-of-n and why frequency can help

The simplest way to use test time compute is to make n independent calls to the model with the same prompt and return the most frequent answer. For example, if n equals 100, you now have 100 candidate responses. Returning the most frequent response can improve results for many tasks.

This works because the model's sampling process sometimes produces the correct answer more often than other answers. If the sampling distribution favors the right answer, frequency will surface it. But there are limits.

Frequency-based selection fails when the model is consistently wrong in the same way. If a model repeatedly follows an incorrect reasoning path, you'll get many repeat errors. It's like trying a recipe 100 times with the wrong ingredients. More attempts won't fix the underlying mistake.

🏷️ Learned reward models: scoring candidate answers

A more refined approach is to train or use a second model that scores candidate outputs. This reward model judges answers on correctness, fluency, and other desired criteria.

Here's how it often works in practice:

1. Produce many candidate responses using the base model.
2. Run each candidate through a reward model that outputs a score for correctness or step-level quality.
3. Return the highest-scoring candidate, or aggregate by grouping identical answers into buckets and choosing the bucket with the highest cumulative score.

You can even have the reward model score each step of a candidate chain of thought instead of just scoring the final answer. That gives granular signals that help identify which chains contain valid reasoning and which diverge into errors.

Score-based selection tends to beat simple frequency when the model produces a variety of answers or when a trained verifier can detect correctness reliably. But it requires an additional model and labeled data or proxy objectives to train the verifier.

🎯 Reinforcement learning: teaching models to think

Generating multiple responses at inference is useful, but how do models learn to produce long, reliable chains of thought by themselves? Reinforcement learning is a key technique that unlocks better internal reasoning.

During pre-training, models learn general language patterns through next-token prediction across massive text corpora. Post-training adapts those models to tasks we care about. Supervised fine-tuning helps the model learn specific input-output mappings. But reinforcement learning can teach the model to prefer action sequences that lead to correct outcomes.

Think of the model as an agent. The agent's observations come from the prompt and the tokens generated so far. The agent's actions are token generations. The environment checks the generated tokens and returns a reward signal based on whether the final result is correct. This is reinforcement learning with verifiable rewards.

Verifiable rewards are important. If you can objectively check correctness for a class of problems, you can give clear feedback. For math or coding problems where the answer is checkable, the model receives unambiguous signals: correct or incorrect. That clarity allows the model to learn which reasoning trajectories lead to success.

During RL training, surprising patterns can emerge. One documented behavior is that models start to produce longer chains of thought. Those longer chains often correspond to better performance. The model has learned that taking more internal computation steps increases the chance of solving the problem correctly.

🧩 Supervised fine-tuning and reinforcement learning: a practical combo

In practice, models benefit from a mix of supervised fine-tuning (SFT) and reinforcement learning. SFT teaches the model to follow instructions and adopt a predictable output style. It provides examples of how to structure chains of thought. RL then refines the model to favor sequences of tokens that lead to correct, verifiable outcomes.

In one useful summary of the interplay between these methods, people say "SFT memorizes, RL generalizes." What that means is:

• Supervised fine-tuning provides templates and patterns the model can reliably copy. It is excellent for shaping form and style.
• Reinforcement learning encourages the model to explore solution paths and identify general strategies that work across novel tasks.

The result is a model that both knows how to present a chain of thought and is motivated to explore reasoning steps that genuinely lead to correct answers. This combination is a practical recipe for getting models to "think" more effectively.

🔍 What's actually happening when the model "thinks"

It helps to demystify the word "thinking." Behind the scenes, the model is optimizing token generation under constraints of its learned weights and any reward signals it received. The process looks like this:

1. The model receives a prompt and context.
2. It generates tokens step by step. Each token generation is a forward pass that conditions on the history.
3. The model can be set up or trained to prefer longer, structured chains of tokens that decompose a problem into intermediate steps.
4. When trained with RL and verifiable rewards, token sequences that lead to correct outcomes are reinforced.
5. At inference, you can either produce a single chain-of-thought trace or generate many candidates and verify them with a reward model or scoring strategy.

So "thinking" is not some separate module. It is a combination of how the model uses tokens at generation time plus how we shaped the model during training to seek reward for long, accurate chains of tokens.

🛠️ Practical strategies for developers and practitioners

If you're building systems that rely on thinking models or planning to integrate them into apps, here are practical approaches and trade-offs to consider.

Use chain-of-thought for multi-step tasks

Ask the model to explain intermediate steps when the problem requires multi-step reasoning. For many math, logic, and code synthesis tasks, this improves correctness.

Combine sampling and verification

Generating multiple candidates using a temperature-controlled sampler and then selecting with a verifier or learned reward model often outperforms naive single-shot responses. This combination balances exploration and reliable selection.

Budget for compute and latency

More tokens and more candidates mean more compute and higher latency. Decide whether correctness or speed is your priority. For critical tasks where accuracy matters, it is often worth spending more compute at test time.

Prefer verifiable tasks for RL fine-tuning

Reinforcement learning with verifiable rewards works best on tasks where you can objectively check the result: math problems, program outputs, proofs, and constrained logic tasks. When reward is noisy or subjective, RL signal quality drops and learning becomes unstable.

Monitor failure modes

Even thinking models make mistakes. Common failure modes include:

• Confident but incorrect chains of thought.
• Repeated errors across many sampled candidates.
• Reward model biases that favor superficial fluency over correctness.

Design tests and logging to catch these problems early and to tune your verifier or selection strategy accordingly.

🔭 Research landscape and where thinking models are heading

The field is evolving fast. Here are research directions and open problems I watch closely:

• Better verifiers. Creating reward models that reliably score correctness across wide domains remains a challenge.
• Efficient test time compute. Finding ways to get the same gains from thinking with less inference cost is a major engineering target.
• Robust chain-of-thought alignment. Ensuring that generated chains of thought are not only plausible but actually causal to the final answer.
• Generalization beyond verifiable tasks. Extending RL-style benefits to more subjective or open-ended tasks where verifiable rewards are unavailable.

These topics tie back to core questions about interpretability and trust. Chains of thought help make the model's process more transparent, but we still need methods to verify that the stated reasoning is faithful to the internal computation, rather than post hoc rationalization.

📈 Putting it all together: the architecture of a thinking system

Here is a simplified blueprint for a production system that uses thinking models effectively:

1. Pre-train a large transformer on massive text corpora to learn general patterns.
2. Supervised fine-tune on instruction-following and chain-of-thought examples to get reliable formatting and initial reasoning behavior.
3. Reinforcement learning fine-tune using verifiable rewards so the model prefers token sequences that produce correct outcomes.
4. Inference strategies that include chain-of-thought prompting, sampling multiple candidates, and scoring with a verifier or reward model.
5. Monitoring and iteration to detect systematic failures, retrain verifiers, and adjust sampling and scoring policies.

This stack explains why contemporary thinking models can produce long reasoning traces: the training process nudges them toward generating longer, helpful chains of tokens, and the inference process can amplify correctness through selection and verification.

🧭 A few misconceptions I clear up often

• Thinking models are not conscious. They do not introspect or have awareness. They generate token sequences that mimic reasoning because those sequences are rewarded or sampled more often.
• Long chains of thought are not always better. Chains must be both correct and faithful. Longer does not imply correct. Training and verification matter.
• Reward models are not infallible. Verifiers can have biases and blind spots. They must be evaluated carefully against gold-standard checks.

💡 Real examples that illustrate the mechanics

Here are concrete scenarios where these ideas show up:

Math problem with chain-of-thought

A short arithmetic puzzle often reveals the effect clearly. If you prompt for a single-token answer, the model may guess the final number and get it wrong. If you prompt for the intermediate arithmetic steps, the model tends to compute accurately across those steps and arrives at the correct final value.

Code generation and test-driven verification

When generating functions, you can ask the model to write unit tests or run the generated code in a sandbox. Those verifiable checks allow reinforcement learning or selection to prefer solutions that actually pass the tests. This is exactly the kind of verifiable reward signal that helps models generalize to unseen problems.

Multi-step planning

For complex planning, ask the model to decompose the task into subgoals and then solve each subgoal. This explicit decomposition mirrors hierarchical reasoning and often yields better, more interpretable plans.

🧾 Practical checklist before deploying thinking models

• Decide whether chain-of-thought is appropriate for the task and whether it will slow down the user experience unacceptably.
• Determine if you can verify outputs automatically. If so, design verifiers and use RL fine-tuning where feasible.
• Set budgets for inference compute and decide how many candidates you can reasonably generate.
• Build monitoring to detect repeated errors and evaluate the faithfulness of chains of thought.
• Keep iterating on the verifier and on the mixture of SFT and RL used during post-training.

🔗 Tools and references I recommend

I often point teams to resources that explain practical implementations of these ideas. Helpful starting points include papers on chain-of-thought prompting, best-of-n sampling strategies, and reinforcement learning from human feedback or verifiable rewards. These works show the experiments and metrics that back up the claims made here.

If you want to experiment on your own, consider starting with tasks that have objective verification such as mathematical problem sets, logic puzzles, or small programming challenges. Those tasks make it easier to measure improvements from SFT and RL.

❓ FAQ

What exactly does "test time compute" mean?

Test time compute is the additional computation used during inference when a model generates a response. It includes generating longer outputs, making multiple calls to the model for diverse candidates, or running verification models. The goal is to give the model more chance to arrive at the correct answer after training has finished.

Why does asking the model to show its work improve accuracy?

When the model writes intermediate steps, it produces more tokens and therefore performs more forward passes through its weights. Those extra forward passes let the model compute intermediate results and reduce the burden of compressing the whole solution into a single-token output. This increases the chance of arriving at a correct end result.

Is chain-of-thought always reliable and faithful?

No. Chains of thought can be fluent and plausible while still being incorrect. Faithfulness is an active research problem. Verification with unit tests, checkers, or human review helps ensure the chain reflects real computation rather than post hoc rationalization.

How does reinforcement learning help models reason better?

In an RL setup, the model generates sequences of tokens and receives rewards for correct outputs. Verifiable tasks give clear rewards, so the model learns to prefer token sequences that lead to success. Over time, it discovers longer and more effective chains of reasoning that increase its expected reward.

What is the difference between supervised fine-tuning and RL fine-tuning?

Supervised fine-tuning shows the model specific input-output pairs to teach style and structure. Reinforcement learning allows the model to explore and choose actions based on feedback, learning strategies that generalize to new problems. SFT is great for bootstrapping consistent behavior, while RL drives generalized problem-solving improvements.

Does generating many candidate answers always help?

Not always. If the model consistently follows a wrong reasoning path, generating many candidates will just produce many similar wrong answers. Candidate generation helps when the sampling distribution includes diverse ways to solve the problem and when you have a reliable verifier to pick the best result.

When should I use a reward model versus returning the most frequent answer?

Use a reward model if you can train or obtain one that reliably scores correctness or step-level quality. This is especially helpful for nuanced tasks where the most frequent answer may not be the best. Frequency selection is simpler and may suffice when the model's sampling errors are random and not systematically wrong.

What tasks are best suited to RL with verifiable rewards?

Tasks where you can objectively check correctness are best. Examples include mathematical problem solving, code generation with test suites, formal logic problems, and certain constrained planning tasks. These allow clear reward signals that drive learning.

Are thinking models more expensive to run?

Yes. Thinking models typically use more tokens or multiple calls at inference, so they increase compute and latency. The extra cost can be worth it for critical tasks that need higher accuracy. You should weigh the accuracy gains against infrastructure budgets and user experience constraints.

How do I detect when a chain of thought is wrong?

Use verifiable checks such as unit tests, formal verification tools, or deterministic checkers where possible. For less formal tasks, human review or contrastive verification using multiple models can help detect inconsistencies or implausible reasoning steps.

🧾 Closing thoughts

Thinking models are built on two main ideas: give the model more compute when it matters, and shape its behavior so that extra compute is used to reason rather than to chatter. Chain-of-thought prompting, multiple candidate generation, learned reward models, and reinforcement learning with verifiable rewards are the practical levers that make modern LLMs better at complex tasks.

These approaches are complementary. You can bootstrap reasoning with supervised fine-tuning so the model knows how to present intermediate steps, refine those capabilities with reinforcement learning so the model prefers correct chains, and deploy inference strategies that select the best outputs with verifiers or scoring systems.

The state of the art will continue to change, but the core idea stays the same: more compute during inference, guided by reliable training signals, helps models produce more accurate, interpretable, and useful reasoning traces. If you build with that principle in mind, you can design systems that take advantage of the best that thinking models offer while managing their costs and limitations.