Prompting, Scaffolding, Post-Training: A Short History of Reasoning

This article is now published on my substack newsletter Loafing in the Valley.

Agents became viable through the development of reasoning. Reasoning evolved from intermediate steps elicited through prompting, to trajectories orchestrated by external scaffolds, and to control decisions increasingly learned by the model itself.

The pursuit of reasoning produced two parallel developments. Test-time scaling became something that could be elicited, explicitly allocated, or controllable by a learned policy. At the same time, tools, evaluators, execution environments, loops, and memory systems converged around the model into an agent harness. This article traces that development of reasoning through prompt engineering, scaffolding engineering, and post-training.

Reasoning in a nutshell

Reasoning is a capability or behavior incurred at test time. It involves the interaction between a fixed model and a surrounding system that guides the reasoning trajectory.

Figure 1 illustrates reasoning at test time. The trajectory could contain intermediate reasoning tokens, plans, partial answers, tool calls, observations, execution results, critiques, revisions, and memory updates. The scaffold includes the inference machinery around the model: the prompt, decoding rule, search procedure, tools, evaluator, sandbox, memory policy, and/or controller.

Looking back at how reasoning was developed originally, the motivation was to enable large models to solve complex problems requiring “System 2 thinking” more reliably. Even now, with the fleet of reasoning models and more advanced agent systems being readily available, the objective of reasoning remains the same and can be reduced to the following form.

The output quality could be designed to reflect task completion, factual accuracy, or some other success criteria. The cost may include generated tokens, tool calls, latency, memory use, external operations, or API expense.

Three Reasoning Regimes

Between 2022 when Chain-of-Thought (CoT)¹ was introduced and early 2025 when DeepSeek-R1² was released, the development of reasoning capability in the field can be interpreted as an effort to improve the expected output value by optimizing different parts of the system. There are three distinctive regimes.

In practice, models, prompts, and scaffolds are often co-designed. Prompt engineering is also technically a restricted form of scaffold engineering, but it is useful to separate static context design from a dynamic runtime controller. To put them into perspective, Figure 2 shows the three regimes as well as a timeline of the works in each regime.

Test-Time Scaling and Agent Stack PoC

In this article, two themes will recur across the three regimes.

Test-Time Scaling: each regime creates a different way to spend computation at test time. Prompting with reasoning steps leads to more compute than direct prompting, scaffolding orchestrates the compute, and post-training learns a better policy of allocating the compute.

Agent Stack Early Emergence: each regime changes the boundary between the model and its surrounding system. Prompt engineering leaves the surrounding system thin. Scaffolding introduces loops, tools, evaluators, and execution environments. Post-training moves some control decisions into the model, producing a brand new catalog, the reasoning models.

Prompt Engineering

Prompt engineering is the “thinnest” one among the three. With fixed models and minimal scaffold, the engineering problem is to design an optimal prompt format and simple decoding procedure to yield a reasoning trajectory. Between 2022 and 2023, prompt engineering became the most viable path to steer models towards better reasoning behaviors. (Prompt engineering remains important in agent stack till this day.) This was also the period that felt most like alchemy because the discoveries often require carefully crafted text with a lot of trial-and-error.

Nevertheless, the central question was valuable:

CoT prompting changes the role of generated text. Instead of directly asking for a final answer, the prompt with few shot examples and reasoning steps could make a big performance gain in terms of output quality. Zero-shot CoT³ makes the prompt-level nature of the effect even more striking: a general instruction such as “Let’s think step by step” could improve performance across several reasoning benchmarks without task-specific demonstrations.

CoT prompts trigger generations of intermediate tokens. These tokens function as a temporary workspace. A model could allocate more compute to the response before giving the final answer. Since then, the intermediate tokens have been seen as a form of “reasoning” capability analogous to human reasoning.

Different prompting methods impose different shapes on that workspace. CoT encourages a linear sequence of intermediate steps. Plan-and-Solve⁴ separates planning from execution. Least-to-Most (L2M)⁵ prompting decomposes a difficult problem into easier subproblems and solves them sequentially.

Test-Time Scaling Begins with Elicitation

Prompt engineering exposes a sequential dimension of test-time scaling. A well designed prompt could encourage the model to produce more reasoning tokens before committing to an answer. A decomposition procedure could invoke the model several times, carrying intermediate results from one stage to the next.

At its core, however, the compute policy remains mostly static. Prompt engineers decide in advance whether to ask for a plan or how many stages to use. Prompting might be able to trigger a pattern of extra computation, but it cannot control the unfolding trajectory beforehand and decide that one task needs another branch while another task is already solved.

Nevertheless, the key insight was established:

More useful computation between the task and the answer can improve output quality.

This was the beginning of the connection between reasoning and test-time scaling.

A Thin Scaffold

The works in the era of prompt engineering also creates the first control surface around the model, but the surrounding architecture remains thin. The control is predetermined. There is no persistent runtime state, explicit loop, controller inspecting progress, or durable memory.

Limitations

Prompt engineering gives access to the reasoning trajectory, but only indirectly. It can influence generated tokens but cannot reliably control their quality.

If the model chooses a poor direction early, errors can propagate through the rest of the trajectory. More importantly, a static prompt cannot easily solve problem requiring dynamic injection of evidence or adapt to new information as it becomes available in real time. At the time, there were two directions to improve the status quo of large model reasoning.

• Scaffolding engineering: Build a runtime system that can control the trajectory externally.
• Reasoning focused post-training: Train the model so that useful reasoning behavior becomes more likely internally.

Scaffolding Engineering

Scaffolding engineering turns reasoning from a static prompting into a runtime procedure. Model parameters remain fixed, but the surrounding system can call the model several times, inspect partial progress, generate alternative paths, execute programs, and invoke tools.

The central question becomes:

Test-Time Scaling Becomes Programmable

This is a qualitative change from static prompting. Test-time scaling is no longer only a request for more intermediate tokens. It becomes a policy for allocating computation while the task is being solved.

The following mechanisms can be understood as progressively richer ways to implement that policy.

Separate Proposal from Selection

Training Verifiers⁶ generates many candidate math solutions and uses a learned verifier to rank them. Self-Consistency⁷ samples several CoT trajectories and selects the most consistent terminal answer. Reasoning is no longer a single point failure as many paths can be produced, and a selector (or verifier) can choose the best among them. However, this makes the quality of the selector just as important as the generator. If every trajectory leads to a wrong answer or a selector credits nicely formatted but incorrect responses, a majority vote would just create false confidence.

Search over Partial Progress

Tree of Thoughts (ToT)⁸ treats reasoning as search over transient states rather than only over the finished trajectories. A runtime controller generates candidate next steps, evaluates states, expands promising branches, prunes weak ones, and backtracks when necessary. At this point, exploration-based planning began to merge into the reasoning process. Reasoning produces or evaluates intermediate conclusions. Planning decides which operation should occur next under a goal, available actions, changing state, and a limited budget.

Delegate Exact Operations to an Execution Environment

Program-aided Language Models (PAL)⁹ and Program of Thoughts (PoT)¹⁰ separate the description of a computation from its execution. The model generates a program or program-like solution, and an external interpreter evaluates it. This introduces an execution environment into the reasoning process. It allows the model to externalize operations while also creating the need for isolation, permissions, resource limits, and error handling.

Reasoning Meets the External World

Earlier works, like WebGPT¹¹, proposed for Q&A or factual grounding tasks, already demonstrated a sequence of actions and observations. This suggests two research streams moving toward each other. One tried to create better intermediate computation. The other tried to connect models to information and operations outside their parameters.

ReAct¹² makes the convergence explicit by orchestrating the reasoning traces, actions, and observations. Reasoning helps the model update its plan. Actions gather information or change the environment. Observations influence the model on what should be doing at the next step. Reasoning becomes a closed loop. The model is no longer limited to the original context and can adapt its behavior based on new information, and course correct its next step accordingly.

Reflection, Refinement, and Memory

Once a scaffold can see failure, it can use the failure as input to improve future reasoning steps. It helps to separate two types of operations:

• Reflection diagonoses what could be wrong or what should change.
• Refinement uses that diagnosis to produce another candidate.

Self-Refine¹³ organizes inference as an iterative loop of generation, feedback, and revision, using the same fixed model as generator, critic, and refiner. The limitation is that model-generated critique is not automatically reliable. A model could fail to notice its own mistake or hallucinate a criticism of a correct answer. Reflection becomes more useful when it is grounded in evidence that was not already contained in the initial output: a failed unit test, binary rubric, or human critique.

Reflexion¹⁴ extends the loop across complete attempts. The agent interprets task feedback, writes a verbal reflection, stores it in episodic memory, and conditions the next attempt on the stored lesson.

Voyager¹⁵ takes persistence one step further. Its Minecraft agent maintains an executable skill library, retrieves useful skills for later tasks, and revises programs using environment feedback, execution errors, and self-verification. A filesystem is can hold procedures, plans, and reusable skills that extend the system beyond the current context window.

Scaffolding Begins to Converge to Agent Harness

During the scaffolding wave, many previously separate components converged into more recognizable shapes which paved the way to the current agent harness. Figure 9 summarizes the relationship with examples.

Post-Training

A model cannot literally perform web search, run Python script, or store data into a database. While these actions are only made possible through the scaffolding, a model can learn a control policy that uses the scaffold effectively. Similarly, a model cannot predict the context ahead of time but it can learn the policy to adapt to the context and produce reasoning tokens.

The central question becomes:

Test-Time Scaling Becomes A Learning Problem

The available budget and external capabilities still come from the surrounding system. But the model can learn to continue reasoning on a difficult problem, or switch strategies after detecting an error. It can invoke a tool when external computation is required, or stop the reasoning and produce an answer.

Training Intermediate Computation

Scratchpads¹⁶ provides an early example. Models can learn to emit intermediate computational states with special tokens while performing tasks such as long addition and program execution. This is related to CoT but represents a different intervention. CoT changes the prompt so that a pretrained model is more likely to produce intermediate reasoning tokens. Scratchpads changes the model parameters so that producing useful intermediate states becomes learned behavior.

Learning from Successful Traces, to Use Tools, and to Evaluate the Process

STaR¹⁷ closes a loop between inference and training. The model generates rationales for many problems. Rationales that lead to correct answers are retained or reconstructed, and the model is fine-tuned on them. The process is repeated. The reasoning trajectory is no longer only a disposable test-time artifact. It becomes training data that changes future behavior. This is an early form of bootstrapping. Inference produces candidate strategies, verification identifies useful ones, and training makes those strategies more probable on later tasks.

Toolformer¹⁸ internalizes another part of the control policy. It trains a model to decide which APIs to call, when to call them, what arguments to provide, and how to integrate the returned result into subsequent prediction. It is clear now that post-training can move decisions about the scaffold into the model without eliminating the scaffold. Through post-training, models can learn a useful tool use policy.

Let’s Verify Step by Step¹⁹ compares these approaches on mathematical reasoning and introduces a large dataset of step-level feedback for training a process reward model. Process supervision matters because a final answer hides where the trajectory succeeded or failed. Step-level feedback can identify a promising partial path, reject an invalid transition, guide search, or steer the model away from recurring local errors. It also connects post-training back to scaffolding. A process reward model can train a better model policy, but it can also be used at test time to score partial trajectories inside a search or verification procedure.

Reasoning-Focused Reinforcement Learning

Post-training existed long before reasoning models. SFT and RLHF were already used to turn pretrained models into useful assistants.

Reasoning-focused post-training increasingly emphasizes tasks with verifiable outcomes, especially on mathematics and code. DeepSeekMath²⁰ introduces Group Relative Policy Optimization (GRPO) to improve mathematical reasoning. It also reports further gains from Self-Consistency over multiple samples, directly connecting a learned reasoning policy with test-time sampling.

In 2024, OpenAI released o1²¹ as a reasoning model trained through RL. In early 2025, DeepSeek-R1-Zero² then provided a remarkable open demonstration of reasoning behavior through from GRPO being applied directly to a base model without preliminary supervised fine-tuning. It applied large scale RL with long CoT²² training samples, and reported the emergence of extended reasoning behaviors, i.e., “aha moment”.

At this point, the reasoning is officially internalized within a model as a trained policy.

Boundary between Model and Scaffold

Reasoning-focused post-training defines a boundary between models and scaffolds. Some responsibilities are moved into the learned policies, and others remain external. Figure 13 summarizes these responsibilities.

Putting It All Together

The short history of reasoning development can be summarized as an evolution from purely eliciting the trajectory via prompting to internalizing the reasoning within a model and building an external system to orchestrate the trajectory.

By the time reasoning-focused RL became a visible success, the field had already learned several lessons:

The rapid development of agents post-2025 is the result of these ideas combined: There is a model that proposes and increasingly learns how to control a trajectory. The harness provides the external world in which that trajectory can be executed, evaluated, and revised.