Hierarchical Planning With Dynamic Subtask Generation

Most LLM agents plan the way a nervous student writes an essay outline: once, upfront, then they stick to it no matter what the paragraph turns out to need. That's a common pattern in production agents today, and it's why so many of them struggle to sustain coherent execution past a handful of steps. Hierarchical planning with dynamic subtask generation fixes this by treating the plan as something the agent keeps rebuilding as it learns, rather than a script it reads off once and executes blind.

The distinction matters because the failure usually isn't about model quality at all. An agent that can't finish a multi-step research task, a multi-page checkout flow, or a long refactor typically has a perfectly capable model bolted to a planning layer that never adapts. Fix the planning layer, and the same model suddenly looks a lot smarter.

What hierarchical planning actually is, and how dynamic subtask generation changes it

Hierarchical planning splits the job of deciding what to do from the job of doing it. A high-level goal gets broken into subtasks, and those subtasks can themselves be broken into smaller ones, so the whole thing looks less like a to-do list and more like a tree, or in some implementations, a graph.

This isn't a new idea invented for language models. Cognitive scientists have described something close to it for decades: people tend to start with a coarse, fuzzy version of a goal and refine it toward concrete action only as they go. Someone planning a dinner party doesn't start with "chop the onions." They start with "feed twelve people," then narrow toward a menu, a shopping list, then knife work. That progressive-refinement pattern shows up across tasks as different as cooking and medical diagnosis, and the mechanism holds in both.

A 2025 paper out of Renmin University and Huawei's Noah's Ark Lab, "From Coarse to Fine," builds this directly into a system called AdaPlan-H. It starts with a coarse macro plan and refines it based on how hard the task actually turns out to be, instead of committing to one level of detail from the start. Tested on ALFWorld and ScienceWorld, environments built around long chains of sequential actions, the approach adapts plan granularity to task difficulty rather than guessing upfront.

Nvidia and CUHK researchers took a more formal route with Polymath (2025), representing the plan as a graph, G(T,E), where nodes are subtasks and edges are dependencies between them. The graph expands as execution proceeds, which is a meaningfully different object than a flat list of steps: dependencies get added, subtasks split further, and branches that turn out unnecessary get pruned without unwinding the whole plan.

Older AI planning research got here first, in a narrower way. Hierarchical Task Network planning, the classical antecedent, decomposes a goal into a logically sound sequence of subgoals using a symbolic representation of world state and formally specified domain descriptions. What's changed is that LLMs now sit on top of these symbolic planners as heuristic generators, suggesting which decompositions are worth trying rather than replacing the symbolic structure entirely.

Underneath all of it sits a framing worth taking seriously. The agent's decision loop behaves like a partially observable Markov decision process: it never sees the full world state, only partial signals it has to interpret as it goes. That's exactly why static plans break. A plan can't be fully specified in advance for a world you can only partially observe, so the planner has to function as a genuine reasoning component, tracking what's been learned and deciding what to decompose next, not a router that picks the next item off a list.

How the orchestrator-subagent split maps the hierarchy onto a running system

In practice, this hierarchy gets built as a split between an orchestrator and a set of subagents. The orchestrator takes the high-level goal, breaks it into subtasks, hands each one to a specialized subagent, collects what comes back, and stitches the results into a final answer.

Because subagents working on independent pieces of the graph run at the same time, wall-clock time collapses to roughly the length of the slowest single subagent, not the sum of every step added together. For long-horizon tasks, that difference is the whole ballgame. A research task with six independent lookups shouldn't take six times as long as one lookup, and with a working orchestrator, it doesn't.

The orchestrator is also the single point of failure, and this is where most teams underinvest. If the initial decomposition sends every subagent down the wrong path, no amount of subagent competence saves the run. Decomposition quality ends up being the highest-leverage variable in the entire system, higher leverage than model choice, higher leverage than tool selection. Teams that pour their budget into swapping models while leaving a brittle decomposition step untouched are optimizing the wrong layer.

Self-correcting orchestrators, ones that check intermediate subagent output and dispatch follow-up work in response, are harder to build than orchestrators that just fire-and-collect. Build the simpler version anyway if the task is short: a three-step task barely notices the difference. But the gap widens fast as tasks get longer, and a thirty-step task falls apart without self-correction built in somewhere.

Anthropic's published guidance names orchestrator-workers as one of five core agentic workflow patterns, calling out its specific fit for tasks where the subtasks can't be predicted in advance. A broader taxonomy circulating among orchestration engineers in 2026 lists six patterns covering most production cases: Supervisor, Sequential Pipeline, Parallel Fan-Out, Router, Hierarchical, and Evaluator-Optimizer loops. Production systems rarely pick just one. They combine two or three inside a single workflow, running a router at the top level and a fan-out pattern underneath it, say.

One practical habit that saves a lot of pain later is defining subagents as discrete, versioned units, each one a markdown file specifying a name, a tool list, a model, and a system prompt. That makes the whole hierarchy auditable. When something breaks three layers deep, the goal is pointing at the exact subagent definition that misfired, not guessing.

Where tool use enters the planning loop and why web data is a first-class tool

Tool calls inside this architecture aren't bolted-on extras. They're planned actions, decided by the orchestrator the same way it decides to spin up a subagent. The model functions as a cognitive controller, combining memory, tool use, and feedback from the environment to chase a goal spanning more than one step.

Web data earns a special place among those tools because of a problem no amount of model scale fixes: knowledge cutoffs. A foundation model reasons over a frozen training snapshot. Anything that depends on current state, a price, a document, a competitor's positioning, a regulatory status, comes out stale or invented unless the agent actually goes and looks.

That going-and-looking is itself a planning decision, not a hardcoded step bolted to the front of the pipeline. The orchestrator has to decide whether a given subtask actually needs live data, what query or URL to send, and what shape the returned data needs to take before downstream subagents can use it.

This is the mechanism behind retrieval-augmented generation: fetch evidence at query time from outside the model's frozen memory. Agentic approaches push that idea further, treating the retrieval step as something the planner can invoke selectively, instead of a fixed preprocessing stage bolted onto every query whether or not it's needed.

None of it works if the retrieved content is stale or messy. A subtask phrased as "research current pricing for competitor X" only earns its keep if it resolves into a web-fetch call that returns something structured enough for the planner to reason over, the same way it reasons over any other subagent's output.

What the planner needs from a web fetch: clean, structured, token-efficient output

Raw HTML is close to useless here, and this is where a lot of home-built agents quietly waste most of their context budget. A full page pulled straight off the wire is token-heavy, full of markup noise, and semantically opaque to a planner that just wants an answer to a question. Feeding raw HTML into a subtask is a good way to burn a token budget on <div> tags instead of the sentence buried inside them.

Clean markdown is the right target format instead: readable, light on tokens, and it keeps the document's structure, headings, lists, tables, without the markup overhead of the original page.

When the planner already knows exactly which fields it needs, price, publish date, author, status, a schema-driven extraction approach beats free-text scraping outright. Pass a JSON schema into the extraction layer, and what comes back is a typed object the planner can use directly, not a wall of text it has to re-parse on its own.

The genuinely hard part is dynamic content. Pages rendered by a client-side scripting language, single-page apps, content that only shows up after a login or a button click or a form submission: none of that shows up in a plain HTTP fetch. The extraction layer needs headless rendering or session-aware handling just to see it.

Researchers at McGill found in 2025 that AI-assisted extraction methods held accuracy at 98.4% even as target page structures changed underneath them, with setup time dropping from weeks down to hours. That inverts the old economics of scraping, where a team spent roughly a fifth of its effort building an extractor and the rest of its time nursing it back to health every time a site redesigned its markup.

Self-healing belongs at the planning layer, not buried inside the scraper. When a web subtask fails because a target page got restructured, the orchestrator should get back a structured error it can act on: recover, retry, escalate. Not a silently corrupted extraction that looks fine until three subtasks downstream produce nonsense.

Compliance isn't optional context here either. As of 2026, Europe's AI Act, the US FTC's draft data-access guidance, and one regulator's 2025 guidance on privacy-relevant scraping all point toward the same requirement: data collection has to be auditable, transparent, and no more invasive than the task demands. That has to be designed into the extraction layer from day one. Retrofitting it later is expensive, and in some jurisdictions, too late to matter.

How self-optimization closes the loop between planning and execution

Generate the task graph once, execute it, never revisit it: that's hierarchical planning in name only. Real dynamism means the orchestrator revises the graph based on what subagents actually return, not just what it predicted they'd return.

Polymath's approach here is genuinely clever: an optimization of the task flow graph paired with self-reflection mechanisms that refine workflows using feedback from an LLM evaluator. Across coding, math, and multi-turn QA benchmarks, it landed an 8.1% average improvement over prior state-of-the-art baselines.

A separate 2025 paper, ReAcTree, builds hierarchical LLM agent trees with actual control flow: branching, looping, conditional dispatch, for long-horizon planning. That control flow is what separates a planning tree from a glorified checklist. A checklist executes top to bottom. A tree with control flow loops back, skips a branch entirely, or forks into parallel paths depending on what it just learned.

Evaluator-Optimizer, one of the six orchestration patterns mentioned earlier, formalizes the same idea as a named pattern: an evaluator subagent scores intermediate output, and an optimizer dispatches revised subtasks in response. Splitting evaluation from execution is exactly what makes self-correction tractable instead of a tangle of ad hoc retries.

Web monitoring fits naturally into this loop. A subagent that watches a target page for a price change, a status update, a new section, and triggers replanning when it sees one, is self-optimization pointed at the outside world instead of at the agent's own internal graph. The planner ends up responding to reality, not to its own stale assumptions about what reality looked like an hour ago.

Building that self-correction loop takes more work upfront than shipping a static planner. A three-step task tolerates a fixed plan just fine, no argument there. But the payoff scales with task length, and a task spanning dozens of steps, or hours of wall-clock time, drifts off course without adaptive replanning somewhere in the loop.

The failure modes that hierarchical planning with dynamic subtasks actually solves

Hallucination stops being a mostly cosmetic problem the moment an agent can take real action: modifying a file, submitting a form, calling a live API. A hallucinated subtask result at that point becomes a corrupted input silently feeding the rest of the pipeline, and that's a concrete, sometimes irreversible failure, not an embarrassing typo. Hierarchical plans with verification checkpoints between subtasks catch bad output before it propagates to whatever subagent acts on it next.

Prompt injection is the web-specific version of the same threat. Malicious instructions embedded in a scraped page get followed by an agent that's simply doing what instructions tell it to do, since an instruction-following model is built to follow instructions. Returning structured, typed fields from the extraction layer instead of raw page text shrinks the surface area available for that kind of injection considerably.

Infinite loops show up when a planner has no explicit termination condition and keeps replanning against a subtask that was never solvable in the first place. Control-flow structures like the ones in ReAcTree, and Evaluator-Optimizer loops generally, need an explicit break condition built in. Otherwise the agent just spins, burning tokens on a problem that was dead on arrival.

Because the orchestrator is a single point of failure, a bad initial decomposition can sink the entire run before any subagent even starts working. Coarse-to-fine refinement, the AdaPlan-H pattern discussed earlier, along with self-reflection before dispatch, is the mitigation that actually works: check the plan's shape before committing resources to executing it.

Architecture misalignment, not immature technology, is the leading reason agentic projects get shelved. Gartner projects that more than 40% of agentic AI initiatives will be canceled by 2027 for exactly this reason. The diagnosis underneath tracks with a rough 80/20 split: a large share of enterprise processes need deterministic execution, while a much smaller share actually benefit from autonomous, dynamic reasoning. Most teams get this backwards. They reach for a fully dynamic planner when a fixed pipeline would have shipped faster and broken less, mistaking flexibility for progress. Hierarchical planning isn't the right tool for every job, and scoping which subtasks genuinely warrant dynamic generation, versus which ones should just run as a deterministic pipeline, is a design decision worth making before writing any code, not after the first outage.

Building a hierarchical planning agent that uses live web data: the implementation path

Start with the decomposition interface. The orchestrator needs a structured prompt contract that outputs an actual task graph, nodes and dependency edges, not a flat numbered list dressed up as a plan.

Register web retrieval as a named tool the planner knows it can call. The function's signature should take a URL and an optional JSON schema, and return either clean markdown or structured JSON, something an LLM can reason over without extra parsing.

Choosing the extraction layer is where teams either save themselves months of pain or sign up for it. Building and maintaining custom scrapers in-house means living inside that old ratio: a small slice of effort spent building, most of it spent nursing brittle selectors every time a target site changes its layout. A dedicated web-to-LLM API sidesteps that maintenance burden entirely, making it the choice worth defaulting to unless there's a specific reason to own the crawling stack. Crawl4AI, an open-source crawler with a natural-language query interface, is a solid option for teams that want it running locally and under their own control, describing what's needed in plain language instead of hand-writing CSS selectors.

Add an evaluator subagent that checks every web-fetch result against the schema and confirms completeness before the orchestrator dispatches anything downstream that depends on it.

Handle dynamic content explicitly, not as a patch applied after something breaks in production. JavaScript-rendered pages and session-dependent content need headless rendering support baked into the extraction tool from the moment it's registered.

Build in termination and escalation conditions from the start. Every planning loop needs an explicit exit path, and long-horizon tasks specifically need a human-escalation route for when replanning crosses some threshold and keeps failing anyway.

Integration speed ends up mattering more than most teams expect going in. The gap between a working prototype and a multi-week infrastructure slog usually comes down to one decision: is the web-data layer a single API call, or a bespoke scraping stack someone has to babysit afterward? Going from nothing to a working call in minutes, rather than weeks, is a real architectural constraint when iteration speed determines what actually ships.

Where the research frontier is heading and what it means for teams building now

Demand for multi-agent systems has climbed sharply, and the shift shows up less in flashy demos and more in the plumbing: task graphs that expand and contract at runtime, evaluators that sit between planning and execution rather than after it, and web retrieval treated as a tool the planner reasons about invoking, not a preprocessing step wedged in before the "real" work starts.

The papers cited here, AdaPlan-H's coarse-to-fine refinement, Polymath's graph optimization, ReAcTree's control flow, all converge on the same underlying claim: a fixed plan is a bet that the world won't change and the goal was fully specified from the start. Neither bet holds for long-horizon tasks, and the failure data on canceled agentic projects backs that up.

Teams building now don't need to wait for the next model release to fix this. The fix is architectural: separate the orchestrator from the subagents, treat web retrieval as a first-class tool with a real schema contract, build an evaluator into the loop instead of trusting output blindly, and scope carefully which parts of the workflow actually need dynamic planning versus which ones are better off deterministic. Get that structure right, and the model underneath becomes a much smaller variable than most teams currently treat it as.