21. Dynamic Workflows at Scale (Escaping the AI Chat Window Bottleneck)

In Episode 21 of Beyond Prompting, author Sho Shimoda explores how to break through the manual chat window bottleneck by programmatically orchestrating AI agents using Dynamic Workflows. When coordinating large-scale engineering tasks across a codebase, managing dozens of parallel tasks manually through a chat transcript becomes a major bottleneck. In this episode, we break down:

• When Scripts Beat Conversations: The three conditions that justify a workflow script—large volume (20+ items), complete task independence, and aggregated findings.
• JavaScript Workflow Mechanics: Writing deterministic orchestration scripts using agent(), pipeline(), parallel(), phase(), and log(), while enforcing strict replayability rules that block non-deterministic calls like Date.now() or import().
• Prompt Caching at Scale: How Claude Code holds parallel agent fan-outs briefly at startup so subagents share a single cached prompt prefix, dramatically lowering execution costs.
• Security & Execution Boundaries: Why workflows require explicit user invocation via the ultracode keyword and cannot be triggered automatically by untrusted webhooks, scheduled tasks, or pull request comments.
• The Two-Phase Pattern: Designing trustworthy automation through a two-phase architecture—Phase 1 surveys and generates candidate findings, while Phase 2 runs an adversarial verification step on each finding before aggregating results.

(Note for listeners: This episode covers Chapter 21 of Sho Shimoda's book RUNNING CLAUDE: The Operator’s Guide to Chat, Cowork and Claude Code, available on Amazon as the successor book to Master Claude: Chat, Cowork and Code).