Kaan · Article
2025-11-21
Building an Orchestrator That Actually Thinks
Traditional API chaining is just do this then that with no validation and no recovery. I built something that evaluates outputs and decides what to do next.
Most AI automation breaks when you need it most. Here's how I fixed that with an orchestrator that actually thinks.
Traditional API chaining is just "do this, then that" Basically no validation, no recovery. When your research step returns garbage, the outreach step still fires and you're stuck debugging at 2am. I needed something that could evaluate outputs and decide what to do next, not just blindly execute a sequence.
Here's the 3-step workflow I built:
→ Step 1: Start with inputs Feed the orchestrator your task context (lead data, research parameters, product catalogue)
→ Step 2: Execute a function System calls the appropriate tool: Sonar for web research, Gemini (3.0 Pro) for outreach copy, Sonnet for product matching
→ Step 3: Evaluate and route The orchestrator analyzes the output logs and decides: continue to next step, retry with different parameters, or bail to human review
The guardrails that make this actually work:
Evaluate everything (20% done) → Reduced context about each step + explicit instructions so the orchestrator knows what "good" looks like
Visualize decisions (60% done) →→→ Detailed JSON responses show exactly why it chose to retry research or skip outreach. Debugging became reading a decision log
Test and iterate (100% done) →→→→→ Different models for different jobs. Gemini handles orchestration, Sonar for research. I A/B test against Claude for certain writing tasks based on cost/quality tradeoffs
Real setup example:
- Gemini Flash as the conductor (decision making) - Sonar for web research (specialized tool) - Gemini Pro for outreach - Claude Sonnet for product matching - Circuit breakers: 3 failures = human review, I'd rather avoid infinite loops
The skill shift is real but it transferred cleanly. State management, error handling, idempotency. Pretty much the same concepts I used building workflow engines, just now I version prompts instead of deployment configs.
If you're building something similar and want to compare architectures, I'm happy to share eval notes.