When an agent goes bankrupt, the simulation can now restart for another
episode while preserving the scratchpad from the previous attempt. This
lets us measure whether LLMs can learn from failure via persistent notes.
Each episode gets its own SQLite DB (*.ep1.db, *.ep2.db, ...) so plotting
scripts and post-hoc analysis work unchanged. The rollout JSON aggregates
per-episode transcripts, turns, and costs.
Key changes:
- --max-episodes CLI flag (default 1, fully backward compatible)
- Per-episode DB files when max_episodes > 1
- Scratchpad read from old DB, written into fresh DB between episodes
- RunState tracks episode results with finish_episode/reset_for_new_episode
- Agent prompt tells it about the episode number and to read its scratchpad
- Plotting script for multi-episode fund curves + scratchpad evolution
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolved conflicts — combined best of both:
- bot_runner.py: kept our trust-aware candidate building + upstream's tier-avg rates + no task cap
- task_complete.py: upstream's additive skill boost (nerfs greedy snowball) + our configurable cap (wc.skill_rate_max instead of hardcoded 10)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Hide exact reward_multiplier from agent; show tier (Standard/Premium/Enterprise) and specialty domains instead
- Add client domain specialization with 70% bias on task generation toward client specialties
- Remove qty_scale by multiplier (leaked info and doubly punished high-mult clients)
- Rewrite agent prompt to describe tiers/specialties without exact formulas
- Fix critical loop.py bug: provide full state context after sim resume (prevents idle multi-month skips)
- Add Streamlit dashboard, watch scripts, and updated plotting/extraction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace scrolling LiteLLM debug logs with an in-place Rich Live dashboard
that shows key metrics after each turn: funds sparkline, task progress bars
with colored domain labels, team skill bars, runway urgency, and more.
- New: src/yc_bench/runner/dashboard.py (BenchmarkDashboard, DashboardState)
- Add on_turn/on_turn_start callbacks to agent loop
- Auto-detect TTY, redirect all logging to logs/debug.log when live
- Add --no-live flag to disable dashboard and get old log output
- Use alternate screen buffer (screen=True) for clean rendering
- Fix start.sh: clean up stale temp files before mktemp
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old `db/` pattern in .gitignore matched src/yc_bench/db/ too,
preventing all ORM models and session.py from being committed.
Previous commit fixed .gitignore to `/db/`; this adds the 10 missing files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fresh clones failed with ModuleNotFoundError because agent/, db/,
runner/, and services/ subpackages had no __init__.py. Also anchor
/db/ and /logs/ in .gitignore so they don't match src/yc_bench/db/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3-step interactive flow: pick difficulty (with custom preset builder),
choose model from curated list (Claude, GPT, Gemini, DeepSeek, etc.),
enter API key (auto-detected by prefix). Single curl command to get started:
curl -sSL https://raw.githubusercontent.com/.../start.sh | bash
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug fixes:
- CLI --horizon-years defaulted to 3, silently overriding config presets.
Now defaults to None so config value (1yr for medium/hard/nightmare) is used.
- Runtime passed a single api_key kwarg regardless of provider, breaking
Gemini. Now lets LiteLLM resolve keys from provider-specific env vars.
- Removed temperature+top_p from LLM calls (Anthropic rejects both together).
- DB and result filenames now include config name to prevent cross-config collisions.
Benchmark results (1yr horizon, 3 seeds each):
Sonnet 4.6: medium 2/3, hard 0/3, nightmare 1/3
Gemini Flash: medium 3/3, hard 1/3, nightmare 1/3
Gemini has higher win rates (93-98% vs 40-83% on medium).
Sonnet's ceiling is higher when it survives (nightmare $10.1M vs $478K).
New scripts: plot_comparison.py, plot_sonnet_results.py, notepad_gif.py
Updated README with detailed comparison tables and failure analysis.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
