mirror of
https://github.com/GoodStartLabs/AI_Diplomacy.git
synced 2026-04-19 12:58:09 +00:00
Adding first pass at instructions
This commit is contained in:
parent
287d845d4c
commit
c9249a13ad
10 changed files with 4766 additions and 0 deletions
1095
ai_diplomacy/agent_docs/examples/claude_code_10142025.md
Normal file
1095
ai_diplomacy/agent_docs/examples/claude_code_10142025.md
Normal file
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,66 @@
|
|||
# Game Summary: Diplomacy
|
||||
|
||||
## What is Diplomacy?
|
||||
|
||||
Diplomacy is a strategic board game of negotiation and conflict set on a WWI-era European map where seven Great Powers compete for territorial dominance through simultaneous military orders and pre-game negotiation. Victory requires controlling 18 of 34 supply centers, achieved through alliance formation, coordinated attacks, and strategic deception—no dice, no chance, pure strategy and diplomacy.
|
||||
|
||||
## The Seven Powers
|
||||
|
||||
**Map:** Standard WWI Europe (1901-1935)
|
||||
|
||||
- **AUSTRIA** - Home Centers: Vienna, Budapest, Trieste
|
||||
- **ENGLAND** - Home Centers: London, Liverpool, Edinburgh
|
||||
- **FRANCE** - Home Centers: Paris, Marseilles, Brest
|
||||
- **GERMANY** - Home Centers: Berlin, Munich, Kiel
|
||||
- **ITALY** - Home Centers: Rome, Venice, Naples
|
||||
- **RUSSIA** - Home Centers: Moscow, Warsaw, Sevastopol, Saint Petersburg (4 centers)
|
||||
- **TURKEY** - Home Centers: Constantinople, Ankara, Smyrna
|
||||
|
||||
**Total:** 34 supply centers on map (7 powers control 21 at start, 13 neutral).
|
||||
|
||||
## Core Mechanics
|
||||
|
||||
### Simultaneous Resolution
|
||||
All players submit orders secretly, then orders resolve simultaneously—no turn order advantage. Conflicts adjudicated by support strength (strongest attack succeeds, ties result in standoffs).
|
||||
|
||||
### Turn Structure
|
||||
1. **Diplomacy Phase** - Private negotiations with other powers (alliances, plans, intelligence)
|
||||
2. **Order Phase** - Submit movement/support/convoy/hold orders for each unit
|
||||
3. **Resolution Phase** - All orders execute simultaneously, conflicts adjudicated
|
||||
4. **Build/Adjust Phase** - Gain units (if centers > units) or disband (if units > centers)
|
||||
|
||||
**Cycles:** Spring Movement → Fall Movement → Winter Adjustments (repeats annually).
|
||||
|
||||
### Territory & Supply Centers
|
||||
- **Supply Centers (SC):** 34 provinces marked with dots—control determines unit count
|
||||
- **Units:** Armies (land movement) and Fleets (coastal/sea movement, can convoy armies)
|
||||
- **Build Rule:** Build new units only in owned home centers during Winter
|
||||
- **Victory Condition:** First power to control 18+ supply centers wins solo
|
||||
- **Draw:** Multiple powers agree to end game (common when solo victory unlikely)
|
||||
|
||||
### Order Types
|
||||
- **Move:** `A PAR - BUR` (Army Paris to Burgundy)
|
||||
- **Support Move:** `A MAR S A PAR - BUR` (Marseilles supports Paris attack)
|
||||
- **Support Hold:** `A VEN S A TRI` (Venice supports Trieste defense)
|
||||
- **Convoy:** `F ENG C A LON - BRE` (Fleet convoies army across sea)
|
||||
- **Hold:** `A MUN H` (Munich stays, default if no order given)
|
||||
|
||||
**Combat Resolution:** Attack strength = moving unit + supporting units. Defender strength = unit + supporting holds. Higher strength wins; equal strength = bounce (standoff, no movement).
|
||||
|
||||
## Distinguishing Features
|
||||
|
||||
### Pure Strategy, Zero Luck
|
||||
No dice rolls or randomness—outcomes determined entirely by player decisions, relationships, and order combinations.
|
||||
|
||||
### Negotiation-Driven
|
||||
Success requires verbal agreements, coordination, and alliance management. Most attacks succeed through pre-arranged support, not force. Deception and timing of betrayals are strategic elements.
|
||||
|
||||
### Simultaneous Orders Paradox
|
||||
All orders submitted before seeing others' moves—creates uncertainty, requires prediction, enables double-crosses. "Guess-the-room" meta-game where reading opponents' intentions is critical.
|
||||
|
||||
### Information Asymmetry
|
||||
Negotiations are private—what Power A tells you about Power B may be lies, truth, or strategic misdirection. Intelligence gathering through conversation cross-referencing is essential skill.
|
||||
|
||||
---
|
||||
|
||||
**Game Complexity:** Simple rules, deep strategic space. New players learn mechanics in 10 minutes; mastery requires understanding positional play, timing windows, psychology, and 7-player coalition dynamics.
|
||||
93
ai_diplomacy/agent_docs/instructions/game_info/notation.md
Normal file
93
ai_diplomacy/agent_docs/instructions/game_info/notation.md
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
# ORDER NOTATION REFERENCE
|
||||
**CRITICAL: Wrong notation = void order. One char wrong → order void → unit holds.**
|
||||
|
||||
## FORMAT: `[A/F] [3CODE] [ACTION] [TARGET]`
|
||||
**Spacing**: Single spaces. ✓ `A PAR - BUR` ✗ `A PAR-BUR`
|
||||
**Codes**: 3 uppercase. ✓ `PAR` ✗ `Paris`/`par`
|
||||
**Units**: `A`(Army) `F`(Fleet) prefix required
|
||||
**Actions**: `-`(move) `S`(support) `C`(convoy) `H`(hold) `B`(build) `D`(disband) `R`(retreat)
|
||||
|
||||
## ALL ORDER SYNTAX
|
||||
```
|
||||
Move: A PAR - BUR
|
||||
Hold: A PAR H
|
||||
Support Hold: A PAR S A MAR
|
||||
Support Move: A PAR S A MAR - BUR
|
||||
Convoy Army: A PAR - LON VIA
|
||||
Convoy Fleet: F ENG C A PAR - LON
|
||||
Build: A PAR B | F STP/NC B
|
||||
Disband: A PAR D
|
||||
Retreat: A BUD R GAL
|
||||
```
|
||||
|
||||
## COAST NOTATION
|
||||
**Dual-Coast**: STP(NC/SC), SPA(NC/SC), BUL(EC/SC)
|
||||
**Format**: `F [PROV]/[COAST]` - Use `/` not `-`/`_`
|
||||
**Adjacency**:
|
||||
- STP/NC: BAR,NWY,FIN | STP/SC: GOB,FIN,LVN
|
||||
- SPA/NC: MAO,GAS,POR | SPA/SC: WES,LYO,MAR
|
||||
- BUL/EC: BLA,CON,RUM | BUL/SC: AEG,GRE,CON
|
||||
|
||||
```
|
||||
F STP/NC - BAR | F GOB - STP/SC | F STP/NC B
|
||||
```
|
||||
|
||||
## COMMON ERRORS
|
||||
| Wrong | Right | Issue |
|
||||
|-------|-------|-------|
|
||||
| `A PAR- BUR` | `A PAR - BUR` | Missing spaces |
|
||||
| `PAR - BUR` | `A PAR - BUR` | Missing unit |
|
||||
| `A Paris - Bur` | `A PAR - BUR` | Use 3-letter codes |
|
||||
| `A PAR HOLD` | `A PAR H` | Use single letter |
|
||||
| `A PAR S MAR - BUR` | `A PAR S A MAR - BUR` | Missing unit in support |
|
||||
| `A PAR - LON` | `A PAR - LON VIA` | Missing VIA |
|
||||
| `F ENG CONVOY A PAR` | `F ENG C A PAR - LON` | Use C, include dest |
|
||||
| `F STP B` | `F STP/NC B` | Coast required |
|
||||
|
||||
## HOME CENTERS
|
||||
**Austria**: VIE(A), BUD(A), TRI(A/F) | **England**: LON(A/F), LVP(A/F), EDI(A/F) | **France**: PAR(A), MAR(A/F), BRE(A/F) | **Germany**: BER(A/F), KIE(A/F), MUN(A) | **Italy**: ROM(A/F), VEN(A/F), NAP(A/F) | **Russia**: MOS(A), STP(A/F), SEV(A/F), WAR(A) | **Turkey**: CON(A/F), ANK(A/F), SMY(A/F)
|
||||
|
||||
## VALIDATION
|
||||
Before submitting:
|
||||
- [ ] Format: `[A/F] [3CODE] [ACTION] [TARGET]` with single spaces
|
||||
- [ ] Unit type matches terrain (A=land, F=coastal/sea)
|
||||
- [ ] FROM/TO adjacent
|
||||
- [ ] Support matches exact move+coast
|
||||
- [ ] Convoy: army has VIA, fleet has `C A FROM - TO`, chain complete
|
||||
|
||||
## CRITICAL EXAMPLES
|
||||
|
||||
**Convoy (ALWAYS use VIA)**:
|
||||
```
|
||||
F BRE C A PAR - LON (fleet convoys)
|
||||
F ENG C A PAR - LON (fleet convoys)
|
||||
A PAR - LON VIA (army MUST use VIA)
|
||||
NEVER: A PAR - LON (invalid without VIA → VOID)
|
||||
```
|
||||
Both fleets convoy same route; army MUST include VIA.
|
||||
|
||||
**Support with coast**:
|
||||
```
|
||||
F BRE S F MAO - SPA/NC
|
||||
F MAO - SPA/NC
|
||||
```
|
||||
Support must match exact destination+coast.
|
||||
|
||||
**Coordinated attack**:
|
||||
```
|
||||
France: A MAR - BUR | A PAR S A MAR - BUR (strength 2)
|
||||
Germany: A BUR H (strength 1)
|
||||
Result: France wins, takes BUR
|
||||
```
|
||||
Support adds strength; must match exact move.
|
||||
|
||||
## NEVER
|
||||
- Omit spaces around separators
|
||||
- Use full names or lowercase
|
||||
- Forget VIA on convoyed armies
|
||||
- Support without matching exact destination
|
||||
- Build in non-home/occupied centers
|
||||
- Use `-` for coast (use `/`)
|
||||
- Move armies to seas or fleets to landlocked
|
||||
|
||||
**ONE CHAR WRONG = ORDER VOID**
|
||||
23
ai_diplomacy/agent_docs/instructions/game_info/unittypes.md
Normal file
23
ai_diplomacy/agent_docs/instructions/game_info/unittypes.md
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Unit Types
|
||||
|
||||
## Army vs Fleet
|
||||
|Property|Army (A)|Fleet (F)|
|
||||
|-|-|-|
|
||||
|Terrain|Land+Coast|Water+Coast|
|
||||
|Cannot Enter|Water (unless convoyed)|Landlocked|
|
||||
|Coast Notation|NEVER|MUST for SPA/STP/BUL|
|
||||
|Role|Inland centers|Sea control+Convoy|
|
||||
|
||||
## Coast Rules
|
||||
**Multi-coast:** SPA, STP, BUL (2 coasts each)
|
||||
- Fleets MUST specify: `F MAO - SPA/NC`✓|`F MAO - SPA`✗
|
||||
- No coast-switching: `F SPA/NC - SPA/SC`✗
|
||||
- Armies ignore coasts: `A SPA`
|
||||
- Support must specify: `F GAS S F MAO - SPA/NC`✓
|
||||
|
||||
<example>F STP/NC can convoy A MOS - NWY<reasoning>North coast borders Barents Sea, enabling convoy</reasoning></example>
|
||||
|
||||
## Validation
|
||||
- Unit type valid? (A: not water|F: not landlocked)
|
||||
- Coast specified for F→SPA/STP/BUL?
|
||||
- Convoy: All fleets issue C orders?
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
# Victory Conditions
|
||||
|
||||
**CRITICAL: NO DRAWS/TIES. SOLO VICTORY ONLY.**
|
||||
|
||||
Draws DISABLED (no voting/shared victories/negotiated endings). Exception: auto-draw Year 100 (failure). All alliances temporary; MUST betray to win. Ignore draw proposals (deception).
|
||||
|
||||
## Win & Elimination
|
||||
- **18 centers (of 34) = Solo Victory** (checked Fall only; 17 ≠ win)
|
||||
- **0 centers = Eliminated** (units disbanded, power removed)
|
||||
|
||||
## Victory Progress
|
||||
|
||||
| Centers | Phase | Action |
|
||||
|---------|-------|--------|
|
||||
| 1-9 | Early-Mid | Secure home/neutrals, build |
|
||||
| 10-17 | End-Game | Eliminate rivals, block coalitions, push 18 |
|
||||
| **18+** | **VICTORY** | **WON** |
|
||||
|
||||
**Each Fall:** Count centers → (18 - current) → Target vulnerable rival centers → Plan path to 18.
|
||||
|
||||
## Common Mistakes
|
||||
1. **Survival over victory** → Pursue 18 aggressively
|
||||
2. **Defensive when ahead** → Press advantage, finish eliminations
|
||||
3. **Poor coordination** → Complexity grows with units/centers, coordinate carefully
|
||||
|
||||
## Absolute Rules
|
||||
- ALWAYS prioritize center-gaining moves
|
||||
- ALWAYS eliminate weakened rivals completely
|
||||
- NEVER stop expanding
|
||||
- NEVER trust permanent alliances
|
||||
|
||||
**Every decision serves ONE purpose: Reach 18 centers. Target: Victory by Year 30.**
|
||||
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
# Adjustment Phase
|
||||
|
||||
## Rules
|
||||
|
||||
**Timing:** Post-Fall movement/retreats. Skip if units=centers.
|
||||
|
||||
**Formula:** Adjustment = Centers - Units (positive=BUILD, negative=DISBAND)
|
||||
|
||||
**Constraints:**
|
||||
- Build only in unoccupied home centers under control
|
||||
- Fleets coastal only (landlocked: VIE,PAR,MUN,MOS,WAR)
|
||||
- Occupied/lost homes: no build
|
||||
|
||||
## Home Centers
|
||||
|
||||
**AUS:** VIE,BUD,TRI | **ENG:** LON,LVP,EDI | **FRA:** PAR,BRE,MAR | **GER:** BER,KIE,MUN | **ITA:** ROM,NAP,VEN | **RUS:** MOS,STP,WAR,SEV | **TUR:** CON,ANK,SMY
|
||||
|
||||
**Dual-Coast:** STP/SPA/BUL need coast (NC/SC or EC/SC)
|
||||
|
||||
## Build Priorities
|
||||
|
||||
**CRITICAL:** Defend home vs adjacent threat (A VIE B if Italy in TYR)
|
||||
**HIGH:** Fill theater gap/alliance op (F EDI B if no North Sea units)
|
||||
**MEDIUM:** Offensive expansion (A MUN B → BOH)
|
||||
|
||||
**Rule:** Address highest priority first. Never LOW if CRITICAL/HIGH exists.
|
||||
|
||||
## Disband Priorities
|
||||
|
||||
**Keep:** Home defenders, front attackers (in/adjacent home centers)
|
||||
**Disband:** Isolated, trapped, orphaned (surrounded/far from theater)
|
||||
|
||||
## Syntax
|
||||
|
||||
```
|
||||
Build: [A/F] [LOC] B (ex: A PAR B, F STP/NC B)
|
||||
Disband: [A/F] [LOC] D (ex: A SIL D)
|
||||
None: {"orders": []}
|
||||
```
|
||||
|
||||
## Example
|
||||
|
||||
**France Build:**
|
||||
```
|
||||
6 centers (PAR,BRE,MAR,BEL,HOL,SPA), 4 units → 2 builds
|
||||
Available: PAR (A only), BRE (A/F) | MAR occupied
|
||||
Threat: Germany adjacent
|
||||
Orders: A PAR B, F BRE B
|
||||
Why: PAR defends home, BRE controls Atlantic
|
||||
```
|
||||
|
||||
**Validation:** Builds ≤ unoccupied homes, disbands match requirement, fleets coastal, dual-coast specified
|
||||
88
ai_diplomacy/agent_docs/instructions/phases/movementphase.md
Normal file
88
ai_diplomacy/agent_docs/instructions/phases/movementphase.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
# Movement Phase Instructions
|
||||
|
||||
## PHASE OBJECTIVE
|
||||
Issue orders maximizing territorial control and supply center gains. Orders execute simultaneously. Spring/Fall twice yearly; builds only in Adjustment Phase.
|
||||
|
||||
## ORDER TYPES
|
||||
Hold (H): `A PAR H` - +1 defense auto. Use when defending.
|
||||
|
||||
Move (-): `A PAR - BUR` - Adjacency required (armies: land; fleets: coast/sea). Strength = 1 + supports. Attack > Defense = success; equal = bounce.
|
||||
|
||||
Support (S): `A MAR S A PAR - BUR` (move) or `F NTH S F LON` (hold)
|
||||
- Adjacent to DESTINATION (not origin)
|
||||
- CUT if: Enemy moves to supporter OR supporter dislodged (except unit being supported against)
|
||||
- Must EXACTLY match supported order
|
||||
|
||||
Convoy (C): `F ENG C A LON - BRE` - Multi-zone: ALL fleets must convoy same army. Fails if ANY fleet dislodged.
|
||||
|
||||
Dual-Coast: STP/SPA/BUL - ALWAYS specify NC/SC/EC: `F STP/NC - NWY`
|
||||
|
||||
## RESOLUTION
|
||||
Simultaneity: All orders resolve at once. Equal strength = bounce; higher = dislodge.
|
||||
|
||||
Strength: 1 + supports (uncut); Hold gets +1 auto.
|
||||
|
||||
Support Cutting: Enemy moving to supporter = CUT (except against supported unit; dislodged attackers don't cut).
|
||||
|
||||
Beleaguered Garrison: Attacks from different origins don't combine.
|
||||
|
||||
## CRITICAL EXAMPLES
|
||||
|
||||
Example: Multi-Fleet Convoy
|
||||
England LON-BEL: `A LON - BEL`, `F NTH C A LON - BEL`, `F ENG C A LON - BEL`
|
||||
Reasoning: ALL fleets must convoy same army to same destination. If ANY fleet dislodged, convoy fails. Verify each fleet adjacent to route.
|
||||
End
|
||||
|
||||
Example: Support Cut
|
||||
France: `A PIC - BEL` + `A BUR S A PIC - BEL` (2v1). Germany: `A RUH - BUR` (cuts). Result: 1v1 bounce.
|
||||
Reasoning: Attacking supporter cuts support, reducing 2v1 to 1v1. Often more effective than attacking target.
|
||||
End
|
||||
|
||||
Example: Head-to-Head with Support
|
||||
Austria: `A VIE - GAL` + `A BOH S A VIE - GAL` (2). Russia: `A GAL - VIE` (1). Result: Austria takes GAL.
|
||||
Reasoning: 2v1 beats 1v1 in head-to-head. Without support would bounce (1v1). Support makes difference.
|
||||
End
|
||||
|
||||
Example: Beleaguered Garrison
|
||||
Belgium: `F BEL H` + `F NTH S F BEL` (2). Attackers: `A PIC - BEL` (1) + `A HOL - BEL` (1). Result: BEL holds.
|
||||
Reasoning: Attacks from different origins don't combine. Defender compares strength against each individually (2>1 each).
|
||||
End
|
||||
|
||||
Example: Support Validation
|
||||
Germany: `A MUN - VIE`. Russia: `A WAR S A MUN - VIE` (valid: WAR adjacent to VIE) + `A GAL - BOH` (cuts Austrian support).
|
||||
Reasoning: Supporter adjacent to destination (VIE), not origin (MUN). Third unit cuts enemy support vs overkill 3v1.
|
||||
End
|
||||
|
||||
## STRATEGY
|
||||
Per-Unit: Threatened? = move/hold with support. Can capture? = 2v1 if defended. Critical support? = ally attack/defend. Else reposition.
|
||||
|
||||
Risk: Low = neutral/2+ supports. Medium = 2v1/contested. High = unsupported vs defended/unconfirmed ally/undefended home.
|
||||
|
||||
Mitigation: Confirm ally orders; redundant supports; fallback plans.
|
||||
|
||||
Stage: Early: capture neutrals. Mid: 10-12 centers. Late: 14+ = 18; <8 = survive.
|
||||
|
||||
## VALIDATION
|
||||
Format: `[A/F] [PROVINCE] [H/-/S/C] [DESTINATION]`; dual-coast `/NC`/`/SC`/`/EC`; supports match EXACTLY
|
||||
|
||||
Legality: Adjacent only; no fleets landlocked, no armies to sea (unless convoy); one order per unit; from possible_orders
|
||||
|
||||
Strategy: Home centers defended; critical moves confirmed; high-risk with fallbacks
|
||||
|
||||
Submit: `{"orders": ["A PAR - BUR", "A MAR S A PAR - BUR"]}`
|
||||
|
||||
## MAXIMS
|
||||
1. Support is king - most attacks need it
|
||||
2. Defend home centers - need for builds
|
||||
3. 2v1 beats 1v1 - usually sufficient
|
||||
4. Cut support when can't win direct
|
||||
5. Convoys fragile - ANY fleet dislodged = fails
|
||||
6. Simultaneous means simultaneous
|
||||
7. Trust but verify - written confirmation
|
||||
|
||||
## RULES
|
||||
ALWAYS: Use possible_orders; validate syntax; confirm ally supports; plan simultaneous; defend home centers
|
||||
|
||||
NEVER: Order units you don't control; rely on unconfirmed ally support; leave all home centers undefended; multiple orders per unit; assume sequential
|
||||
|
||||
Execute precisely, coordinate carefully, validate thoroughly.
|
||||
30
ai_diplomacy/agent_docs/instructions/phases/retreatphase.md
Normal file
30
ai_diplomacy/agent_docs/instructions/phases/retreatphase.md
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
# Retreat Phase
|
||||
|
||||
Process dislodged units post-Movement. Issue retreat/disband using **ONLY** `possible_orders`.
|
||||
|
||||
## Decision Tree
|
||||
|
||||
Per unit: (1) Valid retreats in `possible_orders`? NO→disband, YES→evaluate; (2) Priority: supply center>tactical flexibility>front line>safety; (3) RETREAT: defends centers/rejoins front/units≤centers. DISBAND: surplus (units>centers), trapped, enemy destination.
|
||||
|
||||
## Syntax
|
||||
```
|
||||
[Unit] [Location] R [Destination] | [Unit] [Location] D
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
**NEVER:** Attacker origin, occupied province (Movement start), contested (stand-off), wrong terrain (A→sea, F→inland), multiple orders/unit, orders outside `possible_orders`, omit dislodged.
|
||||
|
||||
**ALWAYS:** 1 order/unit, use only `possible_orders`, check unit/center balance, return `{"orders": ["order1", "order2"]}`.
|
||||
|
||||
## Example: Surplus
|
||||
|
||||
**Russia:** 7 units, 5 centers (2 surplus). A WAR dislodged (UKR threatened, LVN isolated). F SEV dislodged (ARM available).
|
||||
|
||||
```json
|
||||
{"orders": ["A WAR D", "F SEV R ARM"]}
|
||||
```
|
||||
|
||||
<reasoning>
|
||||
7v5→2 disbands needed. A WAR poor retreats→disband. F SEV→ARM keeps Black Sea naval. Choose disbands vs random Adjustment.
|
||||
</reasoning>
|
||||
78
tools/count_tokens.py
Executable file
78
tools/count_tokens.py
Executable file
|
|
@ -0,0 +1,78 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Token counter utility for markdown files.
|
||||
Uses tiktoken with cl100k_base encoding (GPT-4/Claude compatible).
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
import tiktoken
|
||||
|
||||
|
||||
def count_tokens(text: str, encoding_name: str = "cl100k_base") -> int:
|
||||
"""Count tokens in text using specified encoding."""
|
||||
encoding = tiktoken.get_encoding(encoding_name)
|
||||
return len(encoding.encode(text))
|
||||
|
||||
|
||||
def process_file(file_path: Path) -> tuple[str, int, int]:
|
||||
"""Process a single file and return stats."""
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
char_count = len(content)
|
||||
token_count = count_tokens(content)
|
||||
|
||||
return file_path.name, char_count, token_count
|
||||
|
||||
|
||||
def format_output(filename: str, char_count: int, token_count: int) -> str:
|
||||
"""Format output line with stats."""
|
||||
ratio = token_count / char_count if char_count > 0 else 0
|
||||
return f"{filename}: {char_count:,} chars, {token_count:,} tokens ({ratio:.2f} tokens/char)"
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: count_tokens.py <file_or_directory>", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
path = Path(sys.argv[1])
|
||||
|
||||
if not path.exists():
|
||||
print(f"Error: {path} does not exist", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
total_chars = 0
|
||||
total_tokens = 0
|
||||
|
||||
# Collect files to process
|
||||
if path.is_file():
|
||||
files = [path]
|
||||
elif path.is_dir():
|
||||
files = sorted(path.glob("*.md"))
|
||||
if not files:
|
||||
print(f"No .md files found in {path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
else:
|
||||
print(f"Error: {path} is neither a file nor directory", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Process each file
|
||||
for file_path in files:
|
||||
try:
|
||||
filename, char_count, token_count = process_file(file_path)
|
||||
print(format_output(filename, char_count, token_count))
|
||||
total_chars += char_count
|
||||
total_tokens += token_count
|
||||
except Exception as e:
|
||||
print(f"Error processing {file_path}: {e}", file=sys.stderr)
|
||||
|
||||
# Print total if multiple files
|
||||
if len(files) > 1:
|
||||
print(f"TOTAL: {total_chars:,} chars, {total_tokens:,} tokens")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue