mirror of
https://github.com/GoodStartLabs/AI_Diplomacy.git
synced 2026-04-19 12:58:09 +00:00
326 lines
16 KiB
Text
326 lines
16 KiB
Text
# AI Diplomacy Codebase Analysis (Core Logic Modules) - Comprehensive Update
|
|
|
|
This document provides an analysis of key Python modules within the `ai_diplomacy` package, focusing on their roles, functions, interdependencies, and implementation status.
|
|
|
|
**Last Major Update**: January 2025 - Added diary system details, consolidation logic, and comprehensive agent memory management.
|
|
|
|
---
|
|
|
|
## 1. Module Status
|
|
|
|
### COMPLETED MODULES:
|
|
|
|
#### 1.1. `game_history.py` (COMPLETE)
|
|
**Goal:** To structure, store, and retrieve the historical events of a Diplomacy game phase by phase, including messages, plans, orders, and results.
|
|
**Status:** Fully implemented and operational.
|
|
|
|
*Key Components:*
|
|
* `DiplomacyGraph`: Represents map territory connectivity with support for unit-specific movement rules (Army vs Fleet).
|
|
* `bfs_shortest_path`: Finds shortest path from a starting territory to any territory matching criteria.
|
|
* `bfs_nearest_adjacent`: Finds shortest path to a territory adjacent to any territory in a target set.
|
|
* `build_diplomacy_graph`: Constructs the graph representation from the game map.
|
|
|
|
#### 1.3. `phase_summary.py` (COMPLETE, in lm_game.py)
|
|
**Goal:** Generate concise, structured summaries of each game phase for post-game analysis.
|
|
**Status:** Fully implemented via `phase_summary_callback` in `lm_game.py`.
|
|
|
|
**Key Components:**
|
|
* Structured summaries with:
|
|
* Current board state (sorted by supply center count)
|
|
* Successful moves by power
|
|
* Unsuccessful moves by power with failure reasons
|
|
* Optional sections for other move types
|
|
|
|
#### 1.4. `agent.py` (COMPLETE WITH ENHANCED MEMORY SYSTEM)
|
|
**Goal:** To maintain stateful agent representation with personality, goals, relationships, and sophisticated memory management.
|
|
**Status:** Fully implemented with dual memory system (journal + diary) and yearly consolidation.
|
|
|
|
**Key Components:**
|
|
* `DiplomacyAgent` class with:
|
|
* `power_name`: The power this agent represents
|
|
* `goals`: List of strategic goals, dynamically updated each phase
|
|
* `relationships`: Dict tracking relationships (Enemy/Unfriendly/Neutral/Friendly/Ally) with other powers
|
|
* `private_journal`: Unstructured internal logs for debugging/events
|
|
* `private_diary`: Structured, phase-prefixed strategic summaries - THE MAIN MEMORY SYSTEM
|
|
* `client`: BaseModelClient instance for LLM interactions
|
|
* Power-specific system prompts loaded from `prompts/{power_name}_system_prompt.txt`
|
|
|
|
**Memory System Architecture:**
|
|
1. **Private Journal** (Less Important):
|
|
* Simple string entries via `add_journal_entry()`
|
|
* Used for: "Goals updated", "Relationship changed", "Message sent", etc.
|
|
* Not directly fed to LLMs
|
|
* More like internal debug logs
|
|
|
|
2. **Private Diary** (Critical for Decision Making):
|
|
* Structured entries with phase prefix via `add_diary_entry(entry, phase)`
|
|
* Three types of diary entries generated each phase:
|
|
- **Negotiation Diary** (`generate_negotiation_diary_entry`): Analyzes messages, updates relationships
|
|
- **Order Diary** (`generate_order_diary_entry`): Records strategic reasoning behind orders
|
|
- **Phase Result Diary** (`generate_phase_result_diary_entry`): Analyzes outcomes, detects betrayals
|
|
* Fed to LLMs via `format_private_diary_for_prompt(max_entries=40)`
|
|
* **Yearly Consolidation** (`consolidate_year_diary_entries`):
|
|
- Triggered 2 years after a given year (e.g., in S1903M, consolidate 1901)
|
|
- Uses Gemini Flash to summarize all entries from a year into one concise entry
|
|
- Replaces individual entries with `[CONSOLIDATED 1901] summary...`
|
|
- Prevents context bloat while preserving key memories
|
|
|
|
**Advanced JSON Parsing:**
|
|
* `_extract_json_from_text`: Multi-strategy parser handling various LLM output formats
|
|
* Handles ```json blocks, {{...}} blocks, PARSABLE OUTPUT: formats
|
|
* Uses json5 and json_repair as fallbacks
|
|
* Aggressive cleaning for common LLM formatting issues
|
|
* Returns empty dict on failure rather than crashing
|
|
|
|
**State Update System:**
|
|
* `analyze_phase_and_update_state`: Async method that:
|
|
* Uses phase summaries and board state
|
|
* Updates goals based on game evolution
|
|
* Adjusts relationships based on actions (not just words)
|
|
* Validates all updates with allowed values
|
|
* Has robust error handling with fallback to current state
|
|
|
|
**Integration Points:**
|
|
* Diary provides historical context for all LLM calls
|
|
* Relationships influence negotiation tone and alliance decisions
|
|
* Goals drive strategic planning and order generation
|
|
* Power-specific prompts shape agent personality
|
|
* All LLM interactions logged via `utils.log_llm_response`
|
|
|
|
#### 1.5. `negotiations.py` (COMPLETE)
|
|
**Goal:** To orchestrate the communication phase among active AI powers.
|
|
**Status:** Fully implemented and integrated with DiplomacyAgent state.
|
|
**Note:** Relies heavily on `prompts/conversation_instructions.txt` to guide LLMs in generating correctly formatted messages for parsing.
|
|
|
|
#### 1.6. `planning.py` (COMPLETE)
|
|
**Goal:** To allow each AI power to generate a high-level strategic directive or plan.
|
|
**Status:** Fully implemented and integrated with DiplomacyAgent state.
|
|
|
|
#### 1.7. `utils.py` (COMPLETE)
|
|
**Goal:** To provide common utility functions used across other AI diplomacy modules.
|
|
**Status:** Fully implemented.
|
|
|
|
#### 1.8. `clients.py` (COMPLETE)
|
|
**Goal:** To abstract and manage interactions with various LLM APIs.
|
|
**Status:** Fully implemented with agent state integration (including personality, goals, relationships, and the new `private_diary` for summarized history). It now also leverages `possible_order_context.py` for richer order details in prompts.
|
|
**Note:** Uses various files in `prompts/` (e.g., `context_prompt.txt`, `order_instructions.txt`, `negotiation_diary_prompt.txt`, `order_diary_prompt.txt`) to structure LLM requests. `context_prompt.txt` has been updated to use `agent_private_diary` for history and a more structured `{possible_orders}` section generated by `possible_order_context.generate_rich_order_context`.
|
|
|
|
#### 1.9. `initialization.py` (NEWLY ADDED & COMPLETE)
|
|
**Goal:** To perform the initial LLM-driven setup of an agent's goals and relationships at the very start of the game (Spring 1901).
|
|
**Status:** Fully implemented and integrated into `lm_game.py`.
|
|
|
|
**Key Components:**
|
|
* `initialize_agent_state_ext(agent: DiplomacyAgent, game: Game, game_history: GameHistory, log_file_path: str)`: An asynchronous function that:
|
|
* Constructs a specific prompt tailored for Spring 1901, asking for initial goals and relationships.
|
|
* Utilizes the agent's client (`agent.client`) and the `run_llm_and_log` utility for the LLM interaction.
|
|
* Parses the JSON response using the agent's `_extract_json_from_text` method.
|
|
* Directly updates the `agent.goals` and `agent.relationships` attributes with the LLM's suggestions or defaults if parsing fails.
|
|
|
|
**Integration Points:**
|
|
* Called once per agent from `lm_game.py` immediately after the `DiplomacyAgent` object is instantiated and before the main game loop begins.
|
|
|
|
#### 1.10. `possible_order_context.py` (COMPLETE)
|
|
**Goal:** To generate rich, strategic context about possible orders including pathfinding and territory analysis.
|
|
**Status:** Fully implemented and integrated into order generation prompts.
|
|
|
|
**Key Components:**
|
|
* `build_diplomacy_graph`: Creates adjacency graph with unit-specific movement rules
|
|
* `bfs_shortest_path`: Finds shortest paths considering unit movement restrictions
|
|
* `get_nearest_enemy_units`: Identifies closest threats using BFS
|
|
* `get_nearest_uncontrolled_scs`: Finds expansion opportunities
|
|
* `get_adjacent_territory_details`: Analyzes neighboring territories and support possibilities
|
|
* `generate_rich_order_context`: Main function creating structured XML context for each orderable unit
|
|
|
|
**Integration:**
|
|
* Called by `prompt_constructor.py` to enhance order generation prompts
|
|
* Provides strategic insights like "nearest enemy 3 moves away" or "undefended SC 2 territories north"
|
|
|
|
#### 1.11. `prompt_constructor.py` (COMPLETE)
|
|
**Goal:** To centralize and standardize prompt construction for all LLM interactions.
|
|
**Status:** Fully implemented, replacing scattered prompt building logic.
|
|
|
|
**Key Functions:**
|
|
* `build_context_prompt`: Creates comprehensive game context including:
|
|
* Board state, unit positions, supply centers
|
|
* Power-specific goals and relationships
|
|
* Recent game history from agent's diary
|
|
* Strategic analysis from `possible_order_context`
|
|
* `construct_order_generation_prompt`: Specialized for order generation with:
|
|
* Full context from `build_context_prompt`
|
|
* Detailed possible orders with strategic annotations
|
|
* Order validation rules and format requirements
|
|
|
|
#### 1.12. `narrative.py` (PLACEHOLDER/FUTURE)
|
|
**Goal:** To generate narrative descriptions of game events for visualization/entertainment.
|
|
**Status:** Referenced in imports but not yet implemented.
|
|
**Potential Features:**
|
|
* Convert dry game events into dramatic narratives
|
|
* Generate "news reports" of battles and diplomatic developments
|
|
* Create power-specific propaganda/spin on events
|
|
|
|
---
|
|
|
|
## 2. Integration Points
|
|
|
|
The following connections have been established:
|
|
|
|
1. **Initial Agent Setup (New)**:
|
|
* `lm_game.py` calls `initialization.py`'s `initialize_agent_state_ext` for each agent. This function uses an LLM call to populate the agent's initial `goals` and `relationships` before the main game loop and other agent interactions commence.
|
|
|
|
2. **Agent State → Context Building**
|
|
* `BaseModelClient.build_context_prompt` in `clients.py` incorporates the agent's current `goals`, `relationships`, and the concise `agent_private_diary` for historical context.
|
|
* It also calls `possible_order_context.generate_rich_order_context` to provide a detailed and strategically relevant breakdown of possible orders, replacing a simpler list.
|
|
* `prompts/context_prompt.txt` is formatted to accept these inputs, including the structured possible orders and the agent's private diary.
|
|
|
|
3. **Agent State → Negotiations**
|
|
* Agent's personality, goals, and relationships influence message generation
|
|
* Relationships are updated based on negotiation context and results
|
|
|
|
4. **Robust LLM Interaction**
|
|
* Implemented multi-strategy JSON extraction to handle various LLM response formats
|
|
* Added case-insensitive validation for power names and relationship statuses
|
|
* Created fallback mechanisms for all LLM interactions
|
|
|
|
5. **Error Recovery**
|
|
* Added defensive programming throughout agent state updates
|
|
* Implemented progressive fallback strategies for parsing LLM outputs
|
|
* Used intelligent defaults to maintain consistent agent state
|
|
|
|
---
|
|
|
|
## 3. Main Game Loop Flow (`lm_game.py`)
|
|
|
|
The main game loop orchestrates all components in a sophisticated async flow:
|
|
|
|
1. **Initialization Phase**:
|
|
* Create `DiplomacyAgent` instances for each power
|
|
* Load power-specific system prompts
|
|
* Run `initialize_agent_state_ext` concurrently for all agents
|
|
* Set up logging infrastructure
|
|
|
|
2. **Per-Phase Flow** (Movement phases only for negotiations):
|
|
* **Negotiations** (if enabled):
|
|
- Round-robin message generation via `conduct_negotiations`
|
|
- Async generation of messages considering relationships
|
|
- Updates to `game_history` with all messages
|
|
* **Planning** (if enabled):
|
|
- Strategic directive generation via `planning_phase`
|
|
- Considers current goals and game state
|
|
* **Diary Generation** (Negotiation):
|
|
- Each agent generates negotiation diary entry
|
|
- Relationships may be updated based on messages
|
|
* **Order Generation**:
|
|
- Concurrent order generation for all powers
|
|
- Rich context from `possible_order_context`
|
|
- Validation and fallback logic
|
|
* **Order Diary**:
|
|
- Strategic reasoning recorded for chosen orders
|
|
* **Phase Processing**:
|
|
- Orders executed by Diplomacy engine
|
|
- Custom phase summary generated
|
|
* **Phase Result Diary**:
|
|
- Analysis of outcomes vs. expectations
|
|
- Betrayal detection
|
|
* **State Updates**:
|
|
- Goals and relationships updated based on results
|
|
* **Diary Consolidation** (every 2 years):
|
|
- Old diary entries summarized to manage context
|
|
|
|
3. **Error Handling**:
|
|
* Comprehensive error tracking in `model_error_stats`
|
|
* All LLM calls wrapped with retry logic
|
|
* Fallback orders for any failures
|
|
|
|
## 4. Future Work
|
|
|
|
1. **Enhanced Memory Management**
|
|
* Implement selective memory retrieval based on relevance
|
|
* Add episodic memory for specific important events
|
|
* Create power-specific memory strategies
|
|
|
|
2. **Advanced Negotiation Strategies**
|
|
* Implement deception detection algorithms
|
|
* Add negotiation pattern recognition
|
|
* Create coalition formation logic
|
|
|
|
3. **Performance Optimizations**
|
|
* Cache common pathfinding results
|
|
* Implement speculative order generation
|
|
* Add early termination for losing positions
|
|
|
|
4. **Analysis Tools**
|
|
* Post-game relationship evolution visualization
|
|
* Betrayal/alliance pattern analysis
|
|
* Goal achievement tracking
|
|
|
|
5. **UI Integration**
|
|
* Real-time diary viewer
|
|
* Relationship graph visualization
|
|
* Goal tracking dashboard
|
|
|
|
---
|
|
|
|
## 5. Dependency Map (Comprehensive Update)
|
|
|
|
```ascii
|
|
+----------------+
|
|
| lm_game.py |
|
|
| (Main Loop) |
|
|
+-------+--------+
|
|
|
|
|
+-----------------------------+-----------------------------+
|
|
| | |
|
|
v v v
|
|
+-------+--------+ +-------+--------+ +-------+--------+
|
|
|initialization.py| | agent.py | |game_history.py |
|
|
|(Initial Setup) | |(State & Memory)| |(Phase Storage) |
|
|
+----------------+ +-------+--------+ +----------------+
|
|
|
|
|
+-----------------------------+-----------------------------+
|
|
| | |
|
|
v v v
|
|
+-------+--------+ +-------+--------+ +-------+--------+
|
|
|negotiations.py | | planning.py | | utils.py |
|
|
|(Messages) | |(Strategy) | |(Logging/Errors)|
|
|
+-------+--------+ +-------+--------+ +-------+--------+
|
|
| | |
|
|
+-----------------------------+-----------------------------+
|
|
|
|
|
+-------v--------+
|
|
| clients.py |
|
|
|(LLM Interface) |
|
|
+-------+--------+
|
|
|
|
|
+-----------------------------+-----------------------------+
|
|
| | |
|
|
v v v
|
|
+-------+--------+ +-------+--------+ +-------+--------+
|
|
|prompt_constructor| |possible_order_ | | prompts/ |
|
|
|(Prompt Building)| |context.py | |(Templates) |
|
|
+----------------+ |(BFS/Analysis) | +----------------+
|
|
+----------------+
|
|
```
|
|
|
|
**Key Relationships:**
|
|
* `lm_game.py` orchestrates everything, managing the game loop and agent lifecycle
|
|
* `agent.py` is the central state holder, maintaining goals, relationships, and memories
|
|
* `clients.py` abstracts all LLM interactions with multiple provider support
|
|
* `prompt_constructor.py` and `possible_order_context.py` work together for rich prompts
|
|
* `utils.py` provides critical infrastructure for logging and error handling
|
|
* All async operations use `asyncio.gather` for maximum concurrency
|
|
|
|
**Current Integration Status:**
|
|
* All modules fully implemented and battle-tested
|
|
* Sophisticated error handling throughout
|
|
* Async architecture provides 5-10x performance improvement
|
|
* Memory management prevents context overflow
|
|
* Robust parsing handles various LLM output formats
|
|
|
|
**Recent Enhancements (January 2025):**
|
|
- Added comprehensive diary system with three entry types per phase
|
|
- Implemented yearly diary consolidation to manage context size
|
|
- Enhanced relationship tracking with ignored message detection
|
|
- Added phase result analysis for betrayal detection
|
|
- Integrated BFS pathfinding for strategic order context
|
|
- Created centralized prompt construction system
|
|
- Added power-specific system prompts for personality
|