mirror of
https://github.com/NousResearch/atropos.git
synced 2026-04-19 12:57:58 +00:00
Complete quantum-hybrid environment submission
- README.md: Updated documentation with WandB links and metrics - requirements.txt: Updated Python dependencies - atropos.py: Main environment implementation - atopos_quant.py: Additional quantum environment module - quantum_hybrid_artifacts.tar.gz: Compressed training artifacts
This commit is contained in:
parent
39415d299e
commit
c4a972d433
3 changed files with 688 additions and 18 deletions
|
|
@ -1,3 +1,91 @@
|
|||
Quantum-Classical Hybrid Language Model Environment
|
||||
A novel Atropos environment that trains quantum-enhanced language models by combining classical transformers with quantum circuits using PennyLane and PyTorch.
|
||||
Overview
|
||||
This environment implements a quantum-classical hybrid architecture for next-word prediction, trained on high-quality text generated by Hermes-3-70B. The key innovation is using quantum circuits to enhance traditional neural networks for language modeling tasks.
|
||||
Research Question
|
||||
Can quantum circuits provide advantages over purely classical approaches in natural language processing tasks?
|
||||
Architecture
|
||||
Data Flow
|
||||
Input Prompts → Hermes-3-70B (text generation) → Hybrid Model Training → Quantum-Enhanced Predictions
|
||||
Hybrid Model Components
|
||||
|
||||
Classical Pathway: Standard transformer-style neural network head
|
||||
Quantum Pathway:
|
||||
|
||||
Dimensionality reduction: 768D → 8D (quantum space)
|
||||
Two quantum circuit layers with parameterized gates
|
||||
Quantum-to-vocabulary mapping: 8D → 50K vocab
|
||||
|
||||
|
||||
Learnable Mixing: Parameter α balances classical vs quantum contributions
|
||||
|
||||
Quantum Circuit Design
|
||||
|
||||
8 qubits with 3 parameterized layers
|
||||
RY rotation gates for classical data encoding
|
||||
CNOT gates creating entanglement patterns
|
||||
Pauli-Z measurements for classical output extraction
|
||||
|
||||
Installation & Setup
|
||||
Prerequisites
|
||||
bash# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Atropos framework (follow official guide)
|
||||
Environment Setup
|
||||
bashexport ATROPOS_HERMES_API_KEY="your-nous-research-api-key"
|
||||
Quickstart
|
||||
Basic Training
|
||||
bashpython atropos.py process
|
||||
View Results
|
||||
Monitor training at: https://wandb.ai/your-username/atropos-environments_hack0_env_quant
|
||||
Custom Configuration
|
||||
bashpython atropos.py process \
|
||||
--env.n_qubits 16 \
|
||||
--env.n_layers 5 \
|
||||
--env.total_steps 100 \
|
||||
--env.quantum_weight 0.5
|
||||
Environment Design & Motivation
|
||||
Why Quantum-Classical Hybrid?
|
||||
|
||||
Pattern Recognition: Quantum circuits may capture linguistic patterns that classical networks miss
|
||||
Entanglement: Natural language has complex interdependencies that quantum entanglement might model better
|
||||
Optimization Landscape: Quantum interference could provide novel optimization pathways
|
||||
Knowledge Distillation: Transfer capabilities from large models (Hermes-3-70B) to smaller quantum-enhanced models
|
||||
|
||||
Training Strategy
|
||||
The environment employs quantum-enhanced knowledge distillation:
|
||||
|
||||
Teacher Model: Hermes-3-70B generates diverse, high-quality responses
|
||||
Student Model: Hybrid quantum-classical model learns next-word prediction
|
||||
Comparison: Direct evaluation of quantum vs classical pathways within the same model
|
||||
Optimization: Both classical and quantum parameters trained via gradient descent
|
||||
|
||||
Results & Metrics
|
||||
Live Experiment
|
||||
🚀 View our latest run: WandB Dashboard: https://wandb.ai/quaintanceai-nous/atropos-environments_hack0_env_quant?nw=nwuserquaintanceai
|
||||
Key Metrics Explained
|
||||
Training Metrics
|
||||
|
||||
train/hybrid_loss: Combined quantum-classical model loss
|
||||
train/classical_loss: Baseline classical-only model loss
|
||||
train/quantum_loss: Quantum-specific loss component
|
||||
train/alpha_value: Mixing parameter (0 = full quantum, 1 = full classical)
|
||||
|
||||
Evaluation Metrics
|
||||
|
||||
eval/hybrid_performance: Entropy-based comparison of hybrid vs classical predictions
|
||||
eval/quantum_weight: Current quantum contribution (1 - α)
|
||||
train/quantum_coherence: Measure of quantum circuit effectiveness
|
||||
|
||||
|
||||
|
||||
root@e6b2755c1bc0:/workspace/atropos/environments/hack0/env_quant# rm README.md
|
||||
root@e6b2755c1bc0:/workspace/atropos/environments/hack0/env_quant# nano README.md
|
||||
root@e6b2755c1bc0:/workspace/atropos/environments/hack0/env_quant# nano README.md
|
||||
root@e6b2755c1bc0:/workspace/atropos/environments/hack0/env_quant# nano README.md
|
||||
root@e6b2755c1bc0:/workspace/atropos/environments/hack0/env_quant# nano requirements.txt
|
||||
root@e6b2755c1bc0:/workspace/atropos/environments/hack0/env_quant# cat requirements.txt
|
||||
# Core dependencies
|
||||
torch>=2.0.0
|
||||
numpy>=1.21.0
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue