openadapt-consilium¶
Auto-generated from OpenAdaptAI/openadapt-consilium. Last synced: 2026-03-04 01:23 UTC
Consilium¶
Multi-LLM council for consensus-driven AI responses.
Consilium queries multiple LLMs in parallel, has each model review the others' responses, then synthesizes the best answer through a chairman model. Inspired by Karpathy's llm-council.
How It Works¶
┌─────────────────────────────────────────────────────────┐
│ STAGE 1: QUERY │
│ │
│ ┌─────────┐ ┌──────────┐ ┌─────────────────┐ │
│ │ GPT-5.2 │ │ Claude │ │ Gemini 3.1 Pro │ │
│ │ │ │ Sonnet │ │ │ │
│ └────┬────┘ └────┬─────┘ └────────┬────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ Response A Response B Response C │
└─────────────────────┬───────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────┐
│ STAGE 2: REVIEW │
│ │
│ Each model reviews all anonymized responses. │
│ Ranks them best → worst with reasoning. │
└─────────────────────┬───────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────┐
│ STAGE 3: SYNTHESIS │
│ │
│ Chairman model synthesizes the best answer from │
│ all responses + reviews. │
│ │
│ ┌──────────────────┐ │
│ │ Final Answer │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────┘
Installation¶
pip install openadapt-consilium
Or for development:
git clone https://github.com/OpenAdaptAI/openadapt-consilium.git
cd openadapt-consilium
pip install -e ".[dev]"
Quick Start¶
Set your API keys:
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GOOGLE_API_KEY="AI..."
Python API¶
from consilium import Council
council = Council()
result = council.ask("What are the key differences between REST and GraphQL?")
print(result.final_answer)
print(result.cost_summary())
CLI¶
consilium "What are the key differences between REST and GraphQL?"
API Reference¶
Council¶
The main orchestrator class.
from consilium import Council
council = Council(
models=["gpt-5.2", "claude-sonnet-4-6", "gemini-3.1-pro"],
chairman="claude-sonnet-4-6",
max_workers=8,
)
Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
models |
list[str] |
["gpt-5.2", "claude-sonnet-4-6", "gemini-3.1-pro"] |
Models to query in Stage 1 |
chairman |
str |
"claude-sonnet-4-6" |
Model for Stage 3 synthesis |
max_workers |
int |
8 |
Max parallel threads |
council.ask()¶
result = council.ask(
"Your question here",
images=[open("screenshot.png", "rb").read()], # optional
budget=0.50, # max USD spend
system="Be concise", # system prompt for all models
skip_review=False, # skip Stages 2-3
json_schema={...}, # request JSON output
)
Returns: CouncilResult
| Attribute | Type | Description |
|---|---|---|
result.final_answer |
str |
Synthesized best answer |
result.individual_responses |
list |
Each model's Stage 1 response |
result.reviews |
list |
Each model's Stage 2 review |
result.total_cost |
float |
Total estimated cost in USD |
result.total_latency_seconds |
float |
Wall-clock time |
result.cost_summary() |
str |
Pretty-printed cost breakdown |
Agent SDK¶
For AI agents and automated pipelines, use the dict-based interface:
from consilium import council_query
result = council_query(
"Analyze this screenshot and list the UI elements",
images=[screenshot_bytes],
models=["gpt-5.2", "claude-sonnet-4-6"],
budget=0.25,
skip_review=True, # fast mode: Stage 1 only
)
print(result["final_answer"])
print(result["cost"]["total_usd"])
Returns: JSON-serializable dict with keys:
{
"final_answer": "...",
"individual_responses": [
{
"model": "openai/gpt-5.2",
"text": "...",
"latency_seconds": 2.1,
"input_tokens": 1500,
"output_tokens": 400,
"cost_usd": 0.007
}
],
"reviews": [...],
"cost": {
"breakdown": {"gpt-5.2": 0.007, "claude-sonnet-4-6": 0.012},
"total_usd": 0.019,
"total_input_tokens": 3000,
"total_output_tokens": 800
},
"total_latency_seconds": 3.2
}
CLI Reference¶
consilium "prompt" [OPTIONS]
Options:
--models TEXT Comma-separated model IDs (default: gpt-5.2,claude-sonnet-4-6,gemini-3.1-pro)
--chairman TEXT Chairman model for synthesis (default: claude-sonnet-4-6)
--image PATH Image file to include (repeatable)
--budget FLOAT Max spend in USD
--no-review Skip Stages 2-3 (faster, cheaper)
--system TEXT System prompt for all models
--json Output raw JSON
Examples:
# Full 3-stage pipeline
consilium "Compare Python and Rust for CLI tools"
# Fast mode (Stage 1 only)
consilium "Summarize this" --no-review
# With screenshot
consilium "What's on this screen?" --image screenshot.png
# Budget-limited
consilium "Write a haiku about AI" --budget 0.10
# JSON output for piping
consilium "List 3 colors" --json | jq '.final_answer'
# Custom models
consilium "Hello" --models gpt-5.2,gemini-3.1-pro --chairman gpt-5.2
Model Support¶
Consilium supports any model from these providers:
| Provider | Models | Env Var |
|---|---|---|
| OpenAI | gpt-5.2, gpt-5.2-pro, gpt-5, gpt-5-mini, o3, o4-mini |
OPENAI_API_KEY |
| Anthropic | claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5, claude-sonnet-4-5 |
ANTHROPIC_API_KEY |
gemini-3.1-pro, gemini-3-flash, gemini-2.5-pro, gemini-2.5-flash |
GOOGLE_API_KEY |
Use any model with the provider/model format:
council = Council(models=["openai/gpt-5.2", "anthropic/claude-sonnet-4-6"])
Budget Control¶
Consilium tracks costs in real-time and can halt the pipeline when a budget is exceeded:
result = council.ask("Expensive question", budget=0.10)
# If Stage 1 costs > $0.10, Stages 2-3 are automatically skipped
# The best Stage 1 response is returned as the final answer
Error Handling¶
Individual model failures don't crash the council — failed responses are marked with [ERROR: ...] and the remaining models continue:
result = council.ask("Test")
for r in result.individual_responses:
if r.text.startswith("[ERROR:"):
print(f"{r.model} failed: {r.text}")
Development¶
git clone https://github.com/OpenAdaptAI/openadapt-consilium.git
cd openadapt-consilium
pip install -e ".[dev]"
pytest
License¶
MIT