{
  "_about": "Live runs behind the 'Tested' line on each /works-with/ page. Method for coding tools: in an empty folder, chunk.py drops the last partial group (range stops at len(items) - size + 1) and test_chunk.py fails. The tool, configured as shown on its page with model pareto against https://api.unbiased.ai, is asked to fix the bug and run the test. A run passes only if the tool edited chunk.py itself and python3 test_chunk.py then printed ok when rerun by the checker. Keys are redacted. A page without an entry here, or without result pass, makes no tested claim.",
  "task": "test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.",
  "checks": {
    "aider": {
      "date": "2026-10-20",
      "version": "0.86.2",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "export OPENAI_API_BASE=https://api.unbiased.ai/v1\nexport OPENAI_API_KEY=your-unbiased-api-key\naider --model openai/pareto --yes-always --message \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\" chunk.py test_chunk.py",
      "summary": "It found the off-by-one in a failing Python test, edited chunk.py, and the test passed when rerun."
    },
    "claude-code": {
      "date": "2026-10-20",
      "version": "2.1.305",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "Anthropic Messages",
      "config": "ANTHROPIC_BASE_URL=https://api.unbiased.ai ANTHROPIC_AUTH_TOKEN=<redacted> ANTHROPIC_MODEL=pareto ANTHROPIC_DEFAULT_HAIKU_MODEL=pareto ANTHROPIC_DEFAULT_SONNET_MODEL=pareto ANTHROPIC_DEFAULT_OPUS_MODEL=pareto claude -p <task> (ANTHROPIC_API_KEY unset)",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green in 7 turns.",
      "turns": 7,
      "duration_s": 11,
      "notes": "Claude Code printed its unrecognized-model note for pareto and the run finished normally."
    },
    "cline": {
      "date": "2026-10-20",
      "version": "CLI 3.0.73",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "cline auth -p openai-compatible -k $UNBIASED_API_KEY -m pareto -b https://api.unbiased.ai/v1 && cline -y -P openai-compatible -m pareto \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "codex": {
      "date": "2026-10-20",
      "version": "0.162.1",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Responses",
      "config": "~/.codex/config.toml: model = \"pareto\"; model_provider = \"unbiased\"; [model_providers.unbiased] name = \"Unbiased\", base_url = \"https://api.unbiased.ai/v1\", env_key = \"UNBIASED_API_KEY\", wire_api = \"responses\". Run: UNBIASED_API_KEY=your-unbiased-api-key codex exec --skip-git-repo-check -s workspace-write \"<prompt>\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "continue": {
      "date": "2026-10-20",
      "version": "CLI 1.5.47",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "config.yaml: models: [{name: Pareto, provider: openai, model: pareto, apiBase: https://api.unbiased.ai/v1, apiKey: your-unbiased-api-key, roles: [chat, edit, apply], capabilities: [tool_use]}]; run: cn --config ./config.yaml --auto -p \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "crush": {
      "date": "2026-10-20",
      "version": "0.98.1",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "# ~/.config/crush/crushrc\nprovider add unbiased \\\n  --name \"Unbiased\" \\\n  --type openai-compat \\\n  --base-url \"https://api.unbiased.ai/v1\" \\\n  --api-key \"$UNBIASED_API_KEY\"\n\nmodel add unbiased/pareto \\\n  --name \"Pareto\" \\\n  --context-window 1000000 \\\n  --default-max-tokens 32000\n\nmodel large unbiased/pareto\nmodel small unbiased/pareto\npermissions allow view ls grep glob edit multiedit write bash\n\n# shell\nexport UNBIASED_API_KEY=your-unbiased-api-key\ncrush run -m unbiased/pareto \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited chunk.py and reran the test to green."
    },
    "dirac": {
      "date": "2026-10-20",
      "version": "CLI 0.5.22",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "DIRAC_PROVIDER=openai DIRAC_BASE_URL=https://api.unbiased.ai/v1 DIRAC_MODEL=pareto DIRAC_API_KEY=your-unbiased-api-key dirac --yolo \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "droid": {
      "date": "2026-10-20",
      "version": "0.245.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "~/.factory/settings.json: {\"customModels\":[{\"model\":\"pareto\",\"displayName\":\"Pareto\",\"baseUrl\":\"https://api.unbiased.ai/v1\",\"apiKey\":\"${UNBIASED_API_KEY}\",\"provider\":\"generic-chat-completion-api\",\"maxOutputTokens\":32000}]}; UNBIASED_API_KEY=your-unbiased-api-key droid exec --auto medium --model \"custom:Pareto-0\" \"<prompt>\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "goose": {
      "date": "2026-10-20",
      "version": "CLI 1.55.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "GOOSE_PROVIDER=openai GOOSE_MODEL=pareto OPENAI_HOST=https://api.unbiased.ai OPENAI_BASE_PATH=v1/chat/completions OPENAI_API_KEY=your-unbiased-api-key GOOSE_MODE=auto goose run --no-session --with-builtin developer -t \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "hermes-agent": {
      "date": "2026-10-20",
      "version": "0.22.2",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "~/.hermes/config.yaml:\nmodel:\n  provider: custom\n  base_url: https://api.unbiased.ai/v1\n  default: pareto\n  key_env: UNBIASED_API_KEY\n  context_length: 1000000\nRun: UNBIASED_API_KEY=your-unbiased-api-key hermes chat --yolo -q \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "kilo-code": {
      "date": "2026-10-20",
      "version": "CLI 7.8.15",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "~/.config/kilo/kilo.json: {\"model\":\"unbiased/pareto\",\"provider\":{\"unbiased\":{\"name\":\"Unbiased\",\"npm\":\"@ai-sdk/openai-compatible\",\"options\":{\"baseURL\":\"https://api.unbiased.ai/v1\",\"apiKey\":\"{env:UNBIASED_API_KEY}\"},\"models\":{\"pareto\":{\"name\":\"Pareto\",\"tool_call\":true,\"limit\":{\"context\":1000000,\"output\":131072}}}}}} then: kilo run --auto -m unbiased/pareto \"<prompt>\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "kimi-cli": {
      "date": "2026-10-20",
      "version": "2.2.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "~/.kimi-code/config.toml: default_model = \"unbiased/pareto\"; [providers.\"unbiased\"] type = \"openai\", base_url = \"https://api.unbiased.ai/v1\", api_key_env = \"UNBIASED_API_KEY\"; [models.\"unbiased/pareto\"] provider = \"unbiased\", model = \"pareto\", max_context_size = 1000000. Run: kimi -p \"<prompt>\" --output-format stream-json",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "openclaw": {
      "date": "2026-10-20",
      "version": "2026.10.5",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "openclaw.json: models.providers.unbiased = { baseUrl: \"https://api.unbiased.ai/v1\", apiKey: \"${UNBIASED_API_KEY}\", api: \"openai-completions\", models: [{ id: \"pareto\", name: \"Pareto\", contextWindow: 1000000, maxTokens: 131072 }] }; agents.defaults.model.primary = \"unbiased/pareto\"; run: openclaw agent exec --model unbiased/pareto --cwd ./proj \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "opencode": {
      "date": "2026-10-20",
      "version": "2.0.25",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "opencode.json: {\"provider\":{\"unbiased\":{\"npm\":\"@ai-sdk/openai-compatible\",\"name\":\"Unbiased\",\"options\":{\"baseURL\":\"https://api.unbiased.ai/v1\",\"apiKey\":\"{env:UNBIASED_API_KEY}\"},\"models\":{\"pareto\":{\"name\":\"Pareto\"}}}}}; UNBIASED_API_KEY=your-unbiased-api-key opencode run --auto -m unbiased/pareto \"<prompt>\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "openhands": {
      "date": "2026-10-20",
      "version": "CLI 1.16.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "LLM_MODEL=openai/pareto LLM_BASE_URL=https://api.unbiased.ai/v1 LLM_API_KEY=your-unbiased-api-key openhands --headless --override-with-envs -t \"...\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "pi": {
      "date": "2026-10-20",
      "version": "1.3.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "~/.pi/agent/models.json: {\"providers\":{\"unbiased\":{\"baseUrl\":\"https://api.unbiased.ai/v1\",\"api\":\"openai-completions\",\"apiKey\":\"$UNBIASED_API_KEY\",\"models\":[{\"id\":\"pareto\",\"name\":\"Pareto\",\"contextWindow\":1000000,\"maxTokens\":131072}]}}}. Run: UNBIASED_API_KEY=your-unbiased-api-key pi --provider unbiased --model pareto --mode json \"<prompt>\"",
      "summary": "It read the failing test, found the off-by-one in chunk.py, edited the file and reran the test to green."
    },
    "qwen-code": {
      "date": "2026-10-20",
      "version": "0.25.3",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "OPENAI_API_KEY=your-unbiased-api-key OPENAI_BASE_URL=https://api.unbiased.ai/v1 OPENAI_MODEL=pareto qwen --auth-type openai --yolo \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green."
    },
    "copilot-cli": {
      "date": "2026-10-20",
      "version": "1.0.101",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "COPILOT_PROVIDER_BASE_URL=https://api.unbiased.ai/v1 COPILOT_PROVIDER_API_KEY=your-unbiased-api-key COPILOT_MODEL=pareto copilot -p \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\" --allow-all-tools",
      "summary": "It found the off-by-one in a failing Python test, edited the file and reran the test to green, with no GitHub sign-in."
    },
    "deepseek-harness": {
      "date": "2026-10-20",
      "version": "0.2.1-rc.2",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "$DSH_HOME/profiles/headless/cordis.patch.yml: [{id: llm-pi-ai, config: {providers: {unbiased: {apiKeyEnv: UNBIASED_API_KEY, api: openai-completions, baseURL: https://api.unbiased.ai/v1, models: [{id: pareto}]}}}}, {id: agent-default-model, config: {provider: unbiased, model: pareto}}]; UNBIASED_API_KEY=your-unbiased-api-key dsh --profile headless --json \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It read the failing Python test, fixed the off-by-one in chunk.py with its edit tool and reran the test to green."
    },
    "claude-agent-sdk": {
      "date": "2026-10-20",
      "version": "0.2.169 (Python)",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "Anthropic Messages",
      "config": "ClaudeAgentOptions(model=\"pareto\", allowed_tools=[\"Read\", \"Write\", \"Bash\", \"Glob\"], permission_mode=\"acceptEdits\", env={\"ANTHROPIC_BASE_URL\": \"https://api.unbiased.ai\", \"ANTHROPIC_AUTH_TOKEN\": \"your-unbiased-api-key\", \"ANTHROPIC_DEFAULT_HAIKU_MODEL\": \"pareto\"})",
      "summary": "A 31-line Python agent read a sales CSV, wrote and ran an analysis script, and saved a summary with correct revenue totals by region."
    },
    "vercel-ai-sdk": {
      "date": "2026-10-20",
      "version": "ai 7.0.147 with @ai-sdk/openai-compatible 3.0.74",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "createOpenAICompatible({ name: \"unbiased\", baseURL: \"https://api.unbiased.ai/v1\", apiKey: process.env.UNBIASED_API_KEY })(\"pareto\")",
      "summary": "A 51-line JavaScript program streamed an answer that called an order-lookup tool, then turned that answer into a support ticket that passed Zod validation."
    },
    "openai-agents-sdk": {
      "date": "2026-10-20",
      "version": "openai-agents 0.24.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "from agents import Agent, Runner, AsyncOpenAI, OpenAIChatCompletionsModel, function_tool, set_tracing_disabled; set_tracing_disabled(True); client = AsyncOpenAI(base_url=\"https://api.unbiased.ai/v1\", api_key=\"your-unbiased-api-key\"); model = OpenAIChatCompletionsModel(model=\"pareto\", openai_client=client); refund_agent = Agent(name=\"Refund agent\", model=model, tools=[issue_refund]); triage = Agent(name=\"Triage agent\", model=model, handoffs=[refund_agent]); Runner.run(triage, \"...\")",
      "summary": "A triage agent handed a broken-order message to a refund agent, which called an issue_refund tool with the right order id and amount and reported the confirmation."
    },
    "langchain": {
      "date": "2026-10-20",
      "version": "langchain 1.4.6, langchain-openai 1.7.2",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "pip install -U langchain langchain-openai\n\nfrom langchain_openai import ChatOpenAI\nfrom langchain.agents import create_agent\n\nllm = ChatOpenAI(\n    model=\"pareto\",\n    base_url=\"https://api.unbiased.ai/v1\",\n    api_key=\"your-unbiased-api-key\",\n)\nagent = create_agent(llm, tools=[get_price, get_stock])",
      "summary": "A LangChain agent with two tools looked up prices and stock for two items, spotted the one that could not be filled, and returned the correct $84.25 total."
    },
    "litellm": {
      "date": "2026-10-20",
      "version": "1.106.1",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "# config.yaml\nmodel_list:\n  - model_name: pareto\n    litellm_params:\n      model: openai/pareto\n      api_base: https://api.unbiased.ai/v1\n      api_key: os.environ/UNBIASED_API_KEY\n\n# shell\nexport UNBIASED_API_KEY=your-unbiased-api-key\nlitellm --config config.yaml\n\n# SDK\nlitellm.completion(model=\"openai/pareto\", api_base=\"https://api.unbiased.ai/v1\", api_key=os.environ[\"UNBIASED_API_KEY\"], messages=messages, tools=tools)",
      "summary": "A short SDK script had Pareto call a weather tool and answer from its result, and a local proxy with a pareto entry in config.yaml answered a request by that name."
    },
    "promptfoo": {
      "date": "2026-10-20",
      "version": "0.124.1",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "providers:\n  - id: openai:chat:pareto\n    config:\n      apiBaseUrl: https://api.unbiased.ai/v1\n      apiKeyEnvar: UNBIASED_API_KEY\n      max_tokens: 4096\n# UNBIASED_API_KEY=your-unbiased-api-key npx promptfoo eval",
      "summary": "It ran a three-case eval against Pareto (arithmetic, a capital city and a string reversal) and all three assertions passed."
    },
    "mastra": {
      "date": "2026-10-20",
      "version": "@mastra/core 1.78.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "model: {\n  id: \"custom/pareto\",\n  url: \"https://api.unbiased.ai/v1\",\n  apiKey: process.env.UNBIASED_API_KEY, // your-unbiased-api-key\n}",
      "summary": "A 36-line TypeScript agent called a book-slot tool to take the earliest free afternoon slot and returned the confirmation code."
    },
    "oh-my-pi": {
      "date": "2026-10-20",
      "version": "18.12.4",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "# ~/.omp/agent/models.yml\nproviders:\n  unbiased:\n    baseUrl: https://api.unbiased.ai/v1\n    apiKey: UNBIASED_API_KEY\n    api: openai-completions\n    authHeader: true\n    models:\n      - id: pareto\n        name: Pareto\n        contextWindow: 1000000\n        maxTokens: 131072\n\n# ~/.omp/agent/config.yml\nmodelRoles:\n  default: unbiased/pareto\n  smol: unbiased/pareto\n  slow: unbiased/pareto\n  plan: unbiased/pareto\n  commit: unbiased/pareto\n  task: unbiased/pareto\n  tiny: unbiased/pareto\n  advisor: unbiased/pareto\n\n# shell\nexport UNBIASED_API_KEY=your-unbiased-api-key\nomp -p --auto-approve --model unbiased/pareto \"test_chunk.py fails. Fix the bug in chunk.py, then run python3 test_chunk.py to confirm.\"",
      "summary": "It found the off-by-one in a failing Python test, edited chunk.py and reran the test to green."
    },
    "pydantic-ai": {
      "date": "2026-10-20",
      "version": "2.58.0",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "OpenAIChatModel('pareto', provider=OpenAIProvider(base_url='https://api.unbiased.ai/v1', api_key=os.environ['UNBIASED_API_KEY'])); Agent(model, output_type=Invoice)",
      "summary": "A 40-line agent turned a messy vendor email into a validated Invoice model with the right number, due date, line items and a total that matched them."
    },
    "dspy": {
      "date": "2026-10-20",
      "version": "3.4.1",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "import dspy\nlm = dspy.LM(\"openai/pareto\", api_base=\"https://api.unbiased.ai/v1\", api_key=\"your-unbiased-api-key\")\ndspy.configure(lm=lm)",
      "summary": "A typed ChainOfThought signature sorted three support tickets into a fixed set of categories with an urgency flag, and all three came back correct."
    },
    "crewai": {
      "date": "2026-10-20",
      "version": "1.15.28",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "LLM(model=\"openai/pareto\", base_url=\"https://api.unbiased.ai/v1\", api_key=\"your-unbiased-api-key\")",
      "summary": "A two-agent crew ran in sequence: a researcher listed three facts about Python's range() and a writer turned them into a brief under 60 words."
    },
    "llamaindex": {
      "date": "2026-10-20",
      "version": "core 0.14.25 (llama-index-llms-openai-like 0.8.1)",
      "result": "pass",
      "served_model": "pareto",
      "protocol": "OpenAI Chat Completions",
      "config": "from llama_index.llms.openai_like import OpenAILike\nllm = OpenAILike(model=\"pareto\", api_base=\"https://api.unbiased.ai/v1\", api_key=\"your-unbiased-api-key\", is_chat_model=True, is_function_calling_model=True, context_window=1000000)\nSettings.llm = llm",
      "summary": "A 35-line script answered a question over three in-memory documents with a CitationQueryEngine, citing the two right sources, then a FunctionAgent called a multiply tool and returned the correct product."
    }
  }
}