OpenMark

ai.openmark/openmarkVersion 1.0.0

Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.

Status

Responds183 msNov 14, 2026, 13:31 UTCAnswered our handshake without sign-in.

Registry data may be out of date

We could not refresh it, so this page shows the last copy.

Last checked: Nov 14, 2026, 13:31 UTC
Latency: 183 ms
Remotes responding: 1 of 1
Sign-in: API key
Hosting: Hosted outside the EU (detected)
Registry data as of: Nov 14, 2026, 21:12 UTC

Data from the official MCP Registry, checked by run.zero. How we check

Connect

Streamable HTTP

https://openmark.ai/mcpResponds183 msNov 14, 2026, 13:31 UTC

No sign-in

Claude Code

claude mcp add --transport http openmark https://openmark.ai/mcp --header 'Authorization: Bearer <YOUR_TOKEN>'

Replace <YOUR_TOKEN> before running.

Claude Desktop and claude.ai

Settings > Connectors > Add custom connector
Name: openmark
Remote MCP server URL: https://openmark.ai/mcp
This server expects the header Authorization.

Cursor

{
  "mcpServers": {
    "openmark": {
      "url": "https://openmark.ai/mcp",
      "headers": {
        "Authorization": "Bearer <YOUR_TOKEN>"
      }
    }
  }
}

Replace <YOUR_TOKEN> before running.

VS Code

code --add-mcp '{"name":"openmark","type":"http","url":"https://openmark.ai/mcp","headers":{"Authorization":"Bearer <YOUR_TOKEN>"}}'

Replace <YOUR_TOKEN> before running.

Built your own server? run.zero hosts MCP servers on European infrastructure.

Host your own MCP server on run.zero

Tools

10 tools

Tools of OpenMark
	Description		Read-only	Idempotent	Open world
`estimate_cost`Credits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run settings as run_benchmark. Use it to drop an expensive model, or to compare settings (e.g. max_tokens), before running.	Credits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run settings as run_benchmark. Use it to drop an expensive model, or to compare settings (e.g. max_tokens), before running.	Not declared	Not declared	Not declared	Not declared
`export_openclaw_csv`A completed job's results as the CSV the OpenMark Router plugin for OpenClaw reads (the same file as the app's Export > OpenClaw). Save it in the plugin's benchmarks folder, by default ~/.openclaw/workspace/plugins/openmark-router/benchmarks/, and OpenClaw routes this kind of task to the benchmark winner. Free.	A completed job's results as the CSV the OpenMark Router plugin for OpenClaw reads (the same file as the app's Export > OpenClaw). Save it in the plugin's benchmarks folder, by default ~/.openclaw/workspace/plugins/openmark-router/benchmarks/, and OpenClaw routes this kind of task to the benchmark winner. Free.	Not declared	Not declared	Not declared	Not declared
`get_account`The user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.	The user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.	Not declared	Not declared	Not declared	Not declared
`get_results`Status of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and quick picks for the others.	Status of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and quick picks for the others.	Not declared	Not declared	Not declared	Not declared
`get_task_format`How to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scoring, composite, chaining, documents, examples, best_practices, or a scoring mode name.	How to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scoring, composite, chaining, documents, examples, best_practices, or a scoring mode name.	Not declared	Not declared	Not declared	Not declared
`list_models`Current OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reasoning effort.	Current OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reasoning effort.	Not declared	Not declared	Not declared	Not declared
`prepare_attachments`Get an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachments list, then pass the upload_uid to estimate_cost / run_benchmark.	Get an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachments list, then pass the upload_uid to estimate_cost / run_benchmark.	Not declared	Not declared	Not declared	Not declared
`run_benchmark`Save a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's credits; pass max_credits to only start within a budget. Optional run settings match the app's Configure & Run panel (preset, stability_runs, max_tokens, temperature_discovery, temp_preference, timeout_profile, custom_timeout, fail_fast).	Save a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's credits; pass max_credits to only start within a budget. Optional run settings match the app's Configure & Run panel (preset, stability_runs, max_tokens, temperature_discovery, temp_preference, timeout_profile, custom_timeout, fail_fast).	Not declared	Not declared	Not declared	Not declared
`validate_task`Check a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a look.	Check a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a look.	Not declared	Not declared	Not declared	Not declared
`watch_task`Watch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's credits at the normal rate.	Watch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's credits at the normal rate.	Not declared	Not declared	Not declared	Not declared

Details

Registry status: Active
Published: Oct 18, 2026
Updated: Oct 18, 2026
Website: https://openmark.ai/mcp⁠ (opens in a new tab)
Repository: Not given
Categories (suggested): AI modelsSuggested categories are assigned automatically from the registry text and may be wrong. Verified operators can correct them.
GDPR: Not declared

Are you the operator? Claim this listing