This article is part of The MCP Guide, our plain-English reference on the Model Context Protocol.
An MCP vendor evaluation scorecard is a single sheet you use to compare MCP servers, managed MCP platforms or development partners on the same criteria: security, sign-in, logging, support, pricing and spec support. You score each option 1 to 5 on each line, weight the lines that matter most to you, and compare totals. It turns a pile of demos and sales calls into a decision you can explain.
Below is the scorecard to copy, with anchors so two reviewers score the same answer the same way. It works for an off-the-shelf server, a platform that hosts many connectors, or a firm that builds one for you.
It doesn't replace the detailed question sets. Use our MCP vendor security questionnaire to collect the answers, and the MCP RFP template if you are buying a build. The scorecard is where you record and compare what came back.
Key Takeaways
- Start with must-pass checks. A vendor that fails one is out, however well it scores elsewhere.
- Score six weighted areas: security, authentication, logging, support, pricing and spec support, plus fit for your workflow.
- Use written anchors for 1, 3 and 5 so scores mean the same thing to every reviewer.
- Ask for evidence: a doc page, a tool list or a log sample. A claim with no evidence scores 1.
- A directory label or registry listing is not a security review, so score it as context only.
How to use the scorecard
- Pick the vendor type. Server, platform or development partner. Each has a few extra rows below.
- Set the weights before you see any answers. This stops a favorite vendor from shaping the weights.
- Run the must-pass checks. Drop anyone who fails.
- Score each line 1 to 5, using the anchors. Two reviewers score separately, then agree.
- Multiply by the weight and add up. Highest total wins only if no line scored 1 on security or authentication.
- Write one sentence per score saying what evidence you used.
Step 1: must-pass checks
Answer yes or no. One "no" ends the evaluation for that vendor.
- The publisher is a named company or maintainer you can contact.
- You can see the full list of tools, and which ones can change or delete data.
- For a remote server that acts on user accounts, sign-in uses OAuth, not a shared password or a key pasted into chat.
- The vendor states that its server accepts only tokens issued for it. The MCP authorization spec says MCP servers "MUST NOT accept or transit any other tokens."
- Write tools can be switched off, or set to need approval, in the AI apps you use.
- The vendor will tell you how to report a security issue.
- Tool calls can be logged somewhere you can read.
Step 2: the weighted scorecard
Copy this table. The weights shown are a starting point that adds to 100. Change them to suit your risk.
| # | Area | Weight | Score (1-5) | Weighted | Evidence used |
|---|---|---|---|---|---|
| 1 | Security design | 25 | |||
| 2 | Authentication and access | 20 | |||
| 3 | Logging and audit | 15 | |||
| 4 | Spec support and updates | 10 | |||
| 5 | Support and maintenance | 10 | |||
| 6 | Pricing clarity and total cost | 10 | |||
| 7 | Fit for your workflow | 10 | |||
| Total | 100 | / 500 |
Weighted score = weight times score. A perfect sheet scores 500.
Step 3: scoring anchors
Use these so a 3 means the same to everyone. If an answer falls between two anchors, pick the lower one unless you have evidence for the higher.
1. Security design (weight 25)
| Score | What it looks like |
|---|---|
| 5 | Explains input validation, rate limits, output handling and prompt-injection risk in plain words, with a doc page you can read. Has a published security contact and patch history. |
| 3 | Covers the basics in docs, but some answers are vague or only given on a call. |
| 1 | "Enterprise-grade security" with no detail. |
The MCP tools specification says servers MUST "validate all tool inputs," "implement proper access controls," "rate limit tool invocations" and "sanitize tool outputs." A good vendor can show how it does each. The OWASP MCP Top 10, a beta project as of November 2026, is a useful list to ask about, including tool poisoning (MCP03) and prompt injection (MCP06).
2. Authentication and access (weight 20)
| Score | What it looks like |
|---|---|
| 5 | OAuth with narrow scopes, starting with read-only access and asking for more only when needed. Per-user access, so each person sees only their own data. |
| 3 | OAuth, but scopes are broad, or all users share one connection. |
| 1 | Shared API key or password, or a single admin account for everyone. |
The official MCP security best practices describe a "progressive, least-privilege scope model" and list wildcard scopes such as * or full-access as a common mistake. Ask the vendor which scopes it requests on first sign-in.
3. Logging and audit (weight 15)
| Score | What it looks like |
|---|---|
| 5 | Logs every tool call with user, tool, time and result, exportable to your own system, without logging secrets. |
| 3 | Some logging, but only in the vendor's dashboard or with short retention. |
| 1 | No tool-call logs you can access. |
The tools spec says clients should "log tool usage for audit purposes," and OWASP lists "Lack of Audit and Telemetry" as MCP08. Ask for a sample log line. Our MCP audit logging guide explains what a useful log contains.
4. Spec support and updates (weight 10)
| Score | What it looks like |
|---|---|
| 5 | States which MCP spec versions it supports, including the current 2026-07-28 version, and how it handles deprecated features. Lists which extensions it supports and their stage. |
| 3 | Works with your AI apps today, but can't say which spec version it targets. |
| 1 | No version information, or relies on features the spec has deprecated with no plan. |
The MCP versioning page says deprecated features stay in the spec for at least twelve months, or ninety days under an expedited exception. Extensions also have stages. The extensions overview lists MCP Apps as Stable and Tasks as Beta, where "breaking changes are documented." A vendor relying on a Beta extension should say so.
5. Support and maintenance (weight 10)
| Score | What it looks like |
|---|---|
| 5 | Named support channel, stated response times, a changelog and notice before tools change. |
| 3 | Support by email or GitHub issues, no stated times. |
| 1 | No support channel, or last update many months ago. |
Tool changes matter. Anthropic's connector verification page says a connector's developer "controls which tools it exposes and can change them at any time." Score higher when the vendor tells you before that happens.
6. Pricing clarity and total cost (weight 10)
| Score | What it looks like |
|---|---|
| 5 | Published pricing, clear usage units (per call, per task, per seat), and an example of what your expected use would cost. |
| 3 | Pricing on request, but a written quote with assumptions. |
| 1 | No price until after a long sales process, or usage units you can't estimate. |
Compare a year of expected use, not a month. Our MCP business case guide lists the cost lines to include.
7. Fit for your workflow (weight 10)
| Score | What it looks like |
|---|---|
| 5 | Has the exact tools your workflow needs and works in the AI apps your team uses. |
| 3 | Covers most of the workflow, with a workaround for the rest. |
| 1 | Missing a core tool, or doesn't work in your AI app. |
Extra rows by vendor type
Add the rows that apply. Give each a weight and take it from the other lines so the total stays at 100.
For an MCP server
- Read and write split. Are tools marked read-only or destructive? Anthropic's submission docs require every directory connector tool to carry a
readOnlyHintordestructiveHintannotation. Note that the tools spec says clients "MUST consider tool annotations to be untrusted unless they come from trusted servers," so check the behavior too. See MCP tool annotations. - Hosted or local. A local server runs on each laptop and needs a different review.
For a managed platform
- Connector coverage for the apps you use today and expect to add.
- Admin controls to block tools and limit who can connect what.
- Exit plan. Can you move to another platform or a custom server without rebuilding everything? Our build vs buy guide covers the tradeoff.
For a development partner
- Inspectable work. A server you can connect to or a repo you can read, not only screenshots.
- Testing plan using MCP Inspector and your AI apps.
- Code ownership and handover terms.
What not to score as security
Two signals look like security approval but aren't:
- Directory labels. Anthropic's verification page says Verified "isn't a security audit or a guarantee of how the connector will perform."
- Registry listings. The MCP Registry docs say the registry, which is in preview, "delegates security scanning" to package registries and downstream aggregators. It checks namespace ownership, not code.
Treat both as context in the notes column.
A filled-in example (HYPOTHETICAL)
The vendors and scores below are HYPOTHETICAL. They show how the math works, not a real comparison.
| Area | Weight | Vendor A (HYPOTHETICAL) | Vendor B (HYPOTHETICAL) |
|---|---|---|---|
| Security design | 25 | 4 (100) | 3 (75) |
| Authentication | 20 | 4 (80) | 5 (100) |
| Logging | 15 | 2 (30) | 4 (60) |
| Spec support | 10 | 3 (30) | 4 (40) |
| Support | 10 | 4 (40) | 3 (30) |
| Pricing | 10 | 5 (50) | 3 (30) |
| Fit | 10 | 5 (50) | 4 (40) |
| Total | 100 | 380 | 375 |
The totals are close, and Vendor A's weak logging may matter more than five points. That is why the evidence column and the team discussion matter as much as the total.
Next step
If you'd like an outside reviewer to score servers or proposals with you, see eSEOspace MCP development. Before you shortlist, run each server through our 12-point server selection checklist.
More in the MCP Guide
Latest research: State of MCP: October 2026
Frequently asked questions
How many vendors should I score?
Three to five is usually enough. More adds review time without much benefit, and the must-pass checks often remove some early.
Should security always carry the most weight?
For servers that read private data or write to your systems, yes. For a read-only server that only uses public data, you might shift weight to fit and price. Decide before you see any answers.
What if a vendor won't answer some questions?
Score that line 1 and note "no answer." A vendor that won't explain how it handles tokens or logs is telling you something useful.
Can I use the same scorecard for a server and a development partner?
Yes. Keep the seven core areas and add the extra rows for that vendor type. Using the same core lets you compare a ready-made server with a custom build on equal terms.
