skill-eval-harness

Name: skill-eval-harness Review
Item: skill-eval-harness
Rating: 3.0
Author: Aaron Matthews

Skill Development Usable

Works inClaude Code

Usable Scanned — metadata only

Enables systematic, quantifiable assessment of AI agent behavior across variants, reducing uncertainty in production deployments and identifying performance …

Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters

Evaluate code changes across variant implementations to measure performance differences
Generate trace artifacts showing execution paths for debugging production issues
Compare runner adapters side-by-side to validate engineering decisions before deployment

59 starsMIT (commercial OK)FreeQuick setup

Usable rating — This tool is functional but has notable gaps. Review the evaluation notes below before deploying.

Why it matters

Enables systematic, quantifiable assessment of AI agent behavior across variants, reducing uncertainty in production deployments and identifying performance regressions before release.

Best for

ML teams evaluating agentic systems and engineering leads standardizing AI quality gates.

Works in

Claude Code Claude Cowork Claude Chat

Source

https://github.com/adewale/skill-eval-harness

By adewale

How to Get It

Option 1: Claude Desktop App (Code Mode)Click the + button next to the prompt box → Plugins → Add plugin. Search and click Install. Skills work in Claude Code only.

Option 2: Paste into Claude CodeCopy the command below and paste it into your conversation. Claude will install it.

Command

claude plugins install adewale/skill-eval-harness

Tip: Paste this into a Claude Code conversation. Verify command matches your Claude Code version.

Auto-generated from the tool's public listing — not hands-on verified. Cross-check against the source repo's README before running.

First thing to try

After installing, paste this into Claude:

Help me evaluate code changes across variant implementations to measure performance differences

CostFree

Trust Signals Auto-scanned

Stars59Last updated2026-07-31LicenseMIT (OK for commercial use)Known CVEsNone foundSources: GitHub Advisory Database + OSV.dev · Scanned 2026-07-31 · scanner v1

Community Pulse New

No community discussions found yet. This doesn't mean the tool isn't good — it may be new or serve a niche use case.

Reviewer notes

Auto-scanned review. These are observations, not a security certification.

Scored from trust signals (evidence-eval-v1): 59 GitHub stars; contributors unknown; last commit 0d ago; license MIT.

Things to check

Scanned, not hands-on tested — this entry was auto-scanned from public metadata (GitHub metrics, license, security flags). No reviewer has run it, and no tool-specific limitations have been documented yet.

How to evaluate tools before deploying →

Data shown here comes from public APIs and automated scanning. Reviewer notes reflect one person's experience. This is not a security certification or legal recommendation. Always evaluate tools according to your own organization's policies.

Evaluation Scores

Evaluation

Ease of Use

3/5

Versatility

3/5

Reliability

3/5

Security

3/5

Overall score3.00 / 5.00 UsableEvaluatedJul 2026

Scored from trust signals (evidence-eval-v1): 59 GitHub stars; contributors unknown; last commit 0d ago; license MIT.

← Back to the Claude Observatory