Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics.
Part of AI Research Skills (Orchestra)Pack · 98 skills
by Orchestra-Research· AI & agentsUnclaimed · Yours? Claim it
About
Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
Skills people install alongside evaluating-code-models.
Runs on your machine.
Runs on your machine.
Runs on your machine.
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.copilot/skills/evaluating-code-models
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.cursor/skills/evaluating-code-models
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.claude/skills/evaluating-code-models
/plugin marketplace add Orchestra-Research/AI-Research-SKILLs /plugin install rag@ai-research-skills
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.agents/skills/evaluating-code-models
Runs on your machine.
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.gemini/skills/evaluating-code-models
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness .agents/skills/evaluating-code-models
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.openclaw/skills/evaluating-code-models
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.config/opencode/skills/evaluating-code-models
Runs on your machine.
npx -y degit Orchestra-Research/AI-Research-SKILLs/11-evaluation/bigcode-evaluation-harness ~/.hermes/skills/evaluating-code-models
Scanned and passed · checked 47 min ago
From people who use it
Step by step
Nothing new reaches your agent until a person has looked at it.
Scanned on every change, and again when our rules improve.
No reviews yet. Used evaluating-code-models? Say how it went.
Write a reviewSkills people install alongside evaluating-code-models.
Has your agent write a tight, ready-to-paste prompt for the AI tool you name, from Claude Code and Cursor to Midjourney and Sora.
Turns a task into a copy-ready /goal command for Claude Code or Codex, with a checkable finish line and a stop bound.
The .NET team's skill for picking the right AI or ML stack in .NET 8+ apps, from ML.NET to Microsoft.Extensions.AI and Agent Framework.
Turn a repeated workflow, prompt set or SOP into a tested, packaged agent skill, or audit and improve a skill you already have.
Nobody has shared how they use evaluating-code-models yet.
Security scan
Passed
Safe to install in every agent it supports. Runs in your agent, as instructions.