huggingface-community-evals agent skill by huggingface
Runs evaluations for Hugging Face Hub models on local hardware using inspect-ai and lighteval, including backend selection and choosing between vLLM, Transformers, and accelerate.
At a glance
| Publisher | Hugging Face Official |
|---|---|
| Repository | huggingface/skills (24 listed skills) |
| Category | AI Agents and LLM Engineering |
| License | Apache-2.0 |
| Last commit | October 8, 2026 (repository) |
| SKILL.md | 202 lines, 883 words |
| Bundled files | 5 files, including scripts |
| Quality score | 86/100 · B Passes all seven graded checks |
| Security scan | Not scanned |
| README badge | Get the badges for this listing |
Before you install
Read from the skill's files at the directory's October 22, 2026 snapshot: what ships with it, what its code reads, and the terms it is published under.
- Bundled scripts: 3 script files (3 Python). Your agent may run them, so read them first.
- Environment variables its scripts read:
HF_TOKEN. Check SKILL.md for which ones it needs. - License: Apache-2.0, an open-source or open-content license.
Install huggingface-community-evals
The skills CLI works with Claude Code, Codex, Cursor and other agents; the other routes copy the folder by hand.
npx skills add https://github.com/huggingface/skills/tree/main/skills/huggingface-community-evalstmp="$(mktemp -d)"
git clone --depth 1 https://github.com/huggingface/skills.git "$tmp/repo"
mkdir -p .claude/skills
cp -R "$tmp/repo/"skills/huggingface-community-evals .claude/skills/
rm -rf "$tmp"tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/huggingface/skills.git "$tmp/repo"
mkdir -p .agents/skills
cp -R "$tmp/repo/"skills/huggingface-community-evals .agents/skills/
rm -rf "$tmp"For a user-level install, copy the folder to ~/.claude/skills/ (Claude Code) or ~/.agents/skills/ (Codex). Inside Codex, $skill-installer can also fetch a skill from GitHub. New to skills? Read How to install agent skills in Claude Code, Codex and Cursor.
huggingface/skills is third-party code. Read SKILL.md and any bundled scripts before you install it.
How the author describes it
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
From the description field of SKILL.md. Agents read this line to decide when to load the skill.
Sections in SKILL.md
- Option A: inspect-ai with local inference providers path
- Option B: inspect-ai on Local GPU
- Option C: lighteval on Local GPU
Files in the skill folder
SKILL.mdexamples/USAGE_EXAMPLES.mdscripts/inspect_eval_uv.pyscripts/inspect_vllm_uv.pyscripts/lighteval_vllm_uv.py
Suede Skill Check: 86/100 · B
Report snapshot · · graded-v2
The rubric totals 100 points; published scores are capped at 99. Security scan: not scanned.
Source revision: c3ff942be3e0. Report version 6a480ac3face. Badge URLs follow the latest listing; save the report to keep this version.
Download report · Request a fresh review
What to review next
- Example request: 0 of 3 points. 3 when the description quotes an example request (3–60 characters in quotation marks).
- Reference files: 0 of 3 points. 3 when the skill folder ships a references/, reference/ or docs/ folder.
- Tool permissions declared: 0 of 2 points. 2 when the frontmatter declares allowed-tools.
Add declarations and bundled files when they describe the skill’s actual needs. Update the source, then request a fresh review with the changed commit or PR.
huggingface-community-evals passes the required frontmatter check and all seven graded checks, which cover the SKILL.md format, a declared license and a commit in the last year. Those checks earn 56 of the score’s points; the rest come from description quality, instruction depth, bundled files, maintenance and curated-list citations. How the score works.
See the checks and points
| Check | Result |
|---|---|
| Frontmatter parses and declares a name and a description | Pass |
| Name uses lowercase letters, digits and hyphens, 64 characters or fewer | Pass |
| Name matches the skill's folder name | Pass |
| Description is between 50 and 1,024 characters | Pass |
| Description says when an agent should use the skill | Pass |
| SKILL.md body is 500 lines or fewer | Pass |
| A license is declared in the frontmatter or the repository | Pass |
| Repository has a commit in the last 12 months | Pass |
| Score component | Points |
|---|---|
| Structural checks | 56 of 56 |
| Description length | 5 of 5 |
| Boundary clause | 4 of 4 |
| Example request | 0 of 3 |
| Instruction depth | 7 of 7 |
| Reference files | 0 of 3 |
| Bundled scripts | 2 of 2 |
| Recent maintenance | 9 of 9 |
| Curated-list citations | 3 of 7 |
| Tool permissions declared | 0 of 2 |
| Compatibility declared | 0 of 2 |
Two badges for huggingface-community-evals
Choose the numeric badge, the letter badge, or both. Each links to this skill’s Suede AI listing; either valid linked badge can qualify for the feature offer.
[](https://suedeai.directory/huggingface/skills/huggingface-community-evals/)
[](https://suedeai.directory/huggingface/skills/huggingface-community-evals/)<a href="https://suedeai.directory/huggingface/skills/huggingface-community-evals/"><img src="https://suedeai.directory/huggingface/skills/huggingface-community-evals/score.svg" alt="Suede Skill Check: 86/100" width="160" height="175"></a>
<a href="https://suedeai.directory/huggingface/skills/huggingface-community-evals/"><img src="https://suedeai.directory/huggingface/skills/huggingface-community-evals/badge.svg" alt="Suede Skill Check: Grade B" width="160" height="175"></a>Suede Certified on the shield refers to this published structural quality check. Security scan: not scanned. Read the methodology.
Limited-time feature offer: Display either quality badge in your repository or skill README, linked to your listing, then request verification. Suede AI checks the rendered README at a recorded commit. Eligible skills scoring 80–99 with a verified badge can receive a featured placement while the offer is open. Request badge verification.
Publishers: read How to get your agent skill found in search and cited by AI assistants.
Where this skill is cited
huggingface/skills appears in 2 curated lists: heilcheng/awesome-agent-skills, skillmatic-ai/awesome-agent-skills.