Skill Quality: Anti-Patterns and Skill Smells

Right Shape, Still Broken

The previous lesson taught you to pick the correct pattern before writing a skill. That’s necessary, but it isn’t sufficient — a skill can be a perfectly well-scoped Validator and still be badly built. Recent research studying public SKILL.md files found that over 99% contain at least one quality issue — a “skill smell,” by analogy with the code smells that have organized software quality conversations for decades. This lesson names the recurring smells and gives you a concrete way to grade a skill’s quality independent of its pattern.

Ten Skill Smells

1. Vague Description. description: Helps with PDFs. A description with no stated trigger conditions means the skill activates unpredictably — sometimes never on prompts it should handle, sometimes never noticed as broken because nobody realized it should have fired. Covered originally in the free course’s SKILL.md Fundamentals.

2. God Skill. A skill that’s quietly become three patterns wearing one name — the “process an invoice” workflow skill from the previous lesson that’s accumulated five decision branches, a validation step, and a report-generation step, all in one SKILL.md. Every unrelated capability added to a skill increases the chance a future edit to one part breaks an unrelated part.

3. Menu Instead of Default. You can use pypdf, pdfplumber, PyMuPDF, or pdf2image... — presenting several equally-weighted options forces the agent to make a judgment call the skill’s author was better positioned to make once, correctly, rather than leaving every execution to re-derive it.

4. Silent Reference. A references/ file that exists but is never given a load condition — See references/ for details instead of If the API returns a non-200 status, read references/api-errors.md. The agent either never finds it or loads it speculatively every time, defeating the entire purpose of a separate file.

5. Prompt Bloat. Instructions that re-explain what the underlying model already knows well — what a PDF is, how HTTP works — burning context budget on content that adds no capability the model didn’t already have.

6. Hardcoded Instance. A skill that encodes the answer to one specific past task rather than a repeatable procedure — a SQL join written for one exact schema instead of a described method for joining related tables generally. It helps exactly once and needs to be rewritten for the next case that looks almost, but not quite, the same.

7. No Failure Path. Instructions that only describe the happy path — what to do when everything goes right — with nothing about what happens when a tool call fails, an API times out, or input doesn’t match the expected shape. This is common enough, and important enough, to get its own full lesson: Skill Engineering.

8. Untested Trigger. A skill shipped without ever checking it against a labeled set of should-trigger and should-not-trigger prompts — meaning nobody actually knows whether it over-triggers, under-triggers, or both, until a user notices in production. Covered at a larger scale in Testing and Evaluation at Scale.

9. Stale Skill. No version, no changelog, no record of when it was last verified against the current model or API it depends on — a skill that might have silently stopped working weeks ago and nobody would know until it fails visibly. Covered in Skill Lifecycle and Versioning.

10. Secret Leak. Credentials, tokens, or internal URLs hardcoded directly in SKILL.md or a bundled script, rather than referenced through environment variables or a secrets manager — a skill committed to a shared repository is a skill anyone with read access can now see those secrets in. Covered in Security and Governance.

Bad → Average → Production: One Skill, Three Tiers

Naming smells in the abstract is useful; watching one skill improve past all of them is more useful. Here’s a single skill — a config file validator — at three quality tiers.

Bad:

---
name: check-config
description: Checks config files.
---

Look at the config file and tell the user if it's wrong.

Smells present: vague description (2, over-broad and under-specified at once), no stated rules (there’s nothing here to actually validate against), no failure path, never tested.

Average:

---
name: config-validator
description: Validates a project's config.yaml file. Use when the user asks to check, validate, or debug a config file.
---

Check config.yaml against these rules:
1. `port` must be a number between 1024 and 65535.
2. `env` must be one of: development, staging, production.
3. `timeout_seconds` must be present and positive.

Report which rules pass and fail.

Improvement: a real, specific description; actual stated rules; something genuinely checkable. Smells still present: still no failure path (what if the file doesn’t parse as YAML at all? what if it’s missing entirely?), reports which rules fail but not necessarily why in a way the user can act on, no evidence it’s ever been tested against real config files.

Production:

---
name: config-validator
description: Validates a project's config.yaml file against required fields and value constraints. Use when the user asks to check, validate, or debug config.yaml, or reports a config-related startup error.
metadata:
  version: "1.2.0"
---

## Validate config.yaml

1. If the file doesn't exist or fails to parse as YAML, report that specific
   failure immediately — do not proceed to field-level checks.
2. Check each rule below. Report every failing rule, not just the first:
   - `port`: number, 1024-65535
   - `env`: one of development, staging, production
   - `timeout_seconds`: present, positive number
3. For each failure, state the field, the value found, the rule violated,
   and a corrected example value.
4. If all rules pass, confirm explicitly rather than saying nothing.

See `references/legacy-fields.md` if the config contains a `retries` field —
that field was deprecated in v1.2.0 and needs migration guidance, not a
plain validation failure.

Improvement over average: an explicit failure path for the case the average version silently assumed away (a file that doesn’t parse at all), specific and actionable failure messages, a versioned metadata field enabling the lifecycle practices from later in this course, and a references/ file given an actual load condition rather than existing silently.

A Quality Checklist

Run any skill you’ve written against this before calling it production-ready:

• Description states both what the skill does and when to use it, with no vague or over-broad language
• Skill maps to exactly one design pattern from the previous lesson — or is a deliberate, documented composition of a few
• No content that just re-explains what the model already knows
• Every references/ file has a stated load condition somewhere in the body
• At least one documented failure path — not just the happy case
• Tested against a labeled should-trigger / should-not-trigger query set, not just tried once
• Versioned, with a note on when it was last verified
• No secrets, tokens, or internal-only URLs hardcoded anywhere in the skill

A skill that clears every box on this list isn’t necessarily exceptional — it’s simply not carrying any of the ten smells above. Given that over 99% of studied public skills carry at least one, clearing this list already puts a skill ahead of nearly everything currently out there.

Summary

• Picking the right design pattern (previous lesson) is necessary but not sufficient — a correctly-shaped skill can still carry real quality problems
• Ten named smells cover most of what goes wrong: vague descriptions, god skills, menus instead of defaults, silent references, prompt bloat, hardcoded instances, missing failure paths, untested triggers, staleness, and leaked secrets
• Watching one skill move from bad to average to production makes the abstract checklist concrete — each tier fixes specific, nameable problems, not just “improves” vaguely
• The quality checklist is meant to be run against every skill before it ships, not read once

Several of these smells get their own full lesson later in this course — starting next with the one most skills carry silently until something actually breaks in production: missing failure handling.