04Agent skills
When to keep, simplify
or remove agent skills
Anthropic distinguishes skills that supplement a model’s capabilities from skills that encode a team’s workflow. The distinction helps explain why better models can reduce the need for some workarounds while task-specific requirements still need to reach the agent.
THE SHORT VERSION
Key points
- Revisit model workarounds as capabilities improve, while retaining the requirements and context the agent still needs.
- Compare runs with and without a skill using representative tasks and the same acceptance criteria.
- Keep, simplify or remove a skill according to its contribution to reliable results and the effort it adds.
01CAPABILITY UPLIFT
Compensating for a model limitation
Which instructions help the model do something it cannot yet do reliably?
Anthropic calls these capability uplift skills: they help with tasks the base model cannot perform, or cannot perform consistently. Its skill-creator article says they may become less necessary as models improve.
Sources[01]
COMMENTARY
A detailed procedure can be useful because a model otherwise misses steps or produces an unreliable result. If a newer model handles the same task reliably with less instruction, the procedure can be reconsidered for simplification.
A model change is a reason to revisit a skill. Keeping every workaround indefinitely can preserve assumptions about an earlier model. Reviewing each part can help distinguish instructions that still improve the result from those that no longer help.
Conceptual illustration of the distinction and evaluation approach discussed above.
02ENCODED PREFERENCES
Preserving requirements and context
What does the agent need to know about this particular work?
Anthropic uses encoded preference skills for workflows whose individual steps the model already knows, but which need to follow a team’s process. Their value depends on how accurately they reflect that process.
Skills can also contain scripts and reference material. Anthropic’s original introduction describes packaging procedural knowledge and organizational context, with code available for operations that benefit from repeatable execution.
COMMENTARY
Examples include which sources are authoritative, what counts as an acceptable result and which decisions need approval. A model becoming better at analysis does not establish which criteria apply to a specific task. Those requirements need to be provided or retrieved from a reliable source.
One way to handle changing organizational state is to keep it in the system that owns it and use the skill to explain when and how to retrieve it. Technical access restrictions need enforcement in the application or tool. A skill can describe the boundary and the workflow around it.
A single skill can serve both purposes. Obsolete instructions can be removed while retaining criteria, references or useful scripts. These parts can be judged separately.
Conceptual illustration of the distinction and evaluation approach discussed above.
03TEST THE DIFFERENCE
Compare the skill with a baseline
What evidence would justify keeping, simplifying or removing it?
Anthropic recommends evaluations to detect regressions and identify when a base model succeeds without the skill. Its skill-creator workflow supports comparisons between skill versions and between runs with and without a skill.
Sources[01]
COMMENTARY
A useful comparison would use representative tasks and the same acceptance criteria for both conditions, with comparable models, tools and relevant inputs. It would include difficult cases as well as routine ones. For a skill that preserves preferences, the criteria must check whether those preferences were followed.
One good result would not settle the question. Reliability across the chosen tasks needs to be considered alongside the effort, time and context the skill adds. A skill that helps on complex cases may still be useful even if simple cases work without it.
The decision is practical: keep a skill that improves the work, simplify one whose useful content has narrowed, and remove one that no longer adds value. In my view, better models should let us maintain fewer workarounds and clearer requirements.
Conceptual illustration of the distinction and evaluation approach discussed above.
Anthropic provides the two skill categories and the evaluation approach. The commentary explains why I find this distinction useful and how it could inform skill maintenance. This article reports no benchmark of my skills and makes no claim that either category is a permanent boundary on model capability.