CMU 15-114: Effective Coding with AI
README.md, plus a Google form submissionThe premise: At this point you've probably started noticing some instructions that you repeatedly give your AI when doing assignments. "Don't rewrite files I didn't ask you to touch." "Check this against my acceptance criteria." "I need to follow these course policies." Building off of Jack's earlier preview, we're going to make reusable tools that will encode these behaviors for your agents.
You'll build a small, reusable agent toolkit: a skill that configures your AI tools to work the way you have learned you want them to work, and an MCP server that gives them something new they can do. Then you'll prove it does something by using it to do a task and iterating through failures.
Why This Matters: Up to now you've been a user of AI tools. Configuring them is the next step, and it's where most of the leverage is for people who do this professionally. A good toolkit means you stop re-explaining your standards and instead start building a harness that works well for you, specifically you.
This is also preparation for Project 3. Your capstone is completely open-ended and will be the largest thing you build this semester. Build this toolkit so that future-you actually benefits from it.
Learning Objectives:
A skill is a folder with a SKILL.md file in it. That's genuinely all it is. The file has a little YAML frontmatter (a name and a description) followed by markdown instructions telling the agent how to do something.
my-skill/
├── SKILL.md # required: metadata + instructions
├── scripts/ # optional: code the skill can run
├── references/ # optional: documentation it can consult
└── assets/ # optional: templates
Your agent reads only the name and description at startup, so it knows roughly what each skill is for. When a task looks like a match, it loads the full instructions. That's why the description matters so much: it's the part that determines whether your skill ever gets used.
The format is an open standard, originally from Anthropic and now maintained under the Linux Foundation's Agentic AI Foundation. The same folder works in Claude Code, Codex, Cursor, Copilot, Gemini CLI, and a long list of others, which means the toolkit you build this week is not locked to one vendor.
Here's a complete, useful skill. It reviews a project against the acceptance criteria in a SPEC.md, which should look familiar from HW7.
---
name: spec-check
description: Reviews the current project against the acceptance criteria in
SPEC.md and reports which are met, partially met, or not met. Use when
checking progress against a spec, or before submitting work.
---
# Spec Check
## What to do
1. Read `SPEC.md` in the project root and find the numbered acceptance criteria.
2. For each criterion, locate the code responsible and decide whether it is met.
3. Do not fix anything. Report only.
## Output format
One line per criterion:
`[MET | PARTIAL | NOT MET] #<n>: <criterion> — <evidence, naming file and function>`
Then list the three highest-priority gaps.
## Rules
- Cite a file and a line for every MET claim. If you can't cite one, it isn't met.
- Judge what the code does, not what the comments or README claim it does.
We expect this to work because it has a narrow job and a specified output format. Also, the two rules are meant to avoid common failure cases: an agent that quietly fixes things while reviewing and an agent that believes the README. Good skills are mostly made of constraints you learned the hard way. There's no substitute for experience or critical thought here, so take your time and be deliberate when creating these.
Where a skill tells your agent how to work, an MCP server gives it something new it can do. You write a small program that exposes one or more tools, your agent is told where to find it, and from then on it can call those tools the same way it calls a file read or a web search.
A minimal one-tool server in Python is genuinely about this long:
from mcp.server.fastmcp import FastMCP
mcp = FastMCP("my-notes")
@mcp.tool()
def search_notes(query: str) -> str:
"""Search my course notes and return the most relevant passages.
The docstring is very important: this is what the agent reads to decide
whether to call this tool, so describe it the way you'd describe
it to a collaborator.
"""
# Replace this with your own lookup, e.g. the vector store from HW6.
results = f"(no results yet for: {query})"
return results
if __name__ == "__main__":
mcp.run()
Then you register it with your client (a few lines of JSON or TOML, depending on the tool) and the tool shows up in your agent's available actions. The MCP docs have quickstarts for both Python and TypeScript.
Note the docstring. Like a skill's description, it's the only thing the agent sees when deciding whether your tool is relevant to the task at hand, so it needs to describe what the tool does succinctly and accurately. Also, as you might imagine... if the tool does not do what it says it does, the AI probably won't be able to fix it, and may not even detect it.
The skills you build for your AI should come from your past experiences and moments of friction. Memory is fickle, so we've been asking you to share your prompt logs for each assignment, and these provide a valuable way for you to reflect on your process. Go read your own prompt logs from HW3 through HW7. Find a thing (like an instruction or a chunk of guidance) you typed more than once, or a mistake you corrected more than once, and plan on building your skill around that.
Quote the evidence in your write-up. We don't need much; just two or three lines from your old logs is plenty.
You're building two things to solve two kinds of problems. Part of the goal of the assignment is to distinguish which kind of solution you need:
When you want different behavior, write a skill. When you want new reach, build an MCP server. These are easily conflated.
Build a skill that reviews work against a standard and reports without fixing anything. The spec-check example above is one kind; a security checker, an accessibility checker, or a "which files did the agent touch that I didn't ask it to" checker are others.
Run your checker in a separate agent session from whatever produced the work, with a fresh context, and ideally on a different model. An agent reviewing its own output is an unreliable reviewer, for roughly the reason you are a bad proofreader of your own writing. Separating the two is the most valuable habit in this assignment.
Build a small MCP server exposing one tool that your agent can call. One is enough. Do not build five, and it does not need to be hosted anywhere: a local server over stdio (the Standard Input/Output streams on your computer) is exactly what we're asking for.
Make it something you'd actually use. Some options:
Use the toolkit (both the skill and the MCP tool) on an actual task: the agent calls your MCP tool, and your checker reviews the result in a separate session. Perform two runs minimum, so you can show it behaves consistently rather than having gotten lucky once. Record what the agent did.
The task can be anything real: a piece of a past assignment redone properly, groundwork for Project 3, something for another class, a personal project.
This is the requirement that matters most. Your skill will not work correctly the first time. Find the place where it misfired (it was ignored, it was too vague, it was too rigid, it triggered when you didn't want it to), and fix it. Then document the before, the failure, and the after.
Not required, and don't attempt these at the expense of the four requirements above:
AGENTS.md, CLAUDE.md, or your tool's equivalent) that sets conventions for a whole repoOpen your prompt logs from this semester and search for repetition or friction in your process.
Write down two things: a piece of guidance or instructions you keep having to restate across projects (that becomes your checker skill) and something your agent kept not knowing (that becomes your MCP tool). For each, note the evidence from your logs.
Then spend a few minutes browsing existing skills (see Resources) to see how other people write them, and to check whether someone has already solved your problem well. Borrowing and adapting is fine and encouraged, as long as you say what you borrowed and add something uniquely yours.
We'll do this step together in class on Monday, so if you were there, this is mostly already done.
Find where your tool looks for skills and put a folder there. This varies by tool and changes often enough that you should check your tool's current documentation rather than trusting a blog post. Note that asking AI will also be unreliable here. They don't always find the most recent documentation. The Resources section has the official links.
Write a trivial skill first (one that does something obviously visible, like always starting its reply with a specific word) and confirm your agent actually picks it up. Do this before building anything real. Half the difficulty in this assignment is discovering that your agent was never reading your skill at all.
Keep SKILL.md short. A few hundred lines is a lot; most good skills are well under one hundred. Put long reference material in a separate file and point at it.
Spend real effort on the description field. It is the only thing your agent sees when deciding whether your skill is relevant, so write it to say both what the skill does and when to use it.
Specify the output format explicitly, and include at least one rule that prevents a known failure (e.g. "do not modify any files," "cite a file and line for every claim").
Start from a quickstart in the MCP docs and get a single tool returning something, anything, before you wire in real data. Run it standalone first and confirm it starts without errors.
Write the tool's docstring with as much care as the skill's description. Remember that this is what the agent reads when deciding whether to call it.
Add your server to your client's config, restart the client, and verify the tool appears in the agent's available actions. Then ask a question that should make it call your tool, and confirm it actually did rather than answering from its own knowledge.
The format differs by client, so check your tool's current docs. For Codex, the CLI and the VS Code extension share one file at ~/.codex/config.toml, and a stdio server is three lines:
[mcp_servers.my-notes]
command = "python"
args = ["C:/full/path/to/my_notes_server.py"]
In the extension you can open that file from the gear menu under MCP settings → Open config.toml. Use forward slashes even on Windows, since backslashes need escaping in TOML. Claude Code, Cursor, and the rest use JSON instead, but they all want the same three things: a name, a command, and the arguments to pass it.
This is the step most likely to go wrong. Common causes: a JSON or TOML typo, a relative path where an absolute one was needed, the wrong Python interpreter, or a server that writes to stdout and corrupts the protocol (log to stderr instead).
Do the actual task. Work in one session with the MCP tool available, then run your checker in a separate session with a fresh context. Keep the transcripts or copy the relevant parts out as you go. Reconstructing them afterward is painful.
(OH! There's an interesting idea here... while it doesn't quite fit the description of this assignment, it could be very useful to have a skill that aggregates your prompt logs for you as you go. Try this if you want, in addition to making the skill that you've already imagined. We might also do this in class.)
By now you have probably already seen something misbehave, from either piece. If you somehow haven't, go looking: use the skill on a task slightly outside what you designed for, hand it to a different model, or feed your MCP tool an input you didn't anticipate. Then revise, and rerun to confirm the fix.
Document as described in What to Submit. Write this for future-you during Project 3, who will have forgotten the details.
Push to GitHub and fill out the Google form.
One new public GitHub repository containing:
SKILL.md in it, laid out so someone could copy it into their own setup and use it.requirements.txt, package.json).README.md, written by you, covering:
DEMONSTRATION.md, the evidence that it works (also written by you and not AI):
ITERATION.md, the failure and the fix (you guessed it, written by you the human):
Google form: Submit your repo link here before Sunday 11/8 at 8:00 PM.
Graded on whether your process and documentation are thoughtful, and whether your toolkit is real, grounded, and tested (not on how clever it is).
| Component | What We're Looking For |
|---|---|
| The skill (20) | Create a working checker with a narrow job, specific instructions, a defined output format, and a description written so the agent loads it at the right time. |
| The MCP server (25) | Create one tool that works, register it with your client, and demonstrate that it can be called by your agent. Provide installation instructions good enough for us to run it. |
| Grounding (15) | Provide evidence and justification for your toolkit from your own past prompt logs. |
| Demonstration (20) | Use your toolkit on something real, with the checker in a separate session. Run this twice with iteration. Your transcripts must show what the agent did, including where it made the tool call. |
| Iteration (20) | Describe a failure or a place where improvement was necessary, how you found and fixed it, and the results of a rerun. Heavily weighted relative to its size. |
A toolkit that is modest but genuinely used and genuinely revised will score better than an elaborate one with no evidence behind it, or one that's thoughtlessly lifted from online or vibe-coded in a hurry.
Speaking of being in a hurry... a few of you won't read the assignment writeup and won't notice that 55 points come from documentation, but so it goes.
These are deliberately mundane. Mundane and used beats impressive and ignored.
.gitignore before you pushscripts/. If you wouldn't run a stranger's shell script, don't install their skill.
SKILL.md dilutes its own instructions, and you'll hit the same problem HW8 students hit last spring with over-specified specs: too much detail boxes the agent in and removes the benefit.