HW8: The Agentic Build

Phase 1: Write the Spec ~45 min

Before you touch Copilot, write a detailed specification for the quiz app in a file called SPEC.md. This is the most important phase. A good spec leads to a good agentic result; a vague spec leads to a mess.

Your spec must include:

• Behavior description: What does the app do, step by step, from the user's perspective? (e.g., "The app greets the user, asks how many questions they want, randomly selects that many from the question bank…")
• Data format: Your question bank should be a JSON file using the format below. Include at least 5 sample questions in your spec. (You may add more questions if you wish, but we would recommend waiting to do this until you have run one review pass (verification) and then manually assessed whether the quiz app is satisfactory (validation)).
  You may extend this format for your extension feature (e.g., adding a "difficulty" or "hint" field), but keep the base structure.

  {
    "questions": [
      {
        "question": "What keyword is used to define a function in Python?",
        "type": "multiple_choice",
        "options": ["func", "define", "def", "function"],
        "answer": "def",
        "category": "Python Basics"
      },
      {
        "question": "A list in Python is immutable.",
        "type": "true_false",
        "answer": "false",
        "category": "Data Structures"
      },
      {
        "question": "What built-in function returns the number of items in a list?",
        "type": "short_answer",
        "answer": "len",
        "category": "Python Basics"
      }
    ]
  }

• File structure: What files should exist and what does each one do?
• Error handling: What happens if the JSON file is missing? If the user enters invalid input? Specify at least 3 error cases.
• Required features:
  • A local login system that prompts users for a username and password (or allows them to enter a new username and password). The passwords should not be easily discoverable.
  • A score history file that tracks performance and other useful statistics over time for each user. This file should not be human-readable and should be relatively secure. (This means someone could look at the file and perhaps find out usernames but not passwords or scores.)
  • Users should somehow be able to provide feedback on whether they like a question or not, and this should inform what questions they get next.
  • The questions should exist in their own human-readable .json file so that they can be easily modified. (This lets you use the project for studying other subjects if you wish; all you have to do is generate the question bank.)
  • Note: None of this requires a backend, HTML, CSS, a graphical user interface, or the use of any APIs. Everything is local. If your project uses any of these things, you migth be over-engineering it.
• One extension feature of your choice. Examples:
  • Timed questions (user has N seconds to answer)
  • A "streak bonus" system that rewards consecutive correct answers
  • Difficulty levels that affect scoring
  • A "hint" system that costs points
  • Category-based filtering
  • … or your own idea!
• Acceptance criteria: A checklist of 5-8 specific things you'll check to decide if the implementation is "done." (e.g., "Running the app with an empty question bank prints a friendly error and exits with code 1.")

Phase 2: The Agentic Build ~60 min

Now hand your spec to Copilot and let it build. Here's how:

Setup

1. Create a new project folder and initialize a git repo (git init). Commit your SPEC.md as the first commit.
2. Open the folder in VS Code with the GitHub Copilot extension active.
3. Open the Copilot Chat panel (Ctrl+Shift+I or Cmd+Shift+I) and switch to Agent mode (click the mode selector at the top of the chat panel. It should say "Agent" rather than "Ask" or "Edit").

The Prompt

Give Copilot a prompt like this (adapt to your own spec):

Read SPEC.md in this project. Implement the full quiz application
exactly as specified. Create all necessary files. Include the sample
questions from the spec in the question bank JSON file. Make sure
error handling matches what the spec describes. After creating the
files, run the app to verify it starts without errors.

Rules for Phase 2

• Hands off (as much as possible). Let the agent work. Accept its file creation and terminal command prompts (to the extent that you are comfortable doing so). Your goal is to see how far it gets without your intervention.
• Track your interventions. If you do need to step in (to fix something, clarify something, or nudge it in a different direction), note each intervention in your log (see Deliverables). Interventions include follow-up prompts, manual code edits, or re-running something yourself.
• Commit the raw result. After Copilot finishes (or you decide it's done), commit everything as-is with a message like "Phase 2: Raw agentic build output". Don't fix anything first — we want to see what the agent produced.

Phase 3: The AI Review ~45 min

Now switch roles. Instead of using Copilot as a builder, use it as a reviewer — an independent check on the code it just wrote.

Review Prompt

Open a new Copilot Chat session (click "+" to start fresh — this is important so the reviewer doesn't have the builder's context). Then make sure you are in agent mode and give it a prompt similar to this (but probaby not exactly; e.g. you might want to check for usability / UX issues too):

Review the Python code in this project against the spec in SPEC.md.
For each item in the acceptance criteria, check whether the code
actually implements it correctly. Also check for:
- Bugs or logic errors
- Missing error handling
- Code quality issues (unclear naming, repeated code, etc.)
- Security concerns (e.g., unsafe file handling)

Format your review as a numbered list of findings, each marked as
[PASS], [FAIL], or [WARN]. Be specific — reference file names and
line numbers.

Important: Save this review as REVIEW.md and add it to your repository!

Act on the Review

• Read the review carefully. Do you agree with its findings? Are there issues it missed? Are any of its criticisms wrong?
• Fix the real issues (you can use Copilot to help fix them).
• Commit the fixes: "Phase 3: Fixes after AI review".

Note: It is very important that you make multiple, frequent commits, at the very least at the points specified above. This is largely how we will know that you followed the correct process for this assignment.