What good looks like
Every example on this page is made up for practice. None of it comes from a real task.
Specific, not vague
"The response explains that the error happens because the code reads the column 'price', but the file's column is named 'Price' with a capital P."
"The response correctly explains the error."
Why it works: it says exactly what a correct explanation contains, so any reviewer can check it.
Why it fails: "correctly" is not defined. Two reviewers could judge the same answer differently.
Open to every valid fix
"The response gives a fix that would stop the KeyError, for example using df['Price'] or renaming the columns to lowercase first."
"The response rewrites the script to use the csv module instead of pandas."
Why it works: it names the goal and gives examples, so any working fix can pass.
Why it fails: it demands one particular solution the user never asked for.
True to the image
"The response points out the KeyError: 'price' shown in Image 1." (Image numbers: 1)
"The response notes the screenshot shows a FileNotFoundError." (Image 1 actually shows a KeyError)
Why it works: it matches what the image really shows and points to the right image.
Why it fails: it states something about the image that is not there.
The right weight
Critical: "The response gives a fix that would stop the error." Bonus: "The response suggests a way to avoid capital-letter mismatches in the future."
Critical: "The response suggests a way to avoid capital-letter mismatches in the future."
Why it works: fixing the error is essential. The prevention tip is a nice extra.
Why it fails: a correct answer could leave the tip out, so it cannot be critical.
Penalties written as the mistake
"The response says the error is caused by pandas not being installed."
"The response does not fail to avoid blaming the installation."
Why it works: it describes the mistake directly. "True" clearly means the mistake happened.
Why it fails: the double negative makes it hard to tell what "true" means.
Only what the user asked for
"The response explains how to fix the error shown in Image 1." (The user asked: "How do I fix this?")
"The response adds unit tests for the whole script." (The user never asked for tests)
Why it works: it matches the user's actual request.
Why it fails: it adds a requirement the user did not ask for.
One check per criterion
Two criteria: "The response identifies the KeyError on line 12." and "The response gives a fix that would stop the error."
One criterion: "The response identifies the KeyError on line 12, fixes it, and explains how pandas reads CSV headers."
Why it works: each criterion can be marked true or false on its own.
Why it fails: an answer could do one part and miss another, so it cannot be marked cleanly.
A full worked example
This is a made-up task from start to finish.