Build an eval: test 2 AI models like a professional
AI evaluation is one of the fastest-growing jobs in tech: companies need people who can measure whether AI is actually good. Build a 20-question eval set in a domain you know, run it against 2 models, and report the results with real numbers.
Posted by The AI Internships
The work brief
- 01Pick a domain you genuinely know (your school syllabus, sports statistics, local geography, a hobby).
- 02Write 20 questions WITH verified correct answers — mix easy, hard, and trick questions. Verify answers against real sources, not AI.
- 03Run all 20 on two different models, same wording. Record every answer verbatim.
- 04Score them (correct / partially correct / wrong), compute accuracy per model, and report where each one breaks.
What you’ll produce
Submission standard
Submit your eval set as CSV (question, expected answer, source), the results CSV (model answers + scores), and your report. Expected answers must be independently verified — an eval graded against hallucinations is worthless. Include public share links to both models' actual runs so the graded answers can be traced to the real models.
Your eval set as CSV: question, expected answer, how you verified it
20 questions with expected answers and, per question, the independent (non-AI) source you verified against.
CSV dataRequiredResults as CSV: question, each AI model’s answer, scores
Each model's answer recorded verbatim plus your score; the answers must match the shared model-run links.
CSV dataRequiredYour report: accuracy per model, failure patterns, which model for this domain and why
Written responseRequired
You’ll complete these inside your private workspace.
What you must submit as proof
This brief requires evidence an AI can’t fabricate.
Public share link to Model A's run (ChatGPT/Gemini 'Share')
A live chat-share link proving Model A's answers came from the real model, not from you.
Public linkRequiredPublic share link to Model B's run
A live chat-share link proving Model B's answers came from the real model.
Public linkRequired
Submissions without this evidence cannot be submitted.
Protect other people in your proof. Blur faces, names, phone numbers and email addresses before you upload, and refer to anyone you worked with by role or number ("Listener 1", "the stall owner"). Your proof is only ever used to check your work — it is never published, never appears on your certificate, and is never shown in your public portfolio.
How your work is evaluated
The passing benchmark is 70/100.
Eval design
22%20 varied questions with independently verified answers in a domain the student knows.
Testing rigor
22%Both models tested identically; scoring is consistent and honest.
Report quality
11%Accuracy numbers match the results CSV; failure patterns are insightful.
Real model runs
44%The two shared chat links must show the actual questions asked and the models' real, verbatim answers matching the results CSV. Fabricated or unshared runs, or answers that do not match the links, are a fail.
How we grade your AI usage
Using AI is the point — it’s the skill this certificate proves. You’ll answer three short questions about how you used it: what you asked, what was wrong with its first answer, and what you changed. Specific, honest answers score high. “I pasted the brief and submitted the answer” scores near zero.