LA FORJAEnter the studio

A letter from the team

In a world full of intelligence,
we should be forging builders.

The spark

I have spent more than ten years preparing students for a national university entrance exam — first as a math-olympiad kid turned tutor, then as an engineer building an edtech platform for that exam. That means I have watched, from the front row, the fastest transformation in how students learn that I have ever seen.

When AI arrived, something wonderful happened: students stopped getting stuck. The question that used to block them until the next class now gets unblocked in thirty seconds. But something quieter happened too. The same shortcut that unblocks them also lets them step around the problems that require real intellectual effort. They skip the slow maturation of concepts. They avoid the uncomfortable stretch where abstraction is built. They reach the answer without ever owning the reasoning.

If students use AI as an answer machine, they will not learn. The tool is not the problem. The direction of the interaction is.

And there is a deeper question underneath, one every student deserves to hear: you are not graduating into the world you were prepared for. You are graduating into a world where intelligence is suddenly everywhere. So what kind of person becomes more valuable when everyone has access to intelligence? A person with agency. A person who builds.

AI can democratize teaching — that part is already happening. The harder task is democratizing the skills students develop with AI. Everyone should know how to build, to argue, to defend an idea under pressure. If we only democratize answers, we will widen the very gap we promised to close. LA FORJA is our answer to that. It flips the direction of the interaction.

What it does

LA FORJA is an adversarial learning studio for high-school and college mathematics. Students do not answer questions here. They author them.

A student writes a math item: the stem, the alternatives, and a rationale for every wrong option. Then the AI goes on the attack. Three reviewers with different evidence contracts search the item for ambiguity, mathematical errors, and weak distractors, while a deterministic probe checks for superficial answer cues. Every accepted finding must carry evidence — two conflicting readings that produce different answers, a cited source with the exact passage, or a reproducible heuristic. Never just a confidence score.

The AI does not generate the item and it does not hand over a canonical solution to copy. It challenges. The student owns the repair. When the item survives, the student defends it in a short written defense scored on an explicit rubric: name the misconception your distractor captures, explain why the correct alternative is unique, hold up under a variation of the problem. Only then is the item published with a passport — provenance, challenges, revisions and rubric results, all auditable.

Writing a good question demands deeper understanding than answering one. You must master the content, anticipate how others go wrong, and design distractors around real misconceptions. That is builder thinking, applied to mathematics — the skill we want every student to graduate with, not just the ones who can afford elite preparation.

The flame we want to relight

When I was training for the national mathematics olympiad, we had a small online forum where students posted their solutions and proofs — and other students questioned them, poked at the weak steps, demanded rigor. Defending a solution methodologically, in front of peers who genuinely wanted to find the flaw, remains one of the richest learning experiences of my life.

That forum is gone. LA FORJA is our attempt to recreate that experience for a new generation: a place where getting the right answer is not enough — where you forge your reasoning, watch it get attacked, repair it, and stand up to defend it. The mechanism was designed against the constraints of a real high-stakes exam, which is where the assessment expertise comes from, and it is deliberately exam-agnostic. The conviction is universal.

How we built it

The architecture is deliberately boring where it should be boring: explicit concurrent model calls with timeouts, schema validation on every model output, a bounded deterministic solver for the demo topics, and an application-level state machine where published versions are immutable and every repair creates a new version that must re-run the full history of accepted checks. Deterministic checks can never regress. Reasoning becomes observable under challenge.

We also authored a labeled smoke set of original items with seeded flaws — declared author-labeled, never called a gold set — and an evaluation harness that compares a single general reviewer against the specialized gauntlet across repeated runs, reporting exact counts: defects found, false positives, latency, cost. It has now run against live GPT-5.6, and every number in this repo comes from those runs.

The hardest challenges were epistemological, not technical. Our first drafts overclaimed — guaranteed quality, AI that never explains. We ran the project through rounds of adversarial review and cut every claim we could not defend. Which is fitting, because that is exactly what LA FORJA asks of students. Formalizing what “no regression” honestly means forced us to classify checks into deterministic invariants, re-executable counterexamples, and semantic judgments that must be re-adjudicated. A model saying so is never final evidence in our system.

The most valuable thing AI can add to education is not speed but resistance. Productive struggle is the mechanism of learning — and AI is astonishingly good at structuring it when you point it in the right direction.

— The LA FORJA teamOpenAI Build Week · Education track

Stated plainly: what runs, and the first real numbers

The whole pipeline is implemented, pinned by offline tests against a fake model transport, and runs end to end against live GPT-5.6 where a server API key is configured — the studio labels that availability on every surface. The evaluation has run for real: three configurations, three runs each, over a 14-item holdout the reviewers never saw during prompt development. The raw artifacts are committed to the repo, and every number below comes from them.

Runs today

  • Isolated demo sessions — random pseudonyms, auto-reset, zero PII
  • The 12-transition item lifecycle and four bounded discipline solvers
  • Three AI reviewers, the separate adjudication step and the deterministic probe — live over GPT-5.6 where a server key is configured
  • Written-defense scoring, the fail-closed history re-run and the frozen passport
  • Authoring your own item, once the demo cycle publishes
  • The reproducible smoke eval runner

The first real numbers

  • Single-reviewer baseline: finds all 12 planted defects — but flags both clean items (2 false positives)
  • Three specialists without adjudication: find 10–12, at the cost of 7–8 false positives
  • The full gauntlet with adjudication: finds 6–7, with 0–1 false positives
  • The separate adjudication step trades some recall for a large precision gain — exactly its job

The best way to understand the forge is to break something in it.

Enter the studio