How DevCard scores work, and how we know.

Candidate-job matching is deterministic. The math is fixed code, not an LLM prompt. Given the same evidence, the same job, and the same scoring code version, you get the same score every time.

LLMs are used only to translate unstructured input — a resume, a project note, a job posting — into structured ontology data. Once the data is structured, the matching pipeline never calls an LLM again. There is no LLM in the matching loop.

This is a deliberate architectural commitment. It means scores are replayable, auditable, and explainable. Every number on a candidate scorecard traces to specific structured evidence assertions you can inspect.

The 20×20 set covers two archetypal blocks. The first 10 candidates and 10 jobs are non-FAANG developer archetypes — PHP/Laravel, generalist web, .NET enterprise, WordPress/CMS, embedded/firmware, application security, game dev, developer relations, Shopify/e-commerce, database engineering. The second 10 are FAANG and unicorn tier — backend infra, payments, ML, data, mobile, etc.

The diagonal pairs — same-archetype candidates and jobs — should score high. The cross-block pairs should score low. The interior cross-archetype pairs are where matching has to be honest about both readiness and transfer.

Every diagonal canary, every cross-family cap, and several specific archetype pairs are locked in test code as regression guards. A change that improves a number in one place at the cost of breaking a canary elsewhere fails CI.

We maintain a separate holdout set that the scoring code never sees during development. Its only purpose is to detect overfitting against the ground truth set.

We never tune against holdout. The moment we did, it would become a second ground truth set and lose its value as an overfitting detector.

The workflow is strictly one-directional: tune against ground truth, lock the changes, run holdout, report the number. If ground truth MAE improves but holdout MAE regresses, the change is overfitting and gets reverted — even if the GT improvement was meaningful in isolation.

No file in the application code, configuration, or database migration tree may reference holdout data. The only consumer is a single test file that reports MAE without ever influencing the application surface.

A persistent failure mode in match scoring is "evidence-side leakage" — a candidate accumulates broad, weak overlap with a role outside their actual practice family, and the score inflates. To prevent this, DevCard installs an independent signal at the gate.

Work-family classification is derived from declared role titles, tenure, and recency — not from the assertion derivation pipeline that feeds individual scores. Two structurally separate signals: one classifies the candidate, one scores the evidence. The classifier multiplies the score at the gate.

Same-family pairs pass through with a 1.0 multiplier. Adjacent-family pairs are softened. Distant pairs are heavily damped. Unrelated pairs are floored. The gate is the structural primitive that keeps cross-family false positives from inflating regardless of evidence-side signal.

DevCard is ontology-first. About 800 typed ontology items — topics, frameworks, languages, tools, platforms — form a structured graph. Hierarchy, sibling relations, and ecosystem links allow semantic resolution beyond keyword matching: MySQL and PostgreSQL surface as siblings, React and Vue share component patterns, Laravel and Rails share MVC primitives.

Underneath the ontology sits a layer of 315 concept primitives across 12 categories. A primitive is a cognitive pattern — "stateful caching coordination," "schema-driven validation," "incremental migration discipline" — that abstracts above any specific tool. Two technologies that share primitives transfer credibly even when their names do not match.

When evidence enters the system, an LLM resolves the technology name into structured ontology coordinates. Once resolved, all downstream comparison is deterministic against the graph and primitive set.

Three reasons:

1. 1 Replayability. A score that depends on an LLM call is non-deterministic. Two requests may differ. A score from last week may not reproduce this week. Deterministic math reproduces exactly.
2. 2 Auditability. A deterministic score traces to specific structured inputs. If a recruiter asks "why did this person score this?" — there is a real answer. An LLM-driven score collapses that chain.
3. 3 Bias accountability. Bias in deterministic code lives in inspectable formulas, weights, and ontology mappings. We can audit and correct it. Bias in an LLM's implicit ranking lives in opaque weights we did not train and cannot inspect.

LLMs translate. Math ranks. The two layers stay separate by design.

Numbers are about JDs, never about people. DevCard scores capability against a role context — never against an inferred personal characteristic.

We review ontology mappings, score distributions, and disparate-impact risk on a structured cadence. The deterministic architecture means a bias finding can be traced to a specific weight, mapping, or formula and corrected at the source.

DevCard does not predict whether someone is replaceable by AI, whether their career is in decline, or any "obsolescence risk" signal. Forward-effect framing only. AI must reduce uncertainty about relevance, not increase fear of obsolescence.

A mismatch with a role is communicated as context mismatch, never as personal deficiency. One role is not destiny.