BabZituna

How we test our matching for bias

BabZituna scores how well you fit a job. We audit that scoring for bias and publish what we find — including what we haven't yet ruled out.

Internal audit · June 2026 · the first of a quarterly cadence.

What the score is built from

The match score combines six inputs: your skills, experience, location, salary expectation, job type, and work style. It does not receive your name, photo, age, gender, nationality, or the prestige of your school or past employers — those fields are never passed to the scorer.

What we tested

For each test we took one synthetic CV, changed a single feature, and measured the change in score (0–100). A gap above 0.5 points is treated as a flag.

Does your name change your score?

No difference · 0-pt gap

Feature tested: first name (gender proxy) — proxy for protected class (gender)

male-coded names98
female-coded names98

Score is identical (0 pt gap) across all 40 names. The scorer never receives the candidate's name as an input, so this is a structural fairness property — not a statistical result. See methodology critique.

Does your city change your score?

Difference found · 20-pt gap

Feature tested: candidate city — proxy for protected class (race/national origin via geography) + legitimate distance signal

London (GB)98
New York (US)78
Paris (FR)79
Amsterdam (NL)79
Riyadh (SA)78
Dubai (AE)78
Tokyo (JP)78
Toronto (CA)78
Shanghai (CN)78
Jakarta (ID)78

An identical CV against a London job spans 20 pts across candidate cities — Hill-curve distance decay (half-score at 65 km) drives the location dimension toward zero once a candidate is far from the role. The WASPAS weak-link blend would otherwise amplify that single weak dimension well beyond its 15% nominal weight (a ~34-pt swing); a PROPORTIONALITY CAP — a higher geometric floor on the location term (MATCH_GEOMETRIC_LOCATION_FLOOR, CTO decision 2026-06-19) — bounds the realized swing back toward location's ~15% weight (~20 pt). This remains a legitimate commute signal that advantages candidates who live near the role's city — measured here across ten world cities on four continents — the single real geographic disparity the audit surfaces, and remote-capable roles fully neutralise it (Slice 4).

We also measured each city under a GB country filter: the score was identical (max filter effect 0 pts). The cross-border geo-tier penalty (the 0.1x Tier-4 multiplier) keys off the JOB's country vs the candidate's country FILTER, not the candidate's own location — so a country filter does NOT compound the candidate-geography gap.

Mitigation candidate: cap the location dimension's downward pull for remote-capable roles so a distant candidate for a remote-friendly job isn't penalised on commute they'll never make (see methodology doc).

Does your university change your score?

No difference · 0-pt gap

Feature tested: university name — proxy for protected class (race/socioeconomic via education tier)

Oxford (elite)98
Cambridge (elite)98
Manchester Met (mid)98
Wolverhampton (mid)98
City College (access)98

Score is identical (0 pt gap) across elite/mid/access universities. Education contributes a flat +5 for PRESENCE only; the school NAME is never scored. Structural property, not a statistical result.

Remote jobs: does your city still matter?

No difference · 0-pt gap

Feature tested: candidate city (remote-capable job) — mitigation check — geographic proxy neutralised for fully-remote roles

London (GB)98
New York (US)98
Paris (FR)98
Amsterdam (NL)98
Riyadh (SA)98
Dubai (AE)98
Tokyo (JP)98
Toronto (CA)98
Shanghai (CN)98
Jakarta (ID)98

For a REMOTE-CAPABLE job, an identical CV scores within 0 pt across all cities — the distance penalty is waived, so candidate geography no longer moves the score. The same CV against a HYBRID job still spans 20 pt (distance preserved by design — a hybrid hire commutes some days).

Policy: the work-style classifier (job.workStyle) is the authority; hybrid is deliberately excluded from the cap. On-site/unknown fall back to the legacy job.remote flag.

CI pins: remote-capable gap must stay <= 1 pt; hybrid must keep decaying (> 1 pt) so the cap can't silently swallow hybrid commute signal.

Honest limitations

What we found, changed, and re-measured

Found: the one real disparity was geographic — a candidate far from a job lost most of the location dimension, and the weak-link blend briefly amplified that to a 34-point swing across cities for an identical CV, sharper than location's 15% weight describes.

Changed: for fully-remote roles we waive the distance penalty entirely — where you live can't move your score for a job you'd do remotely. For on-site and hybrid roles we added a proportionality cap so the blend can't push geography's realized influence past roughly its declared 15% weight — distance still counts (a hybrid hire commutes some days), but in proportion, not amplified.

Re-measured: remote-capable jobs now vary by 0 points across cities (was 34); on-site/hybrid jobs vary by about 20 points — distance preserved, but capped to match location's published weight (down from the uncapped 34). Our test suite fails the build if either property breaks.

The frameworks we hold ourselves to

We benchmark against NYC Local Law 144 (bias auditing of automated employment decision tools) and the EU AI Act (Article 6 — hiring is a high-risk use case). This is an internal audit; independent third-party certification is planned, not yet complete. We don't claim full compliance — we claim we're measuring, publishing, and fixing.
The test harness and raw results live in our codebase (scripts/bias-audit/, docs/legal/bias-audit-2026-06.results.json) and the methodology in docs/legal/bias-audit-methodology.md. Questions: privacy@babzituna.com.