The problem
Australia's Online Safety Amendment (Social Media Minimum Age) Act 2024 requires platforms to prevent children under 16 from accessing social media. To enforce this, the government needed to identify which age assurance technologies actually work — and which don't — before recommending them to industry.
With 52 vendors in the market and no established evaluation standard, the government required a statistically rigorous, defensible framework for comparing biometric age-estimation systems across diverse demographic groups. The stakes were high: a flawed evaluation could either expose children to harm or create disproportionate barriers for legitimate users.
What I did
I was part of the core analytical team that spearheaded the evaluation framework for this nationally significant trial. My contributions spanned statistical design, pipeline engineering, and vendor benchmarking across a 2,500+ participant study.
The evaluation proposal I co-authored — the D-2.9 Evaluation Proposal — is an 88-page public document published by the Commonwealth of Australia. It defines the methodology, sampling strategy, and scoring criteria used to assess age assurance vendors for compliance with Australia's Social Media Minimum Age legislation.
I designed and engineered a Python and SQL computer vision pipeline integrating cloud image sources across AWS, enabling automated age estimation at scale. The pipeline improved age-estimation performance by approximately 20% relative to baseline vendor models — a meaningful gain when applied across millions of users.
Technical approach
- Stratified 2,500+ participant sampling across 12 demographic groups
- Python + SQL computer vision pipeline on AWS
- Benchmarking of 52 biometric age-assurance vendors
- Statistical scoring framework across 8 evaluation dimensions
- GDPR-equivalent data handling for biometric records
Policy impact
- Directly informed legislative recommendations under Australian federal law
- Established reusable evaluation methodology for future vendor assessments
- ~20% improvement in age estimation accuracy vs vendor baselines
- Public 88-page evaluation proposal — citable, scrutiny-tested
- Demonstrated ethical AI evaluation at national scale
Why this matters for your organisation
If your organisation needs to evaluate AI vendors, design sampling frameworks, or produce methodology documents that can withstand regulatory or legal scrutiny, this project demonstrates exactly that capability. The combination of statistical rigour, computer vision pipeline engineering, and policy documentation is rare in a single analyst.
This work is particularly relevant to public sector bodies, digital identity providers, age verification platforms, and any organisation navigating AI procurement in regulated industries.