Not yet human-reviewed
This source record was written automatically by the research pipeline and has not yet been reviewed by the maintainer or integrated into a topic page. Treat it as a provisional summary and verify claims against the original source.
Bias audit laws: how effective are they at preventing bias in automated employment decision tools?
Citation. Hilliard, A., Gulley, A., Koshiyama, A., & Kazim, E. (2024). Bias audit laws: how effective are they at preventing bias in automated employment decision tools? International Review of Law, Computers & Technology, 39(3), 327–343. https://doi.org/10.1080/13600869.2024.2403053
Summary. This peer-reviewed article evaluates bias-audit laws for automated employment decision tools (AEDTs), centering New York City Local Law 144 and comparable proposed U.S. legislation. It argues that output-focused audits can improve transparency but cannot by themselves detect upstream bias or guarantee fair employment decisions. The authors argue that effective regulation requires appropriate metrics, meaningful scope and enforcement, compliance, and interdisciplinary expertise spanning I-O psychology, law, computer science, and data science.
Key claims
- New York City Local Law 144 established a precedent for requiring annual independent bias audits and public summaries for AEDTs, but its narrowed scope and limited published audits constrain its likely impact.
- Selection-rate and impact-ratio metrics used in output audits can be unreliable or inadequate when subgroup samples are small, distributions differ, or bias enters through training data, feature selection, or model design rather than observed outcomes alone.
- Low penalties, ambiguous enforcement, legal loopholes, and limits on which employers, tools, and workers fall within a law’s scope can reduce compliance and therefore the practical effectiveness of audit mandates.
- Bias prevention requires interdisciplinary audit approaches that combine psychometric and legal expertise with technical examination of data, features, models, and outcomes.
Evidence & limitations
- The article is a legal and conceptual analysis of AEDT bias-audit laws, statutory and regulatory materials, audit metrics, and related literature; it does not provide an empirical causal evaluation of the laws’ effects.
- Its assessment of Local Law 144 relies partly on the early implementation environment, where the number of published audit summaries and enforcement precedents was limited.