AI workforce impact measurement

AI workforce impact measurement is the work of turning AI capability, use, task exposure, and organizational outcomes into defensible evidence about how work is changing. It sits between broad adoption anecdotes and strong labor-market claims: a useful measure has to say what unit is being measured, whether the evidence is predicted capability or observed use, and what inference is still missing before claims about productivity, job redesign, wages, or substitution are warranted.

2025-westover-gdp-benchmark interprets GDPval as a task-level measure of frontier-model capability on economically valuable professional outputs, rather than a direct measure of occupational displacement or realized workforce change. Its practitioner interpretation makes the capability-to-workforce-inference boundary explicit: benchmark success, speed, and cost advantages remain conditional on task specification, human review, rework, quality assurance, accountability, and workflow integration, so they cannot by themselves establish productivity, job redesign, staffing, wage, or substitution effects.

2025-tomlinson-working-with-ai gives this topic its first observed-use measurement spine. Using 200,000 Bing Copilot conversations, the authors map both user goals and AI actions to O*NET intermediate work activities, then combine frequency, completion, user feedback, and scope ratings into occupation-level AI applicability scores. The construct is deliberately narrower than productivity or labor displacement: it estimates where AI appears useful for work activities, not whether jobs have changed or whether firms will hire differently.

The useful distinction is between the human-side goal and the AI-side action. A conversation can show a worker trying to accomplish one activity while the AI performs a different activity in service of that goal; 2025-tomlinson-working-with-ai reports that user-goal and AI-action activity sets are disjoint in 40% of conversations. That makes applicability measurement more precise than a single “AI used for task X” label, because it separates assistance to the worker from work performed by the system.

For I-O use, the measurement lesson is that AI workforce impact evidence needs a nomological network: task taxonomy, observed use, completion/scope, work context, human skill, quality, accountability, and organizational redesign are adjacent but not interchangeable constructs. Applicability can locate where AI is touching work, but it does not validate claims about performance, learning, motivation, staffing, or construct-relevant job change without additional evidence.

2023-zhang-human-capital-resources adds a non-AI measurement analogue that will matter as human work and AI work converge. Human capital resource measures were often deficient or contaminated, and partially contaminated measures sometimes predicted performance better because they included adjacent constructs. The AI-era warning is direct: an index of human-AI capability or AI capital may predict outcomes while still being unclear about whether it measures human KSAOs, AI system capability, workflow design, adoption intensity, managerial practice, or downstream performance.

2025-friis-performance-principle adds social acceptability as a workforce-impact measure. Its performance-principle decomposition shows that AI capability and cost are insufficient for predicting substitution: public support for automation changes sharply under an advanced-AI scenario, while moral repugnance remains concentrated in a smaller set of occupations. Workforce-impact measurement therefore needs to distinguish technical exposure from moral-boundaries-of-ai-automation rather than treating capability as realized adoption.

2025-marguerit-augmenting-automating-labor adds an exposure-to-outcomes measurement design. Instead of treating AI exposure as a single construct, it builds separate Stack Overflow-based measures for automation AI and augmentation AI, then estimates their associations with new job-title emergence, employment, and wages. The measurement lesson is clean: workforce-impact claims change depending on whether the AI construct is task substitution, output enhancement, new-work creation, employment size, or wage movement, and those outcomes vary by occupational skill group.

2026-johnston-shift-agentic-ai-codex adds an agentic-use measurement layer. For agentic systems, active users, chats, and message volume become weak proxies because one request can delegate a multi-step workflow and one worker can manage several agents at once. The paper proposes and observes more work-relevant process measures: delegated task complexity, runtime, concurrency, reusable skills/plugins, output-token volume, and the share of AI-mediated work happening through agentic rather than conversational interfaces.

2026-hitzig-agentic-coding-expertise adds a complementary measurement frame for agentic work: decision attribution, actions per prompt, task value, inferred expertise, occupation, and transcript-based success. Its contribution is not realized labor-market impact but a richer process account of how agentic coding work is being divided and whether sessions produce verifiable success signals. The finding that non-software occupations approach software occupations in code-producing success rates while expertise remains predictive is especially useful for separating occupation, domain expertise, and agentic-work capability.

2026-massenkoff-labor-market-impacts-ai adds a displacement-risk measure that explicitly combines theoretical capability, observed work-related use, and automation-like usage before aggregating to occupations. Its “observed exposure” construct matters because it narrows the gap between what LLMs could theoretically speed up and what is actually showing up in Claude usage, then tests that measure against BLS occupational projections, CPS unemployment, and young-worker job starts. The early outcome evidence is deliberately limited: higher observed exposure correlates with slightly lower BLS growth projections, but not with a clear unemployment increase through the available window.

2026-richmond-ai-jobs-transition-framework adds a policy-facing measurement framework that separates technical exposure from human necessity, demand elasticity, and observed ChatGPT usage. Its contribution is a composite transition map rather than a realized-outcomes estimate: the Drive version classifies 921 occupations into high automation risk, reorganizing, AI-growth, and less-immediate-change archetypes, but it explicitly treats those categories as near-term pressure signals rather than job-loss forecasts.

2026-putka-indexing-ai-impact-onet offers a design-level alternative to one-number exposure measures: a proposed O*NET suite of 16 indices that separates AI capability from observed use, automation from augmentation, and knowledge, skill, task, contextual-performance, vacancy, and whole-occupation units. Its central measurement claim is that an occupational AI-impact score inherits its meaning from the performance domain, unit of analysis, aggregation rule, and AI operationalization behind it; a task-level score aggregated to an occupation is not interchangeable with a direct occupation-level judgment or a vacancy-based indicator.

2026-iscenko-google-ai-economy-atlas broadens the observed-use layer with 14.65 million de-identified Gemini interactions across app, search, and unpaid API surfaces, mapped to occupations, O*NET tasks, household activities, geography, language, task characteristics, and interaction intent. Its core contribution is a diffusion-versus-depth distinction: observed use reaches occupations covering 88% of US employment, yet the median observed occupation uses AI on only 21% of its tasks and fewer than 10% of observed non-routine cognitive-work conversations are classified as end-to-end automation. This is stronger evidence about where one platform’s users bring conversational AI than a capability score, but it remains a platform-bounded behavioral trace rather than evidence of productivity, substitution, or realized job change.

2026-chin-work-at-frontier adds occupational-boundary crossover as a distinct observed-use measure. It compares the worker’s self-reported occupation with the O*NET occupation historically associated with the message’s classified task, finding that 43.5% of occupation-specific non-generic sampled messages crossed that boundary. The construct is not exposure, adoption success, or labor-market impact: it measures an early pattern of AI-aided task recombination that may precede durable changes in roles and occupational classifications.

Connections

• Connects to task-level-ai-adoption because observed use is one input to workforce impact measurement, especially when the data preserve task or work-activity grain rather than only tool-level adoption.
• Connects to human-ai-task-taxonomy because measurement depends on correctly distinguishing user goals, AI actions, authority, scope, auditability, and output definition.
• Connects to automation-and-substitution because applicability scores and observed automation-like exposure can tempt premature substitution claims, yet displacement inference still needs unemployment, hiring, wage, or staffing evidence.
• Connects to job-analysis because occupational impact measures often inherit O*NET or other work-analysis taxonomies, with all the benefits and limits of those upstream definitions.
• Connects to construct-validity because “AI impact,” “applicability,” “exposure,” “augmentation,” and “automation” are not the same construct and should not be treated as interchangeable dependent variables.
• Connects to human-capital-resource-measurement because AI workforce impact measures will increasingly have to distinguish human capability, AI capability, and combined workflow capability rather than treating all performance-relevant capacity as one undifferentiated capital stock.
• Connects to agentic-delegation because once AI acts on behalf of workers, impact measurement has to track delegated execution and review structures rather than only advice exposure or conversational use.
• Connects to domain-expertise-in-agentic-work because workforce-impact measures need to distinguish occupational title, coding skill, task-specific expertise, AI literacy, and combined human-agent workflow capability.
• Connects to human-ai-task-allocation because interpreting capability benchmarks for work requires task decomposition and explicit decisions about review, quality assurance, and accountability in the resulting workflow.

Contradictions & open questions

• Observed conversational use improves on speculative exposure scores, but it still misses embedded automation, enterprise copilots, non-chat workflows, and tasks workers avoid asking AI to do because of risk, policy, visibility, or evaluation pressure.

• 2026-richmond-ai-jobs-transition-framework makes the exposure-to-impact measurement chain more explicit by adding human necessity and demand elasticity, but it also introduces model-estimated elasticity and platform-bounded ChatGPT use as additional construct-validity burdens.

• Applicability can be high where information work is common, but the value of that applicability depends on whether workers can evaluate outputs, integrate them into accountable workflows, and preserve the connective tissue between tasks.

• O*NET-based aggregation makes cross-occupation measurement possible while also importing U.S.-centric, lagging, and task-list assumptions; AI-era work may change faster than the taxonomy used to measure it.

• The wiki still needs field evidence linking applicability measures to realized productivity, quality, skill development, work redesign, hiring, wages, and worker experience over time.

• 2025-marguerit-augmenting-automating-labor helps fill the wages/employment side of that gap, but it also exposes a different validity problem: developer-community traces can distinguish automation and augmentation development, yet still need evidence that the measured exposure corresponds to actual organizational deployment and task recomposition.

• Future human-AI capital measures may face the same validity temptation as HCR: contaminated indexes can become attractive if they predict performance, even when they cannot say which part of the human-machine work system is doing the explanatory work.

• Social acceptability measures are not outcome evidence. 2025-friis-performance-principle tells us where the public draws moral lines around AI automation, but the wiki still needs evidence linking those attitudes to customer behavior, firm adoption, labor demand, professional governance, and regulation.

• 2026-johnston-shift-agentic-ai-codex shows why richer process measures are necessary but also why they are not enough. Output tokens, concurrency, runtime, and skill use can show that work is being reorganized around agents; they still need links to quality, productivity, wages, hiring, learning, accountability, and worker experience before they become workforce-impact evidence.

• 2026-massenkoff-labor-market-impacts-ai improves exposure measurement by using observed usage and automation weighting, but it remains platform-bounded and early. A null unemployment result does not prove no displacement; it may mean the effect is small, delayed, absorbed through hiring changes, hidden in other AI tools, or occurring through job recomposition rather than unemployment.

• 2026-putka-indexing-ai-impact-onet proposes an unusually explicit validity program—multiple models and runs for reliability plus a nomological network for convergent and discriminant evidence—but that is still a proposal. The unresolved question is whether a transparent multi-index architecture predicts realized work change, or simply produces more differentiated theoretical exposure scores.

• 2026-iscenko-google-ai-economy-atlas extends observed-use coverage to Gemini, but its breadth can invite an overclaim: diffuse platform use is not evidence that most tasks, organizations, or labor markets have changed. The 21%-median task-use estimate and platform exclusions make depth, enterprise integration, and outcome linkage the unresolved measurement problem.

• 2026-chin-work-at-frontier operationalizes occupational boundaries through self-reported roles and O*NET mappings, so its crossover estimate depends on a historical U.S. taxonomy and a primary-task classifier. It does not establish that work was completed, that specialists no longer did it, or that organizations changed staffing, job descriptions, pay, performance criteria, or accountability.

• 2025-westover-gdp-benchmark frames GDPval as a useful capability signal for professional tasks, but its practitioner interpretation does not supply field evidence that benchmark gains survive task specification, review, rework, quality assurance, accountability, and workflow integration; that capability-to-workforce inference remains open.

• 2026-hitzig-agentic-coding-expertise improves the process measurement of agentic coding, but its success labels remain transcript- and telemetry-based; workforce-impact claims still need downstream evidence about maintained artifacts, quality, productivity, wages, hiring, and skill development.