The Science Says Your Hiring Process Doesn’t Work

The Sackett Meta-Analysis: What the Science Actually Says

In 2022, Paul Sackett, Charlene Zhang, Christopher Berry, and Filip Lievens published what Sackett called “the most important paper of my career” in the Journal of Applied Psychology. Their meta-analysis re-examined decades of research on the validity of common selection methods and found that prior studies had systematically overcorrected for range restriction — inflating the apparent predictive power of nearly every hiring tool in common use.

Key Findings

• Structured interviews are now #1. They emerged as the strongest standalone predictor of job performance (r = .42), displacing cognitive ability tests from the top position they held for 25 years under the Schmidt & Hunter (1998) framework.
• Cognitive ability tests dropped sharply. GMA validity fell from a previously reported .51 to .31 — a 39% reduction. A follow-up by Griebe et al. (2022) found that for 21st-century jobs, GMA validity dropped further to just .23.
• Personality assessments showed weak validity. Conscientiousness — the best single Big Five predictor — fell from .31 to .19.
• Job-specific methods consistently outperformed generic methods. The top selection procedures in Sackett’s revised hierarchy all share one feature: they are anchored to comprehensive job analysis.

The r² Reality: What These Tools Actually Explain

The correlation coefficient (r) is the standard reporting metric in I/O psychology, but it overstates practical impact. When you square the correlation to get r² — the coefficient of determination — you see the proportion of variance in job performance that the tool actually explains. This is the number that tells CHROs how much of the difference between a top performer and a poor performer is captured by each screening method.

The Validity–Diversity Tradeoff Has Changed

Perhaps the most consequential finding from Sackett’s follow-up research (Berry, Lievens, Zhang & Sackett, 2024): excluding cognitive ability tests from the selection battery generally has little to no effect on overall predictive validity, but substantially decreases adverse impact.

The long-standing argument that “yes, GMA tests produce adverse impact, but we need them because they’re so valid” no longer holds. Companies can achieve essentially the same predictive power with methods that produce significantly less disparate impact. This fundamentally changes the legal and ethical calculus for CHROs.

The Science of Individuality: Why Traditional Hiring Fails

Todd Rose, former director of Harvard’s Mind, Brain, and Education program, published The End of Average (2016), providing the theoretical framework for understanding why traditional hiring tools systematically fail. His three principles of individuality explain why aggregate-based, trait-based selection produces poor outcomes:

The Jaggedness Principle

All meaningful human qualities are multi-dimensional and weakly correlated. A person who excels at strategic thinking may be mediocre at execution speed. Traditional hiring collapses this “jagged” profile into a single score or binary hire/no-hire decision, discarding the very information that predicts success in a specific role.

The Context Principle

Individual behavior cannot be predicted apart from the specific situation. A top performer in a fast-moving startup may fail in a bureaucratic enterprise — not because their abilities changed, but because the context did. Rose writes that the right starting point is to focus on the performance the employer needs and the context in which it will occur. Generic personality assessments and cognitive ability tests, by definition, ignore context.

The Pathways Principle

For any given outcome, there are many equally valid ways to arrive there. When organizations define jobs by credentials — specific degrees, years of experience, certification requirements — they systematically exclude candidates who developed equivalent capabilities through non-traditional pathways. This is both a quality-of-hire problem and a diversity problem.

Legal Exposure: The Compliance Case for Change

David Goldstein, a shareholder at Littler Mendelson (the largest employment law firm in the United States) with over 25 years of experience defending employers before the EEOC and OFCCP, identifies several areas where traditional hiring practices create significant legal exposure:

Disparate Impact from Credential-Based Screening

Traditional job descriptions that emphasize specific degrees, years of experience, and certifications function as selection criteria under the Uniform Guidelines on Employee Selection Procedures (1978). When these criteria disproportionately screen out protected classes, the employer must demonstrate that each requirement is job-related and consistent with business necessity. Most organizations cannot make this showing because they never conducted a proper job analysis linking those requirements to actual performance outcomes.

Personality Assessments and ADA Risk

The EEOC has increasingly scrutinized pre-employment assessments. Big Five personality instruments that probe constructs like neuroticism or emotional stability face growing litigation risk as potential disability-related inquiries under the ADA. Generic, off-the-shelf implementations applied as broad screening gates across diverse roles are especially vulnerable.

ATS and Automated Screening Liability

EEOC guidance on AI and automated selection tools makes clear that employers are liable for adverse impact produced by vendor tools, even if the employer didn’t design the algorithm. State-level laws in Illinois, New York City, Colorado, and other jurisdictions add audit and disclosure obligations.

The Sackett Revision Weakens Legal Defenses

Under the Uniform Guidelines, if a selection procedure produces adverse impact, the employer must demonstrate validity — that the procedure actually predicts job performance. Sackett’s research shows that the validity evidence for most traditional tools has been substantially overstated. The meta-analytic evidence that once served as employers’ best legal defense has been undermined by the field’s own leading researchers.

System Sub-Optimization: The Structural Failure

The convergence of these three evidence streams reveals a structural problem that goes beyond any single tool or practice.

Sub-optimization occurs when each step of a process is made faster and more efficient, but the overall result gets worse because no one is optimizing for the right end goal. In hiring, companies have spent decades making it easier to post jobs, filter resumes, and screen candidates quickly — but none of these improvements are connected to the only thing that actually matters: whether the person hired performs well in the role.

The ATS is optimized for throughput. Job descriptions are optimized to attract volume. Personality tests are optimized for rapid scoring. Recruiters are measured on time-to-fill and cost-per-hire. But no one in the system is accountable for quality of hire. The result is a fast, efficient machine that is very good at hiring the wrong people.

Across all three evidence streams, the same conclusion emerges: the methods that work are job-specific, structured, and anchored to what people will actually do in the role. The methods that fail are generic, trait-based, and disconnected from performance outcomes.

Recommended Actions

1. Audit Your Current Process Against the Science

Map every screening tool and selection criterion in your hiring process. For each one, identify the published validity coefficient (r) and calculate r². Any tool explaining less than 10% of performance variance that is used as a screening gate should be reevaluated immediately — especially if it produces adverse impact.

2. Demand r² From Your Vendors

Assessment vendors routinely report correlation coefficients (r) without context. Require vendors to report r² and explain what percentage of job performance their tool actually predicts. If they cannot provide local validation data for your specific roles and context, treat generic validity claims with skepticism.

3. Restructure Job Descriptions Around Performance Outcomes

Replace skills-and-credential job descriptions with performance-based definitions that specify what the person will actually accomplish in the role. This simultaneously improves hiring accuracy, reduces adverse impact, and strengthens legal defensibility.

4. Invest in Structured Interview Quality

Structured interviews are now the single best predictor in the meta-analytic literature. But “structured” means behavioral questions anchored to specific job requirements, consistent scoring rubrics, independent assessor ratings before group deliberation, and trained interviewers who probe for evidence of comparable past accomplishments.

5. Conduct an Adverse Impact Analysis

If you have not recently analyzed your selection rates by race, gender, and age across every screening stage, do so immediately. With 1,000+ employees, your data is large enough to reveal statistically significant disparities.

6. Quantify the Cost of Status Quo

If your screening tools explain only 4–18% of performance variance, you are making a significant number of suboptimal hires. Calculate the cost of bad hires — failed probationary periods, early turnover, performance management, termination litigation — and compare that to the investment required to rebuild your process around evidence-based methods.