How AI changed our sourcing playbook — and what we kept human

We rebuilt sourcing from the ground up in 2025. LLMs are now load-bearing across our funnel — profile enrichment, semantic match, shortlist scoring, screening question generation, interview scheduling — and the composite result is a recruiting engine that ships better shortlists, faster, at meaningfully lower cost per hire.

The surprises weren't where we expected. This piece is an unfiltered account of what worked, what we kept human on purpose, and what broke in ways nobody warned us about.

The old stack, honestly

Before the rebuild, our sourcing looked like every other RPO on the market: LinkedIn Recruiter, a Boolean library maintained by senior sourcers, a CRM that nobody trusted, and a handoff to recruiters that leaked ~30% of qualified candidates at the "outreach → conversation" step. Time to first qualified shortlist: 9.4 days. Sourcer productivity: 1.8 hires per month.

None of that was broken enough to force change. All of it was slow enough to lose deals.

Where we put LLMs — and what happened

1. Semantic match replaced Boolean for senior roles

Boolean assumes recruiters know the exact vocabulary a candidate would use to describe their own experience. For senior and cross-functional roles, that assumption breaks. Semantic matching against a rich JD embedding (responsibilities, tech stack, stage, industry, comp band, geography) surfaced 2.3× more qualified profiles per hour and increased outreach response rate by 41% — because the candidates we contacted actually looked relevant.

Boolean is still useful for exact-match technical filters (specific certs, specific tools). It is no longer the primary retrieval mechanism.

2. Automated profile enrichment

We stopped asking sourcers to manually stitch together a candidate's LinkedIn, GitHub, personal site, patents, and public speaking. An enrichment pipeline runs the moment a profile enters the funnel and hands the sourcer a synthesized snapshot — ready for outreach in under 60 seconds instead of 6 minutes.

3. Shortlist scoring against a JD rubric

Every profile gets a rubric-based score against the specific role: hard requirements met, adjacent experience, seniority calibration, likely comp fit, and disqualifiers (visa constraints, location, availability). Sourcers work the top of the ranked list first. Recruiter time-to-first-conversation dropped by 52%.

4. Generated screening questions per role

Instead of a generic screening template, the LLM generates 6–8 role-specific screening questions from the JD. Recruiters can edit before sending. Adoption stuck because the questions are better than what most recruiters would have written under time pressure.

5. Interview scheduling

The least glamorous, highest-ROI automation in the stack. Scheduling used to eat 90 minutes per hire in coordination overhead. Now it eats 6.

What we deliberately kept human

Three things we've tested LLM-assisted versions of and consciously walked back:

Outreach voice

Auto-drafted outreach messages performed worse in A/B tests than recruiter-written ones — not because the LLM messages were bad, but because they were generic-good in a way that patterns-matched to spam. We kept a light-touch LLM assist (fact-checking, personalization suggestions), and left the voice with the recruiter.

Calibration calls

The initial hiring-manager calibration is where 70% of shortlist quality is set. Every attempt to shortcut this with a "smart intake form" produced measurably worse shortlists over the next four weeks. Calibration is where the recruiter earns the seat.

Offer negotiation

Nothing about offer conversations tolerates automation. The best offer negotiators read tension in a candidate's voice, close open loops the candidate didn't mention, and know when to escalate. None of that lives in an LLM.

What surprised us

• Sourcer role changed shape, not headcount. We expected to reduce sourcer FTE. We didn't. What changed is what sourcers spend their day doing — 20% research, 60% engagement, 20% pipeline curation, rather than 60% research and 30% engagement. Output per sourcer roughly doubled.
• Hiring managers noticed the shortlist quality before they noticed the speed. The story we thought we'd tell was "faster." The story clients told back was "these candidates are actually right."
• Data hygiene became the bottleneck. LLM-assisted anything is a magnifier — clean data gets cleaner outputs, messy data gets confidently wrong outputs. We spent almost as much on ATS/CRM hygiene as on the LLM tooling itself.
• Bias controls have to be explicit. Rubric-scored ranking is easier to audit than gut-feel screening, but only if you write the rubric with bias-mitigation in mind and monitor demographic drift in shortlists monthly.

The funnel, before and after

• Time to first qualified shortlist: 9.4 days → 3.6 days
• Outreach response rate: 18% → 26%
• Shortlist-to-interview conversion: 34% → 51%
• Sourcer output: 1.8 → 3.6 hires/month
• Cost per hire (blended): −34%

How to think about this for your own team

Three principles that survived the rebuild:

1. Automate the boring, keep the human on the persuasive. Enrichment, scoring, and scheduling are boring. Outreach, calibration, and closing are persuasive.
2. Measure the funnel before and after every change. "LLM adoption" is not a KPI. Response rate, shortlist conversion, and time-to-shortlist are.
3. Invest in data hygiene as if it were the tooling. Because it is.

If you're rebuilding a sourcing engine, our Recruitment & Staffing practice runs this stack for embedded and project RPO engagements. Or read our companion piece on choosing between RPO and staffing agency models.