LinkedIn is launching its new AI-powered people search this week, after what seems like a very long wait for what should have been a natural offering for generative AI.
This comes a full three years after the launch of ChatGPT and six months after LinkedIn launched its AI job search offering. For tech leaders, this timeline reflects a key enterprise lesson: Deploying generative AI in real enterprise settings is challenging, especially at the scale of 1.3 billion users. It’s a slow, brutal process of practical adaptation.
The following account is based on several exclusive interviews with the LinkedIn product and engineering team behind the launch.
First, here’s how the product works: A user can now type a natural language query, like, "Who is knowledgeable about cancer treatment?" In LinkedIn’s search bar.
Older LinkedIn searches based on keywords may have gotten you stumped. Must have seen this only for references "cancer"If a user wants to be sophisticated, he or she will have to run a separate, rigorous keyword search for "cancer" And then "oncology" And try to manually join the results together.
However, the new AI-powered system understands Intention Discovered because under the hood LLM understands semantic meaning. For example, it recognizes that "cancer" ideologically related "oncology" and even less directly "Genomics Research." As a result, it brings up a far more relevant list of people, including oncology leaders and researchers, even if their profiles don’t use the exact term "cancer."
The system also balances this relevance utilityInstead of only showing the world’s top oncologists (who may be an unobservable third-degree connection), it will also measure who is in your immediate network – like a first-degree connection, "quite relevant" And can serve as an important bridge to that expertise.
See the video below for example.
However, arguably, the more important lesson for enterprise practitioners is that "cookbook" LinkedIn has developed: a replicable, multi-stage pipeline of distillation, co-design, and continuous optimization. LinkedIn had to perfect it on one product before trying it on another.
"Don’t try to do too much at once," writes Wenjing Zhang, LinkedIn’s vice president of engineering, in a post about the product launch, and who also spoke with VentureBeat in an interview last week. She notes that before "expanding ambition" Building a unified system for all of LinkedIn’s products "Stalled progress."
Instead, LinkedIn focused on winning one vertical first. The success of its previously launched AI Job Search – which led to job seekers without a four-year degree Chances of getting a job are 10% higherAccording to Aaron Berger, vice president of product engineering – the blueprint was provided.
Now, the company is applying that blueprint to a much bigger challenge. "It’s one thing to be able to do this in millions of jobs," Berger told VentureBeat. "It’s another thing to do so north of a billion members."
For enterprise AI builders, LinkedIn’s journey provides a technical playbook for In fact Moving from a successful pilot to a billion-user-scale product takes time.
New Challenge: 1.3 Billion Member Graph
Berger explained that the job search product has created a strong recipe on which newcomers can build a search product.
Recipe starting with a "golden data set" Out of only a few hundred to a thousand actual query-profile pairs, carefully scored against detailed 20- to 30-page "product policy" document. To scale it up for training, LinkedIn used this little golden set to inspire a larger foundation model generating massive amounts of data. artificial training data. This synthetic data was used to train 7-billion-parameters "product policy" Model – A high-fidelity judge of relevance that was too slow for live production but perfect for teaching small models.
However, the team hit a wall early on. For six to nine months, they struggled to train a model that could balance strict policy adherence (relevance) against user engagement signals. "aha moment" That’s when they realized they needed to solve the problem. He distilled the 7B policy models into one 1.7B Teacher Model Focused only on relevance. They then combined this with different teacher models trained to predict specific member actions, such as connecting to a job application, or searching for people and following them. it "multi-teacher" The ensemble generated soft probability scores that the final student model learned to mimic via KL divergence loss.
The resulting architecture functions as a two-stage pipeline. First, a big 8b parameter model Handles extensive retrieval by casting a wide net to pull candidates from the graph. Then, the highly distilled student model takes over for fine-grained ranking. While job search product deployed successfully 0.6B (600 million) Parameter students, newbies to the product, require even more aggressive compression. As Zhang noted, the team reduced their new student model from 440M to just 220m parametersAchieving the speed needed for 1.3 billion users with less than 1% relevancy loss.
But applying it to people’s pursuit broke down the old architecture. The new problem doesn’t just involve ranking But recovery,
“A billion records," Berger said, there is a "Different animals."
The team’s prior recovery stack was built on CPUs. To handle new scale and latency demands "Fast" For search experience, the team had to move its index GPU-based infrastructureThis was a fundamental architectural change that the job search product did not need,
Organizationally, LinkedIn benefited from several perspectives. For a while, LinkedIn had two separate teams , job search and people search , Efforts are being made to solve the problem in parallel. But once the job search team achieved its success using the policy-driven distillation method, Berger and his leadership team intervened. They reveal the key to job search victory , Product Head Rohan Rajeev and Engineering Head Wenjing Zhang , To directly transplant their ‘cookbook’ to the new domain.
Distillation for 10x throughput gains
With the retrieval problem solved, the team faced a ranking and efficiency challenge. This is where the cookbook was adapted with new, aggressive customization techniques.
Zhang’s technical post (I will post the link as soon as it goes live) Provides specific details that our audience of AI engineers will appreciate. One of the more significant optimizations was the input size.
Team trained to feed the model one more LLM with reinforcement learning (RL) for the same purpose: summarizing the input context. it "Summary" The model was able to reduce the input size of the model 20 times With minimal information loss.
Combined result of 220M-parameter model and 20x input reduction? A 10x increase in ranking throughputWhich allows the team to efficiently serve the model to its huge user base.
Pragmatism on Promotion: Building Tools, Not Agents
Throughout our discussion, Berger remained adamant about something else that might grab people’s attention: The real value for enterprises today lies in improving recommendation systems, not in chasing them. "Agentic promotion." He also declined to talk about the specific models the company used for the searches, suggesting it almost didn’t matter. The company selects the model based on which model it finds most efficient for the job.
The new AI-powered people search is an expression of Berger’s philosophy that it is best to optimize the recommender system first. Architecture includes a new "intelligent query routing layer," As Berger explained, he himself is LLM-driven. This router practically decides whether the user’s query – e.g. "trust the expert" – Should move to a new semantic, natural-language stack or to an older, reliable lexical search.
This entire, complex system is designed to "tool" that one Future The agent will use, not the agent itself.
"Agent products are only as good as the tools they use to accomplish people’s tasks," Berger said. "You can have the best logic model in the world, and if you’re trying to use an agent to search for people, but the people search engine isn’t very good, you’re not going to be able to deliver results."
Now that people search is available, Berger suggested that one day the company will offer agents the ability to use it. But he did not give details about the timing. He also said that the recipe used for finding jobs and people will be spread to other products of the company.
For enterprises building their own AI roadmaps, LinkedIn’s playbook is clear:
Be practical: Don’t try to boil the ocean. Win a vertical, even if it takes 18 months.
codify "cookbook", Turn that win into a repeatable process (policy document, distillation pipeline, co-design).
Constantly Optimize: Real 10x profit comes after Initial models in pruning, distillation, and creative optimization like an RL-trained summarizer.
LinkedIn’s journey shows that for real-world enterprise AI, the emphasis should be on specific models or cool agentic systems. Sustainable, strategic advantage comes from mastering line pipe – An ‘AI-Native’ cookbook of co-design, distillation, and brutal optimization.
(Editor’s note: We’ll soon be publishing a full-length podcast with LinkedIn’s Aaron Berger on the VentureBeat podcast feed that will delve deeper into these technical details.)