As AI companies scramble to build ever more capable models, data labeling has gotten its moment in the spotlight. Ex. the Meta-Scale AI deal, which saw Meta take a 49% stake in Scale and hire its CEO, injected new energy and scrutiny. With foundational models shifting from static, pre-trained systems to continually evolving, reinforcement-tuned AI agents, the role of high-quality, domain-specific data is more important than ever.
Labelbox wants to lead this shift, even as advances in synthetic training models raise existential questions about the future of human-in-the-loop data labeling. Co-founder and CEO Manu Sharma appeared on a16z’s podcast to talk about what he sees as their opportunities.
AI Created an Inflection Point
Over time, as transformers and generative models took center stage, the company, which began in 2018, faced an inflection point. Supervised learning had plateaued in its ability to power increasingly complex tasks. In response, Labelbox expanded its platform from vision to text and audio, and from tools to services. But it wasn’t until the rise of reinforcement learning and human preference modeling, post-training techniques like RLHF and SFT, that the company made its biggest leap.
“We realized that producing the highest quality data for frontier AI models required not just software but a global network of experts,” Sharma said on the AI + a16z podcast. In 2024, Labelbox launched Aligner, a service layer built on top of its tools that recruits and manages domain specialists—PhDs, engineers, medical professionals—tasked with generating the nuanced feedback and rubrics required to fine-tune generative models.
“The frontier of AI is so advanced that nearly every dataset used to improve state-of-the-art model capabilities now requires a fusion of AI software and humans. There is just no way to produce the best data in isolation,” Sharma said.
The shift wasn’t trivial. Labelbox had to reconfigure its product, sales, and operations teams from a pure SaaS model into a hybrid of software and services. But the gamble is paying off. Sharma reports that Labelbox now serves nearly every major AI lab, and in the wake of the Meta-Scale deal, has seen a surge in inbound interest from customers seeking vendor-neutral partners.
Expert-Labeled vs. Self-Training Data
While Labelbox doubles down on human-in-the-loop expertise, researchers at MIT are exploring a very different future, one where large language models train themselves. In June, MIT’s CSAIL team unveiled SEAL (Self-Adapting Language Models), a framework that allows models to generate synthetic data and use it to incrementally update their own parameters. This approach moves beyond standard pretraining, mimicking how humans take notes, revisit insights, and refine their understanding over time.
“The idea was to see if a model’s own outputs could be used to train it,” said PhD student Jyothish Pari, a lead developer of SEAL to WIRED. By generating passages, evaluating them internally, and using reinforcement signals to guide future updates, SEAL enables models to adapt on the fly without human labelers.
Another CSAIL effort, Fast Self-Training (FST), aims to reduce dependence on manually labeled data by iteratively refining model outputs using pseudo-labels on unannotated datasets. Tested on benchmarks like ImageNet and CIFAR-10, FST has achieved near-supervised accuracy with just 10% of the labeled data.
These innovations raise critical questions about whether data-labeling platforms like Labelbox are building for a future that will soon be obsolete. But Sharma remains unconvinced. “Human knowledge is infinitely vast and non-trivial,” he said. For tasks that demand expert reasoning, coding, law, finance, medicine, he argues that only specialists can generate the gold-standard datasets required to align models with human expectations.
In some domains, that alignment work is already underway. Labelbox has partnered with behavioral economists to teach AI to interpret market signals and synthesize macroeconomic insights. Elsewhere, engineers are grading code outputs with custom-written unit tests, while PhD-level mathematicians evaluate model responses for logical soundness. This kind of expert-labeled, task-specific data, often small in volume but high in precision, forms the basis of Labelbox’s competitive strategy.
An Evolving Mission
The market for data labeling is undergoing rapid transformation. Uber, Snorkel AI, Invisible Technologies, and other players are seizing the opportunity created by customer uncertainty following Meta’s acquisition of Scale. OpenAI has begun winding down its relationship with Scale, citing the need for more specialized providers. Labelbox, with its focus on high-quality, domain-specific data, is well-positioned to absorb some of that demand.
Unlike services-based providers, however, Labelbox continues to emphasize scalability through technology. Rather than relying on “labeling sweatshops,” as Snowpoint Ventures put it, the company integrates AI models to support its labeling expert, automating repetitive tasks while preserving human judgment for the hard problems.
While Sharma didn’t mention specific models like SEAL or FST, he acknowledged that synthetic approaches may play a role alongside expert-driven data generation, suggesting a future where models combine self-supervised adaptation with human-taught evaluation rubrics. The key challenge will be creating reliable evaluation datasets that reflect the complexity of human knowledge work.
In an industry chasing AGI, Sharma sees a clear opportunity: “To go after all of these agentic use cases, you’re going to need the best experts in those industries, and you have to translate their problems into environments that models can learn from.”
