Enterprise AI, in production.
    Four desks, one beat.

    Unstructured's Bet on Transforming Data Prep for AI with $0 in the Bank and a Vision for LLMs

    ManufacturingNews

    Unstructured's Bet on Transforming Data Prep for AI with $0 in the Bank and a Vision for LLMs

    I made a bet early on with Unstructured, I was like look we're not going to build anything in the first year of this defensible except for resolution on what the market wants, and the fastest way to achieve that is by building open-source.

    Last updated

    Over the past decade, Brian Raymond and the founding team of engineers at Unstructured Technologies have been navigating a consistent challenge in the natural language processing (NLP) space: clients eager to dive into AI/ML solutions, but hindered by the frustrating reality that their data was trapped in unusable formats. This common pain point became the driving force behind the creation of Unstructured, a company focused on solving the data preprocessing problem that has plagued the industry for years.

    Data Bottlenecks in NLP

    The world of data science has long been held back by a persistent challenge in NLP: preparing unstructured data for machine learning models. Data scientists were continuously forced to build bespoke, one-off data connectors and preprocessing pipelines to make natural language data compatible with AI algorithms. But these solutions were not scalable. Data scientists dreaded these bottlenecks, and with good reason: there was, and still is, a lack of tooling to effectively automate this crucial step. Without efficient ways to convert raw, unstructured data into usable formats for machine learning, the full potential of AI and large language models (LLMs) could never be realized.

    The team at Unstructured recognized early that to unlock the true capabilities of LLMs, the core problem had to be solved: the ability to seamlessly connect, transform, and stage any form of unstructured natural language data at speed and at scale. This led to the development of Unstructured Technologies.

    Making Data Ready for LLMs

    Unstructured began its journey in July 2022 with a bold vision: to provide an open-source toolkit that would transform raw, unstructured data into formats ready for use by LLMs. Initially, the company’s focus was on NLP workflows such as Named Entity Recognition (NER) and relation extraction models. They developed a suite of cleaning functions to ensure high-quality input, integrations with labeling tools like Argilla, and staging code that simplified the process of feeding data to models like HuggingFace.

    by AIM Media HouseFree · No paywall

    Keep reading. It's free.

    Create a free Deployed account to finish this story and every story on the site. Choose the briefings you want with it:

    or

    No credit card. One click to unsubscribe. Already a reader? Sign in

    About the author

    Anshika Mathews

    Anshika is the Global Media Lead for AIM Media House. She holds a keen interest in technology and related policy-making and its impact on society.

    Related stories