Costanoa's John Cowgill: Overlooked Bets On AI in the Enterprise

ABSTRACT

KEY POINTS FROM JOHN COWGILL'S POV

Why is this an important thesis moving forward?

• It’s getting so much easier to embed AI into applications–opening the door to non-specialists to really build on top of AI. “After a lot of focus on ML infrastructure over the past 5 years, we're finally seeing applied ML really take off,” says Cowgill. "Chat-GPT was a lightbulb moment for a lot of people. Companies are seeing how quickly they can leverage it and other large language models for a variety of applications and tasks. These foundational models make it so much easier to embed AI into a product without requiring a team of data scientists, a huge data corpus, and a team that manages all the infrastructure.”
  
  The move towards open source AI models is compounding the pace of innovation. “The trend lines pointing to this flashpoint in AI were clear for the last two years, but the pace at which things have moved towards open source — enabling greater democratization — has come as more of a surprise to many. I think it has played just as large a role in the increase of applications coming to market in the past six months as the capabilities of these models.”

What kinds of businesses or use cases might be attached to this category?

• Companies that create innovative data sets hold the most competitive potential. “A whole lot of investment is coming down the pipe into companies like Stability.ai or Cohere that are building new foundational models. While that's awesome and important, I think we've over-rotated towards believing gains are going to be all about model optimization and compression,” says Cowgill. “The foundational models these companies are building are amazing because they enable you to get to 80-90% of the answer really fast. But I think companies with novel and differentiated data sets will be the ones with true moats in the foundational model era and the ones who can get closer to 100% automation on their use cases. Better data will trump better algorithms, particularly in any narrow use case."
  • Small data > big data:“It’s probable that ‘better data’ looks different in the foundational model era as the value of ‘small data’ — data tightly curated for a use case — will matter more than the large volumes of ‘big data’ previously required to train models end to end," he adds.

• Enduring businesses will be built that tackle overlooked or inefficient areas of the AI ecosystem. “One pattern I've observed investing in Machine Learning over the last 7 years is that the really unsexy spaces – like the hard work of setting up a data stack in order to build and use ML – tend to be great areas to build a business,” he says. These pain points are analogously to how several great companies were founded to address data labeling. “Focusing on unsexy problems tends to be where a lot of unexplored value is. One unsexy problem I see right now is prompt engineering. This is solving for the text inputs into an LLM to generate desired outputs. I would bet that it doesn't go away anytime soon. The gains of truly exceptional prompt engineering at least right now are often greater than the gains of actually fine tuning a model. While it's unsexy and many think it will go away....I'd be willing to bet there might be a good business around abstracting or optimizing prompt engineering.”

• There's also an opportunity emerging for a startup to connect and coordinate communication between different emerging foundation models. “I think we'll see a ton of foundational models released over the next two years. Like other areas of enterprise software where companies buy and leverage multiple vendors (cloud infrastructure, cybersecurity, fraud detection),  I think there may be a role for an orchestration layer that sits on top of all of these tools. You can imagine a brain that intelligently routes prompts or tasks to the model best suited for them. This might be bundled with prompt engineering as well.”

What are some of the potential roadblocks?

• Companies entering in an overinflated hype cycle face a hard path to continued scalability. After the initial enthusiasm, a lot of people will struggle to actually get a ton of enduring value from these tools. “We're probably at or nearing the peak of the hype cycle for generative AI,” says Cowgill. “Without naming names, a lot of generative AI companies have huge churn problems as the output from their models that — while impressive — just isn't quite good enough. It's important to remember that AI is really good at getting to 80 or 90%, and the final 10-20% is often literally 100x harder than the first 80 or 90%. We're about to hit that in several use cases and modalities."
• Effective AI implementation by incumbents can hinder emerging vendors. “As excited as I am about these areas, I'm actually skeptical of any application-layer generative AI company where the application is obvious at this point,” he says. Categories like text generation for writing blog posts, and image-generation for ad copy are going to remain hyper-competitive, and incumbents will move quickly to embed AI. “I also think foundational model companies themselves will attack the obvious horizontal use cases that span many industries. Emerging players should be skeptical and careful about what markets they pick. The less obvious, the better.”

VISUAL: ENTERPRISE ADOPTION OF EMBEDDED AI

IN THE INVESTOR’S OWN WORDS

I look for contrarian bets on applications of data in the enterprise, built by founders who have off-the-charts founder-market fit. Namely, they get their space better than anyone else.

Over the years, this has meant I've invested in everything from vertical SaaS auto software companies (Roadster), to creating the internet in space (Kepler), to securing the SaaS applications that power modern enterprises (AppOmni).

I'm not wedded to one market. But, given my focus on data in the enterprise, it's impossible not to be inspired by the rapid growth of foundational models like GPT-3 and Stable Diffusion and what this trend means for the application of AI in the enterprise.

I love this space because there are huge debates about where it's going and where it's adding value to how we live our lives and build businesses. How much will innovation be driven by startups or incumbents? Where does the value really live: the model, the data, the tuning, the orchestration?

I back founders who have an extremely deep understanding of their market and an unconventional take on what's missing or needed.

I like sectors and trends where even the experts disagree about what the future will look like. Therein lies the opportunity to skate on where the puck is going and not just focus on what people are talking about today.

MORE Q&A

Q: Which applications or use cases are you paying  special attention to?

A: While image generation gets the eyeballs, I think an order of magnitude more value is created with text generation, and applications of LLMs on the tech backend, like search optimization. I'm fairly skeptical of image-generation startups at this point. It's extremely crowded and the experts applying it to obvious use cases - like design, such as those at companies like RunwayML - have been at it for years by now. I think we're still in the early innings of applying LLMs to all the unsexy and thorny language-related problems in the enterprise.

Just as an example, I see a huge opportunity in code translation — rewriting and updating legacy code to more modern languages and stacks. Consider that 95 percent of Fortune 500 enterprises have some reliance on COBOL, and the average COBOL programmer is 50+ years old.

Q: What do you believe is commonly misunderstood by other participants in this space?

A:  For any horizontal app built on a foundational model, I'd worry how they will maintain competitive advantage as their underlying model or models improve. Does the fine tuning you've done on GPT-3 remain as valuable when GPT-4 comes out? And what will your unit economics really look like as so much of your product is build on another company's model?

The emerging analogy describing 'foundational models as the new public cloud' is compelling, but I think this is where it falls down a bit. While the storage and compute you bought from AWS got cheaper and more performant over the last decade, improvements in cloud-native infrastructure did not necessarily have a strong relationship to your product's differentiation. In contrast, improvements in foundational models may turn out to have a large impact on the applications built on them.

Q: What is your view on whether applications will augment human tasks or serve as replacements?

A: While I'm bullish that generative AI can automate a significant portion of creative and white-collar work, perhaps as much as 80% in some cases, I think the net of it is that it just makes that last 20% of fine-tuning work that only humans can do well, all the more valuable.

With that in mind, I'm interested to revisit “AI Agency" businesses, which leverage generative models, but with some human-in-the-loop component which takes the output — be it text or image — to the final level. As an example, I'd be more interested in investing in a 'next-generation Ad Agency that uses generative AI combined with top ad talent to create better content faster than anyone else,' than in a company that says, "let's use generative AI to automatically generate ads for brands," at this stage.

WHAT ELSE TO WATCH FOR

It’s still undetermined whether SaaS business models will apply to the AI space. “At the application layer, we're still figuring out what AI-native products really look like in the enterprise,” says Cowgill. “That said, most of the early winners of generative AI still look a lot like the good SaaS companies of the 2010s. Their business models and products are mostly just web apps operating as a thin layer on top of a foundational model.” That said, given how reliant these companies are on foundational models and the open questions around how those models are priced, it remains to be seen whether the typical AI-native business is an 80% gross margin, high recurring revenue business (like most SaaS companies) or perhaps lower margin with a higher percentage of revenue that is transactional based on 'jobs done'."

Adjacent opportunities will include deep-fake detection. “In terms of next horizon bets, we're going to have to get better at detecting text, image, and video that is generated by AI. A lot of very weird and unpleasant things are going to be unleashed by this stuff. I may already be too late to make a Seed bet on deep-fake detection but I'm convinced a great company will get built there.”

STARTUPS MENTIONED IN THIS BREIF

AppOmni, Cohere.ai, Kepler, Roadster, RunwayML, Stability.ai

Acknowledgments

For McKinsey, The State of AI in 2022-and a half decade in review, see here.