If Jay Kreps were to start his $4.5 billion-valued tech company Confluent over again, he’d take a shortcut, he says: rather than spend five years building the open source software underpinning Confluent’s success, he’d just buy it instead.
Kreps built one of the fastest growing cloud businesses in Silicon Valley off open-source software. Helping businesses work at large-scale with Apache Kafka is one of the key drivers of his startup Confluent, which places at No. 6 on this year’s Cloud 100 list. But the open-source field has radically changed since Kreps and cofounders Neha Narkhede and Jun Rao left LinkedIn to build their business nearly ten years ago. And were Kreps to start a business today, he’d be more likely to grab key infrastructure tools off the (figurative) shelf.
Speaking of the open-source tool he and cofounders Neha Narkhede and Jun Rao built at LInkedIn, Apache Kafka, Kreps says the data software is more important for the community that formed around it than it is irreplaceable code. “It’s not how Kafka was written, it’s how it was adopted,” says Kreps. “You could get the exact same thing from a number of different companies [today].”
The key to Kreps’ thesis: the rise of “open-source-as-a-service,” a trend that has enabled previously expensive IT infrastructure requirements to be purchased rather than built internally. Companies that want to outsource data warehousing can now use Snowflake, this year’s Cloud 100 list No. 1; developers who might have otherwise built custom databases can reliably turn to tools from MongoDB.
What that means, according to Confluent’s CEO: tech entrepreneurs can move faster today by grabbing off-the-shelf options and avoiding having to spend limited resources on technical staff trained in those tools. “If you’re first step in adopting this new capability is hiring a bunch of the highest paid people, it’s an expensive first step,” Kreps says. “That’s why the rise of these services is a big deal.”
Since it launched in 2014, Confluent has raised $455 million from big-name venture capital firms including Sequoia and Benchmark. Following another financing boost in April, when the company raised $250 million in a funding round led by Coatue Management, the company looks primed for a public offering — about which Kreps declined to comment.
What gives Kreps the confidence to weigh in on the state of open-source company-building beyond Confluent’s valuation: he, Narkhede and Rao spent years working on infrastructure at LinkedIn, solving their own problems with “event streaming,” in which engineers
The company operates in a sector of data management known as “event streaming,” managing LinkedIn’s huge amounts of customer interactions (picture every click, touch or search) to extract useful data. Recognizing the larger market need and opportunity, Kreps and his cofounders left with LinkedIn’s backing, raising a $7 million Series A funding round from Benchmark in 2014. The company has since introduced a cloud-based Kafka product, and secured major customers including Capital One, JPMorgan Chase and Priceline.com.
Today, Kreps says that business — Confluent’s own version of off-the-shelf “open-source-as-a-service” — is growing fast, with sales up 450% over 2019 and up a comparable amount so far this year. While Confluent continues to offer on-premise services, Kreps is betting that Covid-19 will accelerate the shift to managed services. “Companies are being asked to double down on the digital part of the business with less budget, less resources and the same amount of people,” he adds.
Confluent is far from the only open-source beneficiary of that trend, with Databricks, valued at $6.7 billion, joining it in the Cloud 100 Top 10. And it faces stiff competition from Amazon and a wave of startups nipping at its heels. Such is the irony of Kreps’ work: the next Confluent might already be working on a challenger, benefitting from its tools.