Dear dbt: all you need is love.

A crude network model of dbt's community stability, and an appeal

👋 Hello! I’m Robert, CPO of Hyperquery and former data scientist + analyst. Welcome to Win With Data, where we talk weekly about maximizing the impact of data. As always, find me on LinkedIn or Twitter — I’m always happy to chat. 🙂

---

I was in high school when Facebook started to kill Myspace. And contrary to the many articles that credit operational excellence or obscure network dynamics, I remember clearly why my friends and I started to make the switch: the social elite — college students¹ — had moved to Facebook. Facebook stole the pillars of the Myspace community, and it wasn’t long before the flock followed.

But this is not a post about Facebook. It’s about dbt. Obviously, dbt is not a social network. But it is undergirded by one — dbt’s core offerings are evangelized through a massive community. And this community is its differentiator, its competitive moat, its lifeblood. And for the first time in the company’s history, this community is showing cracks. Sentiment has been faltering, and this could be critical for a company whose community is such an integral part of its business.

While it still seems most likely that dbt labs will one day end up as Facebook, capturing and preserving the hearts and minds of the analytics elite, growing dissent from dbt’s most vocal champions has started to smell of Myspace. Let’s talk about what’s happening, why I’m still bullish on dbt, and how they might avoid becoming the Myspace of data².

What’s happening: community sentiment shifts

dbt has long been the darling of the data community. They filled a glaring need: analysts wanted to build transformations, but views were brittle, schedulers had a high barrier to entry, and traditional BI was vendor-locked. dbt offered a code-first (robust), SQL-first (accessible), open source (not vendor locked) solution, filling a dbt-shaped hole in our toolchain. Analytics engineering was born, and a vibrant community of evangelists formed around it.

But recently, sentiment around dbt has started to cool. A couple months ago I went to Data Council in Austin, and a teetering ambivalence/gloom clouded mentions of dbt. You’ll hear a lot of complaints these days: the cloud IDE is still disappointing, the dbt core experience hasn’t gotten much better over the years, the semantic layer turned out to be two goats in a trenchcoat, people noticed serious issues at scale, an unsettling price hike came out of nowhere.

While these issues should certainly be addressed, what’s odd is that most of these problems are not that new. But the fact that they’ve started to govern the narrative may point to something precarious. Let’s hypothesize what caused this to happen.

A crude network model of dbt’s community

Dbt’s business is structurally bizarre. For most open source companies, the user of the open source tool and the user of the paid offering are one and the same — think Gitlab, Docker, Elasticsearch, Superset. And so, community management is straightforward: make the product better, the community will grow. The flywheel is linear.

dbt, on the other hand, has two distinct products: dbt core (a CLI tool) and their paid offering, dbt Cloud (a cloud IDE). As a result, their community instead has two sub-communities associated with each — cloud and core — that are deeply entangled. Perception and sentiment flow freely between them, meaning product investment into one can have nonlinear, unpredictable effects on the other. Invest in dbt Cloud, and dbt core users might feel shunned. Invest in dbt core, and dbt Cloud users might migrate over, decreasing revenue. It’s a double pendulum, and so, chaos and nondeterminism reign free. And in a world where product development resources are scarce, this is a product strategy nightmare.

At the present time, I’d hypothesize that all the product love in this system stems from dbt core evangelists, rather than from the paid offering. As a result, these folks are the primary agents keeping general perception afloat. And this is the root of the problem — the sentiment from these folks has been lukewarm and declining for the last few years. It started with product qualms, but has steadily shifted into this suspicion that dbt “isn’t for [them] anymore”. We’ve started to hear whispers of “eh, dbt cloud sucks” or “don’t use the semantic layer” voiced in dark corners by the most technically competent dbt users.

And so dbt is in a curious state where there’s little love for dbt cloud, but the folks that are driving perception — the dbt core evangelists — are slowly starting to feel disillusioned. Yet, they need to somehow move to a world where cloud begins to engender love on its own, or risk failing to sufficiently monetize to justify their hefty valuation.

Love is all you need

So what should dbt do? It’s obvious, isn’t it? Make dbt cloud amazing. Continue to build out dbt core so the pillars of the community stay happy. Follow through on the semantic layer. On the one hand, this seems to be exactly what dbt labs is doing — they’re investing on all fronts with full force. But on the other hand, somehow progress feels marginal.

From a feature-level perspective, the kinds of things I know I want out of dbt are fairly simple: they’re the same things I’ve wanted out of dbt since I started using dbt back in 2021. I want dbt cloud to open immediately. I want someone to finally admit that ELT was probably a mistake (or at least an overcorrection) and tell me how to go back to having less models. I want dbt build & dbt test to be as close to instantaneous as possible. I want dbt-native Intellisense, everywhere.

They’re the flavors of the dream that I think we collectively dreamt when we first executed dbt run. A dream of a seamless, end-to-end engineering-like experience. The lingering taste of engineering practices I’ve gotten so far isn’t enough — I want the rich ecosystem around it too. And yes, sure, now I want a semantic layer, but only because again, I was sold on that dream, and it wove its way into my expectations.

On the one hand, dbt labs has certainly listened. They improved loading times. They’re building the new semantic layer. They’ve built rich Intellisense. They’re constantly pushing to help folks control the model chaos. But it still takes ~10 seconds to just get started. And we’re nearly a year out from the first public release of the semantic layer and two years since their first announcement of intent, with nothing to show for it. ELT still stands strong, and all signs point to crowd control, not policy change.

There’s a feeling that somehow these solutions didn’t quite hit the mark. Pains have been patched, but alongside these came a million things I still don’t know if I need, and I’m still bleeding beneath my bandaids. It feels like they’re succumbing to some variant of the next feature fallacy, pulled in arbitrary directions in an endless game of whack-a-mole.

Instead, imagine if we instead got step function change along the axes that mattered, rather than broad incrementality against everything. Imagine a dbt cloud that loaded instantly. Imagine if dbt run and dbt test could somehow be executed instantaneously. Imagine a semantic layer that worked locally and on dbt cloud, one that put LookML to shame.

In short, I want to fall in love with dbt again. And while I’m far flung enough from the internal guts of dbt to understand whether these are possible, these sorts of things don’t seem so insurmountable—some heavy refactors might be necessary, for sure, but I’d wager they’re worthwhile. Their present roadmap appears to patch problems, when the issue at stake is this: how can dbt labs keep growing love? And love comes from real change, not lip service.

Final comments

If it wasn’t obvious already: I’m not sure who I’m writing this for. The dbt product team certainly knows the nuances of their business far better than I do. But like many others who shared their perspectives around dbt over the last year, I felt the need to scream into the dark, because I still love dbt. I just desperately want it to get better. It’s filled one of the most substantial needs in the market with a tool built with developer experience in mind. And I know I very well may be the boy who cried wolf here rather than the canary in the coal mine. But I hope that this at least gives voice, more depth of character to the user pain that I and, I imagine, others have been silently bearing. And I hope this at minimum gives some small insights to the folks at dbt. At the end of the day, I’d rather run the risk of sounding redundant than see my favorite data tool turn into Myspace.

That said, I’m hopeful they’ll be able to have their cake and eat it too. Product priorities at dbt over the last few months have certainly changed in some profound way. In a recent announcement, Tristan announced a renewed focus on dbt core with practical problems in mind. With their acquisition of Transform, there’s hope that the semantic layer will be salvaged. Dbt cloud, as well, has been making steady improvements to the user experience. And in a recent announcement for dbt core v1.5, Jeremy Cohen did one of the few things dbt could do to win evangelists back coming off of heartbreak after heartbreak — he reminded us of the humanity of their endeavors.

dbt is for people, and people aren’t perfect. We’re going to keep building this imperfect thing, and finding ways to build it together.

And to that: ❤️.

1. as a high school student. AS A HIGH SCHOOL STUDENT.

2. Disclaimer: I know little about the inner workings of dbt labs, so for those of you that actually work there or are more intimately involved with them, take this with a heavy dose of salt.