The response you deserve!

❗ This is a hot take, written in the hours following reading Pedram’s post for the first time. It hasn’t gone through “corporate comms” (we don’t really have that but you get what I mean) and I haven’t talked to every relevant human at dbt Labs who might have thoughts / info to share. I’ll do my best!

There are parts I’m not going to wade into, but I’ll get into a lot of it.

First off: Pedram, thank you. It seems like you believed you were taking a personal risk here, and I’m sorry if you felt that way.

dbt is not perfect. The dbt Community is not a cult. dbt Labs is just a fallible collection of humans. It should be safe to say what you said out loud. Someone in our internal Slack put it so very well:

I'm interpreting it as an arc of a relationship:
At the beginning, everything is ✨
But after that is the growing pains of intimacy, which can include unfulfilled needs or wants 🤒
Also there can be a sense of loss or disappointment if the other takes different paths than you hoped/expected.
Each member of the community will journey through relational phases differently, and many will never make it all the way to 💖
These moments provide an opportunity for a relationship to mature.
On the other end of this type of refining fire, the deepest and strongest relationships can emerge.

Pedram: you are safe. I love you for what you wrote—you made it safe for other people to publicly voice dissatisfaction. A community that has such a strong behavioral norm towards positivity can, in fact, squash constructive criticism. I couldn’t do what you just did, no one at the company could’ve done it, it required you. And you fuckin’ showed up and did it. Thank you.

What a gift. I hope it makes our relationship stronger.

One of the things that was hardest to hear was that you feel that we’re not being very transparent. I hate that. Transparency is one of our values, and I’ve worked so hard to live that value internally. It seems like I’ve taken my eye off of it in the Community. That needs to change, and it certainly was not intentional. You know how I feel about this community:

In case you can’t tell, I really, deeply, care about this community. It is my life’s work. I pour everything I have into it every single day. The thing I want to make sure you’ve heard me say is that Fishtown Analytics exists to support the dbt community, not the other way around.

Maybe we need to figure out some kind of regular routine to make sure that there’s a healthy conversation where folks feel heard and I can listen. Monthly office hours? I’m open to suggestions.

Honestly, I don’t always know how to navigate my own voice in community conversations these days. Sometimes I stay away from commenting on stuff because I don’t want to seem to have all the answers, or I want to create space for others on our team to have the conversation, or I want to just let people voice stuff publicly without trying to solve it (and sounding defensive in the process). 🤷 I don’t know. But the dynamic has made me less willing to just dive into public Twitter or Slack threads. I’m open to guidance.

Ok, I wanted to get all of that out of the way. I want to go point-by-point on a bunch of stuff. Quotes are from the original piece.

In the last 2 years, dbt has had four rounds of funding. With funding comes pressures to generate revenue and demonstrate growth. Rightly, investors want returns on their capital. I don’t begrudge dbt for raising money or for pursuing profits, they aren’t, after all, a charity. They are a company, and their duty is to generate revenue for their shareholders, some of whom are their employees.

I’ve seen this sentiment expressed elsewhere, and I get it. But it really (honestly) doesn’t resonate. Yes, we do need to become a “real” company (one that earns $$), and yes we are in the process of doing that. But that process is going well. I honestly have a lot more stress associated with exactly the product challenges you write about vs. our ability to monetize. The minute that the community no longer values what we do, our commercial potential evaporates.

What I think VC funding has forced us to do is go through this really painful J-curve (a curve in which things get worse before they get a lot better). Previously, there was this small handful of people who were working on dbt, and we had been doing so for years. We all knew it so incredibly well, we didn’t have to talk about anything. Now, at any given point in time roughly half of our engineers have been on the team for less than six months. Are we hiring great people? AbsoLUTEly. Do they know the codebase, the workflow, the mission, the market like we did back in 2019? No. That’s just the math of growing the team quickly—it takes real time to actually build up velocity. We’re well on our way up the other side of the J.

I will say that the whole “raise money to invest in the community” thing is actually happening. We have 10 FTEs dedicated fully to dbt Core (product and engineering), and 8 FTEs dedicated to community, documentation, events, etc. (You can see our entire up-to-date org chart publicly here.) This is meaningfully more than two years ago; we’re getting there.

dbt Slack has grown so big it has become hard for me to follow, and I’ve been a very active member of the Slack community in my two years as a part of it. I’ve live-tweeted Coalesce, made memes, responded to questions, asked my fair share, and tried to keep up with the great content people shared.

But these days, I barely check the Slack community. Part of the problem is just one of sheer scale. Many channels that could be interesting have become troubleshooting catch-alls for people with technical questions.

There’s a lot there. So much. Let me just riff a bit:

• There is no longer a “dbt Community” in the same way there was 2+ years ago. “How do I do X thing?” isn’t a community. Instead, there are now many many smaller communities. Most of these will be formed around certain geographic regions so that people can meet up and develop real human relationships (at least…when case counts are low)—it’s still possible to get to know real humans at your local meetup. There are three events in London this month, two of which we had nothing to do with(!). So community is very much alive and well…just different.

• As such, Slack is no longer a perfect tool. And, yes, we’re looking to drive certain engagement to different platforms (we probably need to be more aggressive about this).

• But it’s possible that part of this is actually OK. One of the biggest values I get from dbt Slack today is immediate DM connections to basically my entire data network. I’m not the first person to say this, but it’s freaking great to have a DM channel to a large majority of my professional connections. I think each of us will go through phases of evolution with the community over time.

Again—these are hot takes, and I’m very much interested to hear your thoughts and feedback. I also don’t want to step on our community team’s toes too much.

I have bigger gripes though. The biggest one is the lack of transparency and communication from dbt’s leadership about their vision for core. Where’s the love it used to get? What’s the plan for development? Is there one? Has all development on core been pushed aside in favor of cloud and metrics? My biggest issue here is that I just don’t know.

Truly I’m sorry if we aren’t doing a good job here. I’ll do another brain dump.

• You can always see the upcoming milestone and what we’re working on in Github. Next up is 1.2, scheduled for early July. We now target a minor version release 1x/quarter.

• It turns out that there is a ton of pure maintainership work to do with a user community this large. Scan through the issues that get logged—these are not such visionary ideas, but they do really matter to members of the community.

• The 1.0 version introduces a whole new level of rigor required for releases. This includes adapters, and now many vendors are maintaining their own adapters. We’ve had to invest significantly in tooling and automated testing to increase our confidence that we won’t introduce regressions.

• All that said, we are making real forward progress towards user-facing goals. We plan to support non-SQL languages that are now increasingly supported by the cloud data platforms (you mentioned Python!). We plan to more natively support grants, which has been super-budget literally forever. This has the potential to grow into more configuration-as-code for non-relation parts of the warehouse over time. We could certainly do a better job sharing here.

Should dbt do a better job namespacing models? (yes) Are Jinja macros…challenging? (yes) Is jinja + SQL actually the forever answer? (potentially not, but likely for a while yet) Are exposures as useful as they ideally would be? (no) Although, on that last one, it seems like you have ideas:

there’s no clear way for vendors to integrate with dbt exposures

Very interested to hear more…!

There’s the Analytics Engineering Roundup, which could be a perfect venue for discussing the future of dbt. (…) What if we had insight into what the vision of dbt is for practitioners from the CEO of dbt? I long for that.

Hmm! This may be the one single thing I disagree with. I try very hard to take my CEO hat off when I write the Analytics Engineering Roundup and just write as a longtime data practitioner. Does dbt come up? Yep. But the point of the newsletter has never been product update or content marketing. I’ve published this now for just shy of 7 years and it’s always primarily been about linking to whatever it is that I’m reading at the time. It forces me to read and to think in public. Like I said before, I’m totally down to find other ways to make sure there’s an open conversation though!!

dbt Cloud is a really bad experience. I hate saying this, but it’s also so universally known that I don’t think it is even controversial to say. In the end, it’s nothing more than a text editor with some syntax highlighting. Loading it is exceptionally slow, it has almost no awareness of the dbt ecosystem, and the interface is so confusing that I find it impossible to recommend to people learning dbt, which is so painful for me.

I hear you. I just said something on this topic to our team earlier this week:

What’s keeping me up at night?
The one thing I currently exhibit a physiological stress response over is the customer experience of using dbt Cloud. We are not delivering a quality of experience in the IDE that I am proud of today. There are a lot of reasons for this and *I want to take a lot of ownership for the strategic decisions that have led us to where we are*, but it’s not a position that’s aligned with users are our best advocates and we have to act with urgency here.

Fortunately, we’ve had some very honest assessments of what it will take to achieve our goals and I have more confidence than ever that we will be able to assertively move to a good place.

It’s not good. There are a lot of humans getting a lot of value out of it—thousands of weekly actives with 10+ minutes engaged—but it’s not at all where it needs to be. There are a lot of us working on this, and we very much share your vision of what good looks like. It’s not so hard to imagine how it should work, we just have to do it and do it well.

I will say that dbt Cloud != the dbt Cloud IDE. Job scheduling is still by far the most widely-used part of dbt Cloud, and we’re recently made very significant improvements to the infrastructure behind this part of the product.

I think in the end, my issues with dbt might simply be a consequence of the fact that maybe dbt isn’t for me anymore. With large funding comes a demand for large customers, and large customers care more about the features on their roadmap than I do.

I hope that’s not true! You did link to a post that was exclusively about our enterprise roadmap, so that might be less relevant for you.

I do agree that there is an expanding tent—as the community matures, it’s not all early adopters any more. In order to give data professionals inside large businesses access to the same tooling that you’ve used for two years we have to build some stuff that you don’t care that much about. Our mission is to empower all data professionals, not just ones in the F500 and not just ones at VC-backed startups. We have to do both. And I think we’re actually building the team that can do that.

I am sorry that it’s taking longer than you’d like. IMO it’s not that our interests are misaligned, it’s that the work is hard and we’re not perfect.

---

Pedram, again, thank you for taking the time and care you did with your post. For now I’m going to get some sleep, but I’ll check back on comments tomorrow.