How to Actually Move Up the Stack
A handbook for interesting times
Last week I wrote about why analytics engineers are being called, again, to move up the stack. The piece tried to make the case that the work many of us have been doing for the past several years was, in retrospect, preparation for exactly this moment. The primary response was - ok that sounds great. What do I actually do, right now, to set myself up for the coming wave.
Before I get into any of the practical stuff, the caveat I want to repeat throughout this piece: nobody knows how this is all going to play out - and in fact the smartest people have very wide bars on the possibility space for the next few years. We know this train is going somewhere, but exactly where is unclear.
The following is not a perfect plan but it is what I have seen work, mostly from watching people I trust, partly from things I have stumbled into myself. The best set of moves I currently know how to make, offered in that spirit. So let’s get into it.
Timing matters
These transitions tend to have a sweet spot.
If you try to push too early, you have to drag everyone else with you, and that is exhausting and often unrewarding (hopefully we’re past this bit in most orgs).
If you wait until everyone else has done it, the easy wins are gone. The window where it is just slightly early is the window where it feels like surfing a wave instead of getting overtaken by it. (My team has gotten a little sick of me scheduling meetings called “ride the wave” where we prototype AI demos but you know what it works).
We are in that window right now. And it is moving faster than any of the previous ones, which means the window is also narrower than the ones before it.
What that means in practice is a mindset shift, and the mindset shift comes before any of the tactical stuff. It also means making the emotional transition, while this is fun and exciting, it is also scary and highly uncertain. Take the time to sit with that and reflect in it, then decide your action plan.
Your job, starting now, is to move up the stack and figure out how to be impactful in the coming paradigm.
Not as a side project or a 10% time experiment after you finish the your ticket queue. As the actual point of what you are doing. If you are lucky, your organization will recognize this and give you space to do it.
The honest truth is that, in most cases, nobody is going to walk over to your desk and tap you on the shoulder and say we have decided you should spend a quarter learning how to build agentic systems on top of our data. It mostly does not work like that. It almost always requires some amount of courage, or initiative, or both. It means carving out time. It means making this your priority even when it is not the official priority.
Finding Signal in All the Noise
Just as important as committing to doing this is finding good information inputs that will actually help you accomplish this transition.
The signal to noise ratio out there right now is not great If you go looking for “best practices for AI in data,” you will find a thousand posts of varying quality. The patterns and best practices for actual data work in the agentic era are still being built. They are very much still being figured out and still up for debate. The signal is real but it is sparse, and you have to work to find it.
Some of what I would point you at: the OpenAI in-house data agent post, the Ramp data agent writeup, and a small handful of others. Read them carefully. Not skim. Actually read them, and ask yourself what the people who wrote them did differently from what your team is doing right now. Consume everything they put out publicly. Be prolific in how you absorb their work. And then build your version of what they did, scaled to whatever your situation allows.
Depending on the size of your organization, the resources you have, and the political surface area you control, “your version” might look very different from theirs. That is fine. The point is not to copy. The point is to internalize the pattern of how good work in this space gets made, and then, to apply it locally.
The One Thing You Cannot Skip
At the end of the day, nothing beats hands on experience which is why you need access to real production data, with best-in-class agent tooling on top of it, and you need it now.
If you cannot get an agent pointed at production data in your current role, your first priority is either to fix that internally or to find a role where it is possible. I am not saying that lightly. I know that internal security and access controls exist for very good reasons, and I am not suggesting anyone try to route around them. But I am saying that the experience of working with a real data agent on real data is so different from reading about it, or watching demos of it, or playing with toy datasets, that until you have done it you are essentially flying blind on the most important question of the next few years.
Push to get the access. Make the case. Find the security-approved path. Get the budget for the tools. If after a lot of effort you still cannot get there, take that as serious data about the environment you are in.
Case studies in finding projects with an edge
This also sounds a little abstract - I want to give you some examples of how I’ve applied this exact formula over the past few years to move myself, my team and all dbt users up the stack and prepare us for this moment.
I am not a product manager, but it is my job to make sure that you all have the best tools you can have in order to ride the AI wave. So I’ve been keeping my eyes and ears open for things that we at dbt can do to help dbt users adopt AI. By making it my job to do so, even when it wasn’t obvious, by making sure I had the right information input and then by acting locally when the information made it clear it was time to move.
First - it was by proving that dbt is useful for connecting language models to your data to ask business questions.The semantic-layer versus text-to-SQL work we have been prototyping at dbt got a much sharper external reference point when a paper got published putting numbers around the same intuitions our team had been chasing, and we suddenly had something concrete to benchmark against.
Next it was by reading a blog post from Ethan Mollick Now is the Time for Grimoires - it changed how I thought about prompt-as-artifact, and led me to work on building dbt Assist, the first official copilot for dbt.
Next was the first prototype of the dbt MCP server. I vibe-coded it on a weekend because I had been tracking MCP for a while, and then I saw to a talk at Swyx’s AI Engineer conference about MCP, and somewhere in the middle of that talk I realized: we need this, and we need it now. There was no notification. There was no Slack message from leadership saying it was time. The signal came from being in the room, paying attention, and trusting the prickle on the back of my neck when I felt it. And then being fortunate that talented engineers across the company picked up my half-baked prototype and turned it into a real product growing exponentially to this day.
The same thing happened with skills. I saw a talk on Claude’s skills feature, also at one of Swyx’s events, and a month later we launched dbt agent skills.
The pattern, if there is one, is this: immerse yourself in the work, in the community, in the writing, in the talks, in the experiments other people are running. And then when something clicks, when you feel that we need this and we need it now feeling, trust it and act on it. Get involved in the conversation. Build the thing internally. Post about it publicly. Submit conference talks. Submit meetup talks. Write the LinkedIn post.
The flywheel of finding interesting ideas or patterns, doing the work to apply that to where you are as best you can and then sharing the work is, I think, one of the most important career moves available to anyone during periods of high change.
There are a huge number of specific things you could be working on. The dbt MCP server. Agent skills. Building an analyst agent. Building a dbt-native harness on top of Codex. Building automated CI workflows. There are many, many more, and the list is growing every week. The specific project matters less than the fact that you are doing one.
Once you get in the game everything feels different.
Of course it’s not that simple
I want to bring in something that Salim wrote in response to last week’s piece, because it gets at one of the important differentiators between the last phase change and this one. Quoting from the comment directly:
If you were a passionate analytics engineer before the existence of coding agents with the boring work included, the future should be just as exciting. At least, I feel that way. But this time, the transition has one difference in my opinion, which I think makes it harder: it requires a mindset change across the entire company, not just the data team.
When dbt came along, you could largely adapt your own workspace in isolation, and the external environment in the company did not need to move with you to a large extent. Moving up the stack with agents is different. If the goal is to democratize data across the organization, make everyone a data person, and free the analytics engineer for high value work, then the whole company has to rethink how it operates internally. For example, a business update shared by the head of marketing in an all hands, previously held in an analyst’s head, now needs to be captured in a format an agent can consume. This organizational knowledge management problem is everyone’s job in the company. Similarly, it will not be enough to deploy a data agent to Slack, but make sure that every stakeholder has a base understanding of how to ask a question to the agent. These problems are not actually related to any context engineering problems that we have mostly been talking about in the data community.
So change management will be the biggest barrier to capturing the value of the agentic era for data, and a visionary data team will not move an organization alone.
This is a very fair point. Analytics engineering has always involved organizational change, of course. It created a whole new job title, a career ladder, an org structure. That was hard!. When dbt came along, you could adapt your own corner of the world in relative isolation. The marketing team did not have to change anything about how it operated for you to start using version control on your transformations.
The agentic transition is different. If the goal is to truly democratize data, to make every employee a data person, and to free analytics engineers for higher-value work, then the whole company has to rethink how it operates.
These are not problems that look much like the context engineering work the data community has been talking about. They look like culture work. They look more like organizational design and change management. And change management may very well end up being the biggest barrier to capturing the value of all of this. A visionary data team does not move an organization on its own.
I do not have a clean answer for this. I am not sure anyone does yet. But I think the people who figure out how to work this dimension, the people who can build the agentic data systems and help their organizations metabolize the change at the same time, are going to be doing some of the most important work in the industry.
Charting the future, together
Before I close I want to come back to the thing I said at the top.
The level of uncertainty in this moment is extraordinarily high. Higher than it has ever been in my career, by a margin that makes my head spin a little when I sit with it. I do not want any of this piece to read as I figured out how to do this in the dbt transition, so now you can do the same thing here. I am not saying that. I do not think anyone knows how this plays out.
What I am saying is something more limited. If your goal is to set yourself up as well as possible for a world in which analytics engineers are deeply integrated with agentic workflows, then in aggregate, from what I have seen, this is the highest-leverage set of moves I currently know how to recommend. It is not a guarantee. It is the best bet I can offer.
And if you find yourself having to choose between clearing the ticket queue and spending a morning building something with agents on real data, I think the second one compounds in ways the first one does not.
It’s the bet I’m making myself.
We have been here before, in a smaller way. We moved up the stack once already, and the things we learned along the way, the modeling instincts, the systems thinking, the hard-won understanding of how organizations actually use data, all of that came with us and made the next thing possible. I think that is going to be true again.
The wave is here. It will not wait for us. And the work of figuring out what comes next is some of the most interesting work I have ever gotten to do. I hope you find a way to do some of it too.
Jason
Appendix - what I’m reading to stay up to date on AI
It’s important to have a lot of input, to recognize the strengths and weaknesses of various commentators and learn over time how to sensemake across them. Here are my sources, ordered from most measured to most speculative, but all sources I consider high quality for the niche they occupy.
Ethan Mollick - One useful thing - for high level, accessible overviews of the AI landscape
AI daily brief - For solid daily analysis of the latest AI news from an enterprise perspective
Anything from the AI engineer conference or associated properties
METR and Redwood research for technical research on AI capabilities including the personal writings of their team’s including Ajeya Cotra
The Cognitive revolution podcast - practical conversations with people across the AI industry, focus on people working at the forefront of interesting problems
Hyderdimensional by Dean ball for reflections on AI progress and what it means for policy and the longer arc of history. Particularly recommend the most recent post on Mythos
Don’t Worry About the Vase - good collection of relatively high signal information from across the internet - a lot of content but still more manageable than trying to track it all yourself
Andrew Curren on X - breaking news and theorization from industry insiders. Shares rumors and speculation but as far as I can tell one of the more accurate accounts to do so