Purple People! The Impact of Analysts. Management Seat Time. AlphaFold.
Episode 3 of the Analytics Engineering Podcast just dropped! Brian Amadio, data platform engineer extraordinaire at Stitch Fix, schooled Julia and me on experimentation and what a badass data org Stitch Fix is.
At this early stage of the podcast, reviews matter a lot! If you're enjoying the show, I’d be forever grateful if you’d leave a review on your platform of choice.
Enjoy the issue!
-Tristan
The Conversation Continues!
The past month has been 🔥🔥—the conversation that I highlighted last month has continued apace since then. The two posts I want to highlight are Anna Filippova’s We the Purple People and Benn Stancil’s The Third Rail.
Purple People is a meme that Anna, dbt Labs’ Director of Community, has quickly infected all of us with, so it’s something you’ll likely hear me talk about a lot. Here’s the core idea:
This short paragraph describes the core value I’ve always felt that I’ve brought to any organization I’m a part of. I always find myself curious about both the technical and business aspects of a problem and find myself confused when many folks prefer to sit solidly on one side of that divide. Bridging divides of all kinds is where creativity comes from! And your and my ability to bridge the technical / business divide is why we find ourselves able to solve problems that our colleagues cannot.
Anna’s post goes much deeper than this core problem, and presents plenty of food for thought. One thing that she didn’t get into here but is a question that I feel like the post absolutely begs is: if there are not enough purple people today, how do we create more of them as an industry? My guess is that this process happens primarily by starting with people who already speak “red” (business folks) and teach them to speak some “blue”. As per the conversation from last issue, it’s really the core domain knowledge, knowledge of the problem space, that is the central complexity of analytics.
In The Third Rail, Benn presents a hypothesis: what if much of the work produced by data analysts actually doesn’t provide a tremendous amount of business value?
In this hypothesis, there are certain pieces of analytical work that are almost priceless, but the median work done by a data analyst has lower value than the median work done by a software engineer. Benn self-describes as a data analyst, so he’s not coming from the outside to shit-talk the profession. He’s instead trying to ask hard questions and push for progress. He’s not saying this is true, he’s saying “is it? and what if it were?”
I find this to be an incredibly productive framing…it’s a framing that helps us see something that might have been hard for us to see otherwise. It’s easy to take a question like “how much value are you providing?” as a personal one…as if a suboptimal answer here is somehow just the fault of the individual or the role, and not of its entire organizational context.
In reality, though, the value that software engineers provide to their organizations has just gone through the roof in the past two decades. Software engineers in the 1980’s were absolute shadows of their modern incarnations. We like to talk about the advances in tooling, but it’s not just about the technology, it’s about the systems, processes, and culture of the organizations that software engineers participate in.
Today, it’s widely recognized that software engineers are critical organizational resources. VPs of Engineering don’t just operate sweatshops full of “code monkeys” who bang out features…less hyperbolically, they don’t take orders from other departments, they are peers of those departments. This power relationship has flipped, and this has happened because the profession itself has earned the trust and demanded the power that it now has. Treating software engineers with respect, getting them involved in the entire problem-solving process, caring about their career paths, has actually been seen to be good for an organization’s bottom line. The fact that FAANG companies know how to create contexts where engineers can be epically productive is not uncorrelated to their dominance in the modern economy.
I personally know that I have the ability to produce consistently high-impact analytical work when I have a) the time to do so and b) the organizational power to create the context that enables me to make an impact*. This doesn’t just look like finding needles in haystacks…it looks more like helping everyone at the organization ask great questions and come up with the simplest possible answers that help the team move forwards. Bridging the gap between
If you find yourself doing low-leverage work most of the time, ask yourself if there is a way you could be doing this work differently that creates more value (usually there is). And examine the context your organization is (often invisibly) creating for you. Analytics done well creates not only insights but alignment, clarity, and focus for an organization.
*: “What does this mean?” you might wonder. This is a great question, but will have to be explored another day! If this sounds interesting to you, hit reply and tell me your thoughts on the topic.
Putting the power of AlphaFold into the world’s hands
[…] today we’re sharing high-quality predictions for the shape of every single protein in the human body, as well as for the proteins of 20 additional organisms that scientists rely on for their research.
As researchers seek cures for diseases and pursue solutions to other big problems facing humankind – including antibiotic resistance, microplastic pollution, and climate change – they will benefit from fresh insights into the structure of proteins. Proteins are like tiny exquisite biological machines. The same way that the structure of a machine tells you what it does, so the structure of a protein helps us understand its function.
This is a very big deal, as recognized by a lot of folks…
This will be one of the most important datasets since the mapping of the Human Genome. (from the post)
[this] work is more significant because it opens a tremendous new vista for research and because it lives, in the public dataset, as a permanent gift to humanity[…] (source)
Et cetera. There are a lot of reasons to be frustrated with big tech and its “stewardship” (as it were) of innovation in AI. But this is a truly important advancement in human knowledge, and it’s worth pausing to recognize the achievement and the openness.
Management Seat Time
One of the things we as a profession need to figure out is what data teams look like and how to manage them. Often, data “managers” are just…the first person doing data at an organization who ends up getting stuck hiring the next set of data people! There are not many leaders or managers who have real experience building modern data teams…the math just works out that way because the whole practice is so new.
So when Erika Pullum wrote a piece on managing teams I jumped for joy. There is a whole Twitter / blogging community dedicated to software engineering management and it has truly made a difference in the field. Data needs to go through the same process for itself.
Here’s my favorite part of the post:
Time is an essential component of this kind of learning. You can’t shortcut the process. (…)
The thing is, reading about how to fire someone is different from doing it. It’s easy to read about the need for clear feedback and nod along, but hard not to soften feedback to someone you like at the moment you’re giving it. The impact of your decisions plays out over a longer timescale and it’s not always easy to see your own mistakes.
Seat time is sitting across the table from someone, listening to them describe a situation, and realizing you’ve seen this pattern before. You know how to navigate the situation — even or especially if it involves tricky interpersonal dynamics. When you come across a situation you haven’t seen before, it takes more time and energy to handle it than if you’ve seen variations of it several times.
Yep…this resonates hard. I am honestly not a great manager, but I have gotten better, and it’s what Erika calls “seat time” that I credit with most of this improvement.
While I’m on management…one of the most foundational posts in all the writing on software engineering management is Charity Majors’ The Engineer/Manager Pendulum. It’s so fantastic and bleeds Charity’s personality straight through the pixels into your soul. Here’s the core of the argument:
Fuck the whole idea that only managers get career progression. And fuckkkk the idea you have to choose a “lane” and grow old there. I completely reject this kind of slotting.
The best frontline eng managers in the world are the ones that are never more than 2-3 years removed from hands-on work, full time down in the trenches. The best individual contributors are the ones who have done time in management.
And the best technical leaders in the world are often the ones who do both. Back and forth. Like a pendulum.
🙌