I've been thinking about this idea of co-evolution of culture and genetics lately as well - glad to see someone else has already written about it. Just here to say I love this post and the analogy. You've nailed what's been wrong with the data world for the past few years. 👍
It’s a tough time in history to encourage trust in any institution… however, I think you do that by making the institution as human as possible.
In my days as an analyst (the one sending the spreadsheet), I built trust by developing domain knowledge - so I could catch non-obvious (from a technical standpoint) errors in the data. I learned gross margin should be around x% or total revenue would be around y. I still made some mistakes, though, and by being quick to fix errors and transparent about how they happened, I could still keep the trust. As much as MDS tooling can reflect that humanity - I think the better.
Fascinating thoughts. Will think out loud as this is a really neat thing to explore. Prior to dbt, there were layers of trust embedded into the "typical" data workflow. For example, an analyst had to trust the Data Engineering Team that owned the MicroStrategy cube or Kimball model(s). But you're right that the trust has shifted from a single point (centralzed data infra owners) to a democratized audience: anyone that can use dbt. The risk still remains that if the analyst - either in consuming someone else's model or building their own - lacks understanding, incorrect data can get out there. But at least there's a chance for a knowledgeable/curious recipient to investigate and ask questions...
100% absolutely. i absolutely believe that the direction we're headed on represents massive progress (maybe i should have been more explicit here). what i'm trying to do is just help untangle (maybe for myself as much as anyone else) what we need to solve to get to the next level of success.
I've been thinking about this idea of co-evolution of culture and genetics lately as well - glad to see someone else has already written about it. Just here to say I love this post and the analogy. You've nailed what's been wrong with the data world for the past few years. 👍
It’s a tough time in history to encourage trust in any institution… however, I think you do that by making the institution as human as possible.
In my days as an analyst (the one sending the spreadsheet), I built trust by developing domain knowledge - so I could catch non-obvious (from a technical standpoint) errors in the data. I learned gross margin should be around x% or total revenue would be around y. I still made some mistakes, though, and by being quick to fix errors and transparent about how they happened, I could still keep the trust. As much as MDS tooling can reflect that humanity - I think the better.
Fascinating thoughts. Will think out loud as this is a really neat thing to explore. Prior to dbt, there were layers of trust embedded into the "typical" data workflow. For example, an analyst had to trust the Data Engineering Team that owned the MicroStrategy cube or Kimball model(s). But you're right that the trust has shifted from a single point (centralzed data infra owners) to a democratized audience: anyone that can use dbt. The risk still remains that if the analyst - either in consuming someone else's model or building their own - lacks understanding, incorrect data can get out there. But at least there's a chance for a knowledgeable/curious recipient to investigate and ask questions...
100% absolutely. i absolutely believe that the direction we're headed on represents massive progress (maybe i should have been more explicit here). what i'm trying to do is just help untangle (maybe for myself as much as anyone else) what we need to solve to get to the next level of success.