The Analytics Engineering Roundup

Share this post

Creating a Data Roadmap. ML Engineering. Plotly. Reliable Events. Airflow. [DSR #169]

roundup.getdbt.com

Creating a Data Roadmap. ML Engineering. Plotly. Reliable Events. Airflow. [DSR #169]

Tristan Handy
Jan 13, 2019
Share this post

Creating a Data Roadmap. ML Engineering. Plotly. Reliable Events. Airflow. [DSR #169]

roundup.getdbt.com

❤️ Want to support this project? Forward this email to three friends!

🚀 Forwarded this from a friend? Sign up to the Data Science Roundup here.

This week's best data science articles

Creating a Data Road Map

Creating a Data Road Map

As the new year rolls around, many Data leaders are thinking about (or have already created) 2019 road maps for their team and function. Since Data often works cross functionally with other teams, it’s key that you consider other team’s priorities and objectives in developing your road map. Below is a blueprint you can use to get started.

Annual planning is something that I suck at. When leading a company or team, I like to preserve optionality and allow myself to make planning decisions as information arises. But in many organizational contexts, the ability to set an annual roadmap is what will get you resources and the collaboration required from other teams. So…kind of important.

www.locallyoptimistic.com • Share

The Rise of ML Engineering

I fundamentally believe that 2019 will be the year of ML Engineering.

This article is spot on. It also pokes at one of my favorite frustrations curating this newsletter:

nobody needs yet another library or tutorial to build a 3-layer neural network on MNIST

Yes! Too much attention to model-building, which ends up only being about 10-20% of the time spent in the actual job. Instead, it’s time for the industry to focus on the things that will return business value: tooling, infrastructure, and best practices for creating ML pipelines.

medium.com • Share

What They Don’t Teach You in Machine Learning Courses

There is much more that goes into delivering impactful data science project than just a working model.

Similar topic: model-building isn’t the hard part, the hard part is how to operationalize data science within an organization. Software engineering projects ran poorly for decades before approaches and tooling became popular that allowed these projects to be successful more often than not.

medium.com • Share

The Next Level of Data Visualization in Python

Now that it’s 2019, it is time to upgrade your Python plotting library for better efficiency, functionality, and aesthetics in your data science visualizations.

Will Koehrsen has a great post on why it’s time to throw away matplotlib in favor of Plotly + cufflinks. I hadn’t heard of cufflinks before; it’s a good find. If you’re still plotting in matplotlib, you really should scan through the examples in this tutorial and consider making the switch.

towardsdatascience.com • Share

Understanding Your Users with Consistent and Reliable Event Data

One of the most common challenges companies have with event tracking is ensuring that they’re pushing consistent and reliable data through their event pipeline. Tools like Segment make it very easy to start pushing data, but that simplicity comes with a tradeoff: there are no guardrails to ensure the data you’re pushing conforms to any set of expectations.

The team at Airtasker worked to build their own tooling for enforcing a common set of event definitions and has incorporated it into their development workflow. In this post they explain their solution. It’s a quick read, and highly worthwhile. If you’ve experienced similar issues (and most of us have), it’s worth thinking about incorporating a similar approach at your org.

medium.com • Share

The Apache Software Foundation Announces Apache® Airflow™ as a Top-Level Project

Airflow was in incubation until now; it’s just been upgraded to an Apache TLP (top-level project). This is a mark of the maturity of its consensus-based governance processes.

It’s unlikely that this has any immediate impact for you, but it’s worth noting that one of the main tools in the data engineering ecosystem is now a mature project in the world’s leading open source software foundation. Good news.

globenewswire.com • Share

Lazydata: Scalable Data Dependencies for Python Projects

Very cool project!

Problem: Keeping all data files in git (e.g. via git-lfs) results in a bloated repository copy that takes ages to pull. Keeping code and data out of sync is a disaster waiting to happen.

Solution: lazydata only stores references to data files in git, and syncs data files on-demand when they are needed.

Why: The semantics of code and data are different - code needs to be versioned to merge it, and data just needs to be kept in sync. lazydata achieves exactly this in a minimal way.

github.com • Share

Thanks to our sponsors!

Fishtown Analytics: Analytics Consulting for Startups

At Fishtown Analytics, we work with venture-funded startups to build analytics teams. Whether you’re looking to get analytics off the ground after your Series A or need support scaling, let’s chat.

www.fishtownanalytics.com • Share

Stitch: Simple, Powerful ETL Built for Developers

Developers shouldn’t have to write ETL scripts. Consolidate your data in minutes. No API maintenance, scripting, cron jobs, or JSON wrangling required.

www.stitchdata.com • Share

By Tristan Handy

The internet's most useful data science articles. Curated with ❤️ by Tristan Handy.

Tweet Share

If you don't want these updates anymore, please unsubscribe here.

If you were forwarded this newsletter and you like it, you can subscribe here.

Powered by Revue

915 Spring Garden St., Suite 500, Philadelphia, PA 19123

Share this post

Creating a Data Roadmap. ML Engineering. Plotly. Reliable Events. Airflow. [DSR #169]

roundup.getdbt.com
Comments
TopNewCommunity

No posts

Ready for more?

© 2023 dbt Labs Inc.
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing