Sterling JacksonUpgrading to Airflow 2.0We ran into a couple of gotchas during the update that weren’t covered by the excellent migration document that the Airflow team put…3 min read·Dec 18, 2020----
Sterling JacksonAirflow > Inject REST API Parameters Into Docker/Kubernetes OperatorsThis is a rough draft, but I wanted to get my thoughts out into the ether because I wasn’t able to find a solution on Google. Hopefully…1 min read·May 19, 2020----
Sterling JacksonSnowflake Deployment Quick StartWhen deploying a new Snowflake instance these are some of the steps that I typically perform to stand up the instance (e.g. clean up…3 min read·Apr 22, 2020----
Sterling JacksonAirflow at LirioLirio uses Airflow extensively to define and automate data engineering, data science and machine learning jobs across 4 environments. We…3 min read·Feb 4, 2020----
Sterling JacksonThoughts on scaling from 10GB to 10TB with Snowflake.This story has been on my mind for awhile now. I wanted to write this up around my one year anniversary at Lirio and look back on the…3 min read·Dec 16, 2019----
Sterling JacksonBERT-serving worker fails (silently) to start within Kubernetes pod.Silently failing is bad. Don’t do that.1 min read·Jul 26, 2019----
Sterling JacksonData, Privacy and Stewardship of Consumer DataI had an interesting conversation at lunch today about data, privacy and the responsible stewardship of consumer data. These are important…1 min read·Jul 19, 2019----
Sterling JacksonAirflow > Kubernetes Pod Operator & Resource LimitsI ran into an issue this week with a job consuming all available resources and crashing the host so I started exploring the operator’s…1 min read·Jul 16, 2019----