Posts

This piece is part of a larger series where we explore setting up a no-cost data stack in a home lab. In an earlier installment, we covered how to pull data from CoinMarketCap’s API and orchestrate it using Airflow. Now, we’ll shift gears to focus on transforming that raw cryptocurrency data into structured insights. Think of this as moving from raw ingredients to a well-prepped meal. 🍽️ Previous articles: Part 1 Part 2 Part 3 Here’s what we’ll tackle in this section: ...

Getting my PL-300 certification wasn’t just about passing a test—it was a way to validate the skills I’ve honed over the past year with Power BI. This post is for anyone considering the certification or curious about how Power BI fits into the world of data analytics. Let me walk you through my journey, from starting with Power BI to acing the exam. The Backstory: From SSRS to Power BI 🕰️ Back in January 2023, I picked up a personal Power BI license to see what all the buzz was about. My workplace had been using MicroStrategy since early 2022 after transitioning away from a decade of SQL Server Reporting Services (SSRS) and Excel. While MicroStrategy was a big improvement, there was growing interest in Power BI, especially from leadership, for its flexibility and modern approach to analytics. ...

So, Grafana is, like, your go-to tool for making charts and dashboards that really grab attention. 📊 It’s the kind of thing that gets people talking about data in a way that, you know, doesn’t feel so dull. Anyway, let’s jump into getting it up and running with Docker! 🐳 Step 1: Toss Together a Compose File 📄 Alright, the first thing you’ll need is a little file called docker-compose.yml. It’s basically like a recipe for Docker. 🍴 Here’s what it might look like if you’re keeping things simple: ...

Alright, Let’s Start You know how some things just need all the pieces lined up perfectly, like assembling furniture without an instruction manual? That’s kind of what we’ve been doing here—building a whole setup for handling cryptocurrency data, step by step. And now, it’s time to piece it together with Airflow. We’re not just moving data around aimlessly, though. The plan is, as it usually goes, to get the data from CoinMarketCap, shape it into something useful, and send it to the FastAPI app for storage. And once it’s sitting pretty in a database, we’ll have what you could call a playground for analysis. ...

In the last part of this series I explored the CoinMarketCap API. In this part i’ll be extending my pydantic API with a new domain to handle the data. While I could’ve gone the boring route of shoving CoinMarketCap data straight into my Postgres database using an Airflow DAG, I opted for the cooler, more maintainable, and yes, slightly over-engineered option: building an API layer. Why, you ask? Well, because APIs aren’t just for making developers look smart at conferences. They’re functional, versatile, and provide a host of benefits that I’m going to gleefully outline before diving into the process of extending my FastAPI app to handle cryptocurrency data. ...

Introduction We all have hobbies, right? Some people collect stamps; I like to collect data on small-cap cryptocurrencies. Because nothing says “good use of free time” like tracking obscure digital coins with names like MoonFluff and RocketPoodle. But in all seriousness, tracking small-cap coins isn’t just about watching volatile charts (though that’s part of the fun). It’s about building a robust data pipeline, exploring how to transform raw API outputs into actionable analytics, and flexing some data engineering muscles along the way. ...

Observability is one of those words that sounds impressive in meetings but gets downright intimidating when you sit down to implement it. Sure, logging failures and sending notifications sounds straightforward—until you’re knee-deep in designing custom tables, chasing down import errors, and trying to remember why you thought this was a good idea in the first place. But here’s the thing: observability isn’t just about logging and dashboards; it’s about creating a system where the data works for you, not the other way around. Alerts and notifications are the heart of this approach, delivering critical insights directly to you so you can act in real-time. When done right, observability transforms complexity into clarity and makes being data-driven not just possible, but practical. ...

So, you’ve got weather data from the Weather.gov API. You’ve built a FastAPI app that transforms and stores data using a Type 2 Slowly Changing Dimension (SCD) model. And now you’re wondering: “How do I tie this all together in a way that looks effortless but secretly makes me feel like a data wizard?” Enter Airflow, the glue that orchestrates your data pipeline. Today, we’re bringing together orchestration, transformation, and storage into a single, elegant workflow that will: ...

In the last article we talked about making our pydantic documentation as robust as possible. In this part, we’ll focus on containerizing the Weather Forecast API using Docker. By the end of this tutorial, your API will be running inside a Docker container, alongside a PostgreSQL database, all orchestrated using Docker Compose. Why Dockerize? 🛠️ Docker simplifies application deployment by packaging your application and its dependencies into a portable container. With Docker: ...

In the first two parts of this series (previous article), we laid the foundation for our weather forecast API and explored how to handle forecasts and historical revisions efficiently. In this part, we’ll focus on optimizing our Pydantic models to make the API documentation more accessible to AI tools, such as those that scrape OpenAPI specs for training or dynamic usage. The Problem 🧠 While your API might work perfectly for humans, AI tools often need more context and structure to understand it effectively. For example: ...