Data

This piece is part of a larger series where we explore setting up a no-cost data stack in a home lab. In an earlier installment, we covered how to pull data from CoinMarketCap’s API and orchestrate it using Airflow. Now, we’ll shift gears to focus on transforming that raw cryptocurrency data into structured insights. Think of this as moving from raw ingredients to a well-prepped meal. 🍽️ Previous articles: Part 1 Part 2 Part 3 Here’s what we’ll tackle in this section: ...

Alright, Let’s Start You know how some things just need all the pieces lined up perfectly, like assembling furniture without an instruction manual? That’s kind of what we’ve been doing here—building a whole setup for handling cryptocurrency data, step by step. And now, it’s time to piece it together with Airflow. We’re not just moving data around aimlessly, though. The plan is, as it usually goes, to get the data from CoinMarketCap, shape it into something useful, and send it to the FastAPI app for storage. And once it’s sitting pretty in a database, we’ll have what you could call a playground for analysis. ...

In the last part of this series I explored the CoinMarketCap API. In this part i’ll be extending my pydantic API with a new domain to handle the data. While I could’ve gone the boring route of shoving CoinMarketCap data straight into my Postgres database using an Airflow DAG, I opted for the cooler, more maintainable, and yes, slightly over-engineered option: building an API layer. Why, you ask? Well, because APIs aren’t just for making developers look smart at conferences. They’re functional, versatile, and provide a host of benefits that I’m going to gleefully outline before diving into the process of extending my FastAPI app to handle cryptocurrency data. ...

Introduction We all have hobbies, right? Some people collect stamps; I like to collect data on small-cap cryptocurrencies. Because nothing says “good use of free time” like tracking obscure digital coins with names like MoonFluff and RocketPoodle. But in all seriousness, tracking small-cap coins isn’t just about watching volatile charts (though that’s part of the fun). It’s about building a robust data pipeline, exploring how to transform raw API outputs into actionable analytics, and flexing some data engineering muscles along the way. ...