Api

Alright, Let’s Start You know how some things just need all the pieces lined up perfectly, like assembling furniture without an instruction manual? That’s kind of what we’ve been doing here—building a whole setup for handling cryptocurrency data, step by step. And now, it’s time to piece it together with Airflow. We’re not just moving data around aimlessly, though. The plan is, as it usually goes, to get the data from CoinMarketCap, shape it into something useful, and send it to the FastAPI app for storage. And once it’s sitting pretty in a database, we’ll have what you could call a playground for analysis. ...

In the last part of this series I explored the CoinMarketCap API. In this part i’ll be extending my pydantic API with a new domain to handle the data. While I could’ve gone the boring route of shoving CoinMarketCap data straight into my Postgres database using an Airflow DAG, I opted for the cooler, more maintainable, and yes, slightly over-engineered option: building an API layer. Why, you ask? Well, because APIs aren’t just for making developers look smart at conferences. They’re functional, versatile, and provide a host of benefits that I’m going to gleefully outline before diving into the process of extending my FastAPI app to handle cryptocurrency data. ...

Introduction We all have hobbies, right? Some people collect stamps; I like to collect data on small-cap cryptocurrencies. Because nothing says “good use of free time” like tracking obscure digital coins with names like MoonFluff and RocketPoodle. But in all seriousness, tracking small-cap coins isn’t just about watching volatile charts (though that’s part of the fun). It’s about building a robust data pipeline, exploring how to transform raw API outputs into actionable analytics, and flexing some data engineering muscles along the way. ...

Observability is one of those words that sounds impressive in meetings but gets downright intimidating when you sit down to implement it. Sure, logging failures and sending notifications sounds straightforward—until you’re knee-deep in designing custom tables, chasing down import errors, and trying to remember why you thought this was a good idea in the first place. But here’s the thing: observability isn’t just about logging and dashboards; it’s about creating a system where the data works for you, not the other way around. Alerts and notifications are the heart of this approach, delivering critical insights directly to you so you can act in real-time. When done right, observability transforms complexity into clarity and makes being data-driven not just possible, but practical. ...

So, you’ve got weather data from the Weather.gov API. You’ve built a FastAPI app that transforms and stores data using a Type 2 Slowly Changing Dimension (SCD) model. And now you’re wondering: “How do I tie this all together in a way that looks effortless but secretly makes me feel like a data wizard?” Enter Airflow, the glue that orchestrates your data pipeline. Today, we’re bringing together orchestration, transformation, and storage into a single, elegant workflow that will: ...

In the last article we talked about making our pydantic documentation as robust as possible. In this part, we’ll focus on containerizing the Weather Forecast API using Docker. By the end of this tutorial, your API will be running inside a Docker container, alongside a PostgreSQL database, all orchestrated using Docker Compose. Why Dockerize? 🛠️ Docker simplifies application deployment by packaging your application and its dependencies into a portable container. With Docker: ...

In the first two parts of this series (previous article), we laid the foundation for our weather forecast API and explored how to handle forecasts and historical revisions efficiently. In this part, we’ll focus on optimizing our Pydantic models to make the API documentation more accessible to AI tools, such as those that scrape OpenAPI specs for training or dynamic usage. The Problem 🧠 While your API might work perfectly for humans, AI tools often need more context and structure to understand it effectively. For example: ...

In the first part of this series, we set up a FastAPI application with a PostgreSQL database to store weather forecasts. We also designed a robust schema that tracks changes over time using a composite primary key. If you missed it, check out Part 1. In this post, we’ll: Create query endpoints to fetch forecasts. Implement a type-2 SCD approach for tracking forecast revisions. Discuss strategies for filtering and pagination. The Challenge 🧠 Weather forecasts often change over time as meteorological data improves. To support historical analysis and ensure data consistency, we need to: ...

When I set out to create a weather forecast API, I wanted something scalable, efficient, and built on modern technology. What started as a simple project to integrate weather data from the weather.gov API quickly grew into a complex, yet rewarding system. To keep things manageable, I decided to break this journey into multiple parts. In this post, I’ll cover the foundational steps to get started: setting up FastAPI, creating the database, and handling weather data with a robust schema. ...

Building a personal data project starts with gathering data. What better way to start than by exploring a free and well-documented API? In this article, we’ll dive into the National Weather Service API (api.weather.gov) to understand how it works, and we’ll write Python code to retrieve and explore weather data. So… what are you going to be doing? Let’s read through the docs and try to understand how to navigate the API.📜 I’m gonna write some Python code to fetch weather data. I’m gonna analyze the data structure of the forecastHourly endpoint. 📍 Understanding the API The National Weather Service API organizes data using a grid-point system. Each grid point covers an area of 2.5 km x 2.5 km. The /points endpoint is where you get information about retrieving data for a specific location. ...