Replicate our Pipeline

📄️ NATS Setup

💡 In the NATS Setup Section, you will be guided through creating a new EKS Cluster, connecting to it, starting a NATS Server, connecting to it, and finally creating a Network Load Balancer.

📄️ Annotator

This documentation will take you through the process of containerizing the annotator and running it locally, and then migrating it to AWS.

ElasticSearch (ES) is a distributed search and analytics engine that provides database-like functionality to store, search, and/or analyze real-time data. This is a core component in our pipeline that comes sequentially after the Annotator. It consumes all the annotated data from the pipeline and serves multiple purposes–including monitoring the Client status, storing the data, and finally serving as a persistent database to the backend flask server.

📄️ Backend

Overview

📄️ Frontend

The application demoed here is an Airplane Tracker, which is similar to some other relevant applications, such as FlightAware and Flightradar24. However, we want to emphasize that the main focus of our work is not on the Airplane Tracker, but instead on the whole pipeline. When you set up the pipeline by following our tutorial, you can create a wide range of applications, not just the Airplane Tracker. The application here can also serve as one place for you to check if your pipeline is working or not.

📄️ Dashboards

⭐️ Grafana Endpoint (Dashboards)//a524500a80d314a64953fb349920eceb-1736286925.us-west-2.elb.amazonaws.com

📄️ Stress Test

This page will show you how to stress test our data pipeline

📄️ Overview