A scalable cloud-based AI/ML-facilitating data pipeline for spectrum
By: Victor Li, Zhen Jiang, Shruti Satrawada, Sivani Voruganti, Gen Yang
This tutorial and documentation is for an adaptable end-to-end data pipeline for radio spectrum data that allows interested practitioners or the general public to harness, understand, and utilize distributed radio spectrum data without access to company-level resources. Currently, there is no mature end-to-end solution for access to distributed spectrum data, which is complex and dynamic in nature, with additional requirements for custom post-processing depending on the intended use.
Spectrum is a resource consisting of the range of electromagnetic radiation used to transmit information wirelessly. The radio frequency spectrum powers all communication around us–from cell phones to WiFi and more. There are already a myriad of streams of this data being broadcasted without people being aware that this information is accessible and can be used for new and innovative applications. Even if people are aware of this data, reading, processing, and utilizing it is not straightforward–which is exactly what our project is meant to address with our customizable pipeline and documentation.
We have built out an example Airplane Tracker application built on top of the pipeline from beginning to end, showcasing how the pipeline can be adapted to process diverse data types and improve accessibility to spectrum resources.
đź’ˇ Link to our poster & paper for more information: SpecPipe.pdf, Paper.pdf
1. About the Project​
This document contains a project brief, image of a physical model, and a link to our paper for additional context that may be useful.
2. Quick Local Setup & Run Through​
This quick tutorial guides you through connecting a Software Defined Radio to your laptop, reading in Airplane Data, unpacking it, and outputting it to your console after cleaning and appending outside data information. We recommend doing this part before moving on to step 3.
3. Replicate our Pipeline​
This final segment of the tutorial guides you through building out the entire pipeline, component-by-component, within a distributed AWS cloud environment.
A core aspect of our project is the documentation and tutorials we provide, to allow others to replicate and tune our work for their own use cases. Our aim was for the documentation to be available on a publicly-accessible website and to be as easy to use as possible.
We would greatly appreciate any feedback on how the documentation can be improved!