Skip to content

AD4GD/Component-AirQualityDataIngestion

 
 

Repository files navigation

Deployment Tests


Logo

AstonAirQuality API

A Fast API project to manage Aston University's fleet of air quality sensors and handle batch data ingestion tasks.
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents
  1. About The Project
  2. Getting Started
  3. Addtional Information
  4. Contact

About The Project

Project Screenshot

This project is a Fast API application that provides a RESTful API to manage Aston University's fleet of air quality sensors and handle batch data ingestion tasks. The API is hosted on AWS Lambda and is connected to a PostGreSQL database hosted on AWS RDS. The API is secured using JWT authentication and is documented using Swagger UI. The API is also connected to a cron job that runs every 24 hours to ingest data from the Plume Labs API, the Zephyr API and the SensorCommunity API.

(back to top)

Built With

  • Fast-API
  • AWS-Lambda
  • PostGreSQL
  • Docker

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

Complete setup project with Firebase Services

  1. Setup a firebase project
  2. Enable Authentication and Realtime database services
  3. Copy the admin sdk into the app/config directory in the project files.

Local setup without Firebase Services

  1. Firevase services are not required, however you will still need to complete the environment variables setup in the next step.

Docker Setup

  • Docker Desktop - install from here

Local Setup

  • Python 3.9.6+ - install from here

Installation

The following steps are requried for both local and docker setups.

  1. Clone the repo

    git clone AstonAirQuality/AirQuality-API
  2. Enter your enviornment variables in .env-template

     PLUME_EMAIL= YOUR_PLUME_EMAIL
     PLUME_PASSWORD= YOUR_PLUME_PASSWORD
     JWT_SECRET= YOUR_JWT_SECRET
     ZEPHYR_USERNAME = YOUR_ZEPHYR_USERNAME
     ZEPHYR_PASSWORD = YOUR_ZEPHYR_PASSWORD
     SC_USERNAME = YOUR_SC_USERNAME
     SC_PASSWORD = YOUR_SC_PASSWORD
     CRON_JOB_TOKEN= YOUR_CRON_JOB_TOKEN

    If you are using Firebase services, you will also need to add the following variables:

     FIREBASE_API_KEY= YOUR_FIREBASE_API_KEY
     FIREBASE_AUTH_DOMAIN= YOUR_FIREBASE_AUTH_DOMAIN
     FIREBASE_PROJECT_ID= YOUR_FIREBASE_PROJECT_ID
     FIREBASE_STORAGE_BUCKET= YOUR_FIREBASE_STORAGE_BUCKET
     FIREBASE_MESSAGING_SENDER_ID= YOUR_FIREBASE_MESSAGING_SENDER_ID
     FIREBASE_APP_ID= YOUR_FIREABASE_APP_ID
     FIREBASE_MEASUREMENT_ID= YOUR_FIREBASE_MEASUREMENT_ID
     # remember to store the JSON inside app/config
     FIREBASE_SERVICE_ACCOUNT= YOUR_PATH_TO_SERVICE_ACCOUNT_JSON 
     FIREBASE_DATABASE_URL= YOUR_FIREBASE_DB_URL
  3. Copy .env-template to .env and enter your enviornment variables

    cp .env-template .env
  4. Now follow the setup instructions for your chosen setup method.

Docker Setup

  1. From the project root directory, run the command
    docker-compose up
  2. After the containers have been built visit http://localhost:8000/docs to view the API documentation

Local Setup

  1. From the project root directory, run the command below to install the project dependencies
    pip install -r requirements.txt
  2. From the project root directory, run the command below to start the API
    uvicorn app.main:app --reload
  3. After the API has started visit http://localhost:8000/docs to view the API documentation

Running Tests

Docker Setup

  1. From the project root directory, run the command below to start the test container
    docker-compose -f docker-compose-testenv.yml -p test up -d
  2. Choose one of the following methods to run the tests
    1. Using VSCode, open the project root directory and run the tests using the test explorer
    2. From the project root directory, run the command below to run the tests using Docker
      docker exec -it test_app_ python -m unittest discover -s testing -p test_*.py
    3. From the project root directory, run the command below to run the tests with coverage in a local terminal (Python installation required)
      python -m coverage run -m unittest discover -s testing -p test_*.py
      Then run the command below to view the coverage report
      python -m coverage report --omit="*/testing*
      Or run the command below to export the coverage report in html
      python -m coverage html --omit="*/testing*"
  3. To delete the container, run the command
    docker-compose -f docker-compose-testenv.yml -p test down --volumes

(back to top)

Additional Information

Contact

Twitter (X): @dev_riyad
Email: [email protected]
Project Link: https://github.com/AstonAirQuality/AirQuality-API

(back to top)

About

A Fast API project to manage Aston University's fleet of low-cost air quality sensors and handle automated data ingestion tasks.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 93.1%
  • Python 6.9%