Skip to content

Pipeline automation using ETL and PostgreSQL database

Notifications You must be signed in to change notification settings

cmwardcode/Movies-ETL

Repository files navigation

Analysis

Build an automated pipeline that takes in new data, performs appropriate transformations, and loads the data into existing tables that will update on a daily basis. This task was accomplished by creating an ETL function to collect and process data from three different sources and formats (JSON, Kaggle metadata, .csv) and adds it to a ProstgreSQL database.

Project Breakdown:

  1. Intake 3 data files
  2. Write an ETL function to read data
  3. Extract and transform the data
  4. Create a final combined Movie Database

Releases

No releases published

Packages

No packages published