Skip to content

GoldenGate is dynamic/static web scraper, and It is integrated to Google Cloud Storage and Firebase Realtime DB

License

Notifications You must be signed in to change notification settings

emigatech/goldengate

Repository files navigation

GoldenGate

GoldenGate Schheme

environment files on production

  • .env
  • docker-compose.yaml
  • Dockerfile
  • process.yml

add google storage config json to ./config/google-cloud as google-storage.json add firebase config json to ./config/google-cloud as firebase.json

GoldenGate Api

  • POST /api/auth/login (Rate Limit for 3rd request)
    • email:string
    • password:string
  • POST /api/auth/register (Rate Limit for 3rd request)
    • email:string
    • password:string
    • firstname:string
    • lastname:string
  • POST /api/fetch/(desktop|mobile)
    • Authorization: Bearer (token)
    • url:string
  • POST /api/storage/migrate
    • Authorization: Bearer (token)
    • key:string
  • POST /api/scrape
    • Authorization: Bearer (token)
    • cache:string (optional)
    • url:string
    • key:string (optional)
  • POST /api/worker/start
    • url:string
    • token:string
    • device:string
      • default desktop, mobile
  • GET /api/data/results:key (Cache)
    • Authorization: Bearer (token)
    • key:string

About

GoldenGate is dynamic/static web scraper, and It is integrated to Google Cloud Storage and Firebase Realtime DB

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published