Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IBCDPE-1004] Implement Postgres server #22

Closed
wants to merge 45 commits into from

Conversation

BryanFauble
Copy link
Contributor

@BryanFauble BryanFauble commented Aug 17, 2024

TODO: Clean up the branch before marking the PR for ready for review. I had deployed out a testing stack in spacelift.

Problem:

  1. Airflow did not recommend using the airflow instance from their helm chart for a production load due to the maintenance requirements of handling it.

Solution:

  1. Implementing https://cloudnative-pg.io/documentation/current/
  2. Updating the connections for airflow to use the new connection and set up the account secrets automatically.

Description of tool:

CloudNativePG is an open source operator designed to manage PostgreSQL workloads on any supported Kubernetes cluster. It defines a new Kubernetes resource called Cluster representing a PostgreSQL cluster made up of a single primary and an optional number of replicas that co-exist in a chosen Kubernetes namespace for High Availability and offloading of read-only queries.

Applications that reside in the same Kubernetes cluster can access the PostgreSQL database using a service which is solely managed by the operator, without having to worry about changes of the primary role following a failover or a switchover.

Testing:

  1. Verified that a deployment of the operator and cluster for airflow worked, airflow came up healthy. I was able to log in, and trigger a DAG.

@spacelift-int-sagebionetworks spacelift-int-sagebionetworks bot temporarily deployed to spacelift/root-spacelift-administrative-stack August 19, 2024 16:02 Inactive
@spacelift-int-sagebionetworks spacelift-int-sagebionetworks bot temporarily deployed to spacelift/root-spacelift-administrative-stack August 19, 2024 16:32 Inactive
@spacelift-int-sagebionetworks spacelift-int-sagebionetworks bot temporarily deployed to spacelift/dpe-dev-kubernetes-deployments August 19, 2024 16:35 Inactive
@spacelift-int-sagebionetworks spacelift-int-sagebionetworks bot temporarily deployed to spacelift/root-spacelift-administrative-stack August 19, 2024 17:15 Inactive
@spacelift-int-sagebionetworks spacelift-int-sagebionetworks bot temporarily deployed to spacelift/dpe-dev-kubernetes-deployments August 19, 2024 17:21 Inactive
@BryanFauble
Copy link
Contributor Author

Will open this back up shortly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant