Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start of production no possible due to errors in docker-compose-non-dev.yml and docker/.env #29309

Open
2 of 3 tasks
matsvitt opened this issue Jun 19, 2024 · 3 comments
Open
2 of 3 tasks
Labels
deploy install:docker Installation - docker container

Comments

@matsvitt
Copy link

Bug description

Hi there,

I tried to spin up the prod version with docker-compose-non-dev.yml from the current master branch and get two problems: The call

docker compose -f docker-compose-non-dev.yml up -d

results in the error:
validating docker-compose-non-dev.yml: services.db.env_file.0 must be a string

So I changed the env file entries on the docker-compose file to:

env_file: ./docker/.env-non-dev

The the spin up works however when I try to create a new admin user using

docker exec superset_app superset fab create-admin

I see that the database connection is refused? The reason seems to be that the docker/.env file sets DATABASE_HOST = db. However the database service “db” defined inside the docker-compose-non-dev.yml is set to superset_db.
After I adjusted the entry in docker/.env to

DATABASE_HOST=superset_db

the superset service works fine.
Did I miss anything in order to prepare the spinup for production? Did I overlook anything?

How to reproduce the bug

  1. checkout the master branch.
  2. run docker compose -f docker-compose-non-dev.yml up -d
  3. run docker exec -it superset_app superset fab create-admin

Screenshots/recordings

The startup of the prod service using docker compose -f docker-compose-non-dev.yml gives this error:

validating docker-compose-non-dev.yml: services.db.env_file.0 must be a string

Upon the creation of a new admin user the error is:

2024-06-19 16:37:17,703:DEBUG:superset.utils.logging_configurator:logging was configured successfully
2024-06-19 16:37:17,718:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) connection to server at "db" (10.0.1.231), port 5432 failed: FATAL: password authentication failed for user "superset"

Superset version

master / latest-dev

Python version

Not applicable

Node version

Not applicable

Browser

Not applicable

Additional context

No response

Checklist

  • I have searched Superset docs and Slack and didn't find a solution to my problem.
  • I have searched the GitHub issue tracker and didn't find a similar bug report.
  • I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
@dosubot dosubot bot added deploy install:docker Installation - docker container labels Jun 19, 2024
Copy link

dosubot bot commented Jun 19, 2024

@matsvitt Hello there! I'm here to help you with any bugs, questions, or contributions you have. Just give me a shout, and I'll do my best to assist you while we wait for a human maintainer.

To address the issues you're encountering with the docker-compose-non-dev.yml and docker/.env files, follow these steps:

  1. Ensure the .env file is properly configured:

    • Create a docker/.env file with your unique random secure passwords and SECRET_KEY.
    • Example content for docker/.env:
      POSTGRES_DB=superset
      POSTGRES_USER=superset
      POSTGRES_PASSWORD=your_postgres_password
      SECRET_KEY=your_secret_key
      DATABASE_HOST=superset_db
  2. Update the docker-compose-non-dev.yml file:

    • Ensure the env_file entries are correctly specified as strings.
    • Example docker-compose-non-dev.yml:
      version: "4.0"
      services:
        redis:
          image: redis:7
          container_name: superset_cache
          restart: unless-stopped
          volumes:
            - redis:/data
      
        db:
          env_file:
            - ./docker/.env
          image: postgres:15
          container_name: superset_db
          restart: unless-stopped
          volumes:
            - db_home:/var/lib/postgresql/data
            - ./docker/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
      
        superset:
          env_file:
            - ./docker/.env
          build:
            context: .
            target: dev
            cache_from:
              - apache/superset-cache:3.10-slim-bookworm
          container_name: superset_app
          command: ["/app/docker/docker-bootstrap.sh", "app-gunicorn"]
          user: "root"
          restart: unless-stopped
          ports:
            - 8088:8088
          depends_on:
            - db
            - redis
          volumes:
            - ./docker:/app/docker
            - superset_home:/app/superset_home
      
        superset-init:
          container_name: superset_init
          build:
            context: .
            target: dev
            cache_from:
              - apache/superset-cache:3.10-slim-bookworm
          command: ["/app/docker/docker-init.sh"]
          env_file:
            - ./docker/.env
          depends_on:
            - db
            - redis
          user: "root"
          volumes:
            - ./docker:/app/docker
            - superset_home:/app/superset_home
          healthcheck:
            disable: true
      
        superset-worker:
          build:
            context: .
            target: dev
            cache_from:
              - apache/superset-cache:3.10-slim-bookworm
          container_name: superset_worker
          command: ["/app/docker/docker-bootstrap.sh", "worker"]
          env_file:
            - ./docker/.env
          restart: unless-stopped
          depends_on:
            - db
            - redis
          user: "root"
          volumes:
            - ./docker:/app/docker
            - superset_home:/app/superset_home
          healthcheck:
            test:
              [
                "CMD-SHELL",
                "celery -A superset.tasks.celery_app:app inspect ping -d celery@$$HOSTNAME",
              ]
      
        superset-worker-beat:
          build:
            context: .
            target: dev
            cache_from:
              - apache/superset-cache:3.10-slim-bookworm
          container_name: superset_worker_beat
          command: ["/app/docker/docker-bootstrap.sh", "beat"]
          env_file:
            - ./docker/.env
          restart: unless-stopped
          depends_on:
            - db
            - redis
          user: "root"
          volumes:
            - ./docker:/app/docker
            - superset_home:/app/superset_home
          healthcheck:
            disable: true
      
      volumes:
        superset_home:
          external: false
        db_home:
          external: false
        redis:
          external: false
  3. Start the services:

    • Run the following command to start the services defined in docker-compose-non-dev.yml:
      docker-compose -f docker-compose-non-dev.yml up -d
  4. Initialize the database and create an admin user:

    • Run the following command to initialize the database and create an admin user:
      docker-compose -f docker-compose-non-dev.yml run --rm superset-init

By following these steps, you should be able to spin up the production version and create an admin user without encountering errors related to the env_file and database connection [1].

To continue talking to Dosu, mention @dosu.

@matsvitt
Copy link
Author

matsvitt commented Jun 19, 2024 via email

Copy link

dosubot bot commented Jun 19, 2024

Yes, it would be correct to create a pull request to address the issues in the docker-compose-non-dev.yml and docker/.env files in the master branch [1].

To continue talking to Dosu, mention @dosu.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deploy install:docker Installation - docker container
Projects
None yet
Development

No branches or pull requests

1 participant