Skip to content

Commit

Permalink
[dagster-io#21] fix evidence can't locate local duckdb files
Browse files Browse the repository at this point in the history
export case sensitive evidence environment variables.

also provided instructions in .env.example on how to configure project
for local use.

removed `dagster_hybrid_demo.` from schema name in FROM statement as the
tables created by duckdb exist in the `main` schema, was causing error
where evidence couldn't find the data sources.
  • Loading branch information
maxestorr committed Apr 28, 2024
1 parent 32c9156 commit e5f480a
Show file tree
Hide file tree
Showing 7 changed files with 22 additions and 7 deletions.
14 changes: 12 additions & 2 deletions motherduck-dagster-hybrid-compute/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,15 @@ DUCKDB_DATABASE="md:dagster_hybrid_demo?motherduck_token=..."
# - https://motherduck.com/docs/integrations/evidence/
# - https://docs.evidence.dev/cli#environment-variables
#
EVIDENCE_SOURCE__DAGSTER_HYBRID_DEMO__TYPE="duckdb"
EVIDENCE_SOURCE__DAGSTER_HYBRID_DEMO__FILENAME="md:?motherduck_token=..."
EVIDENCE_SOURCE__dagster_hybrid_demo__type="duckdb"
EVIDENCE_SOURCE__dagster_hybrid_demo__filename="md:?motherduck_token=..."

# Running the Project Locally
#
# To run the project using a duck db file you've saved to disk, use the following example config.
# The duckdb file has to be specified with an absolute path, whilst the evidence source has to be
# specified as a relative path from the `./reports/sources/dagster_hybrid_demo/` directory.
#
# DUCKDB_DATABASE="/home/UserName/dev/dagster-devrel-fork/motherduck-dagster-hybrid-compute/data/db/local.duckdb"
# EVIDENCE_SOURCE__dagster_hybrid_demo__filename="../../../data/db/local.duckdb"

Empty file.
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ select
population_city,
survey_site_size

from dagster_hybrid_demo.main.all_birds
from main.all_birds

where
lower(bird_name) like '%duck%'
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# This file was automatically generated
name: dagster_hybrid_demo
type: duckdb
# The following `filename` env variable is read from the .env file in the project root.
options:
filename: md:?motherduck_token=<replace>
# For local use, use relative path from this directory, e.g.
# filename: ../../../local.duckdb
#
# For use with motherduck cloud, use connection string, e.g.
# filename: md:?motherduck_token=<replace>
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ select
bird_name,
total_count

from dagster_hybrid_demo.main.top_ducks_by_region
from main.top_ducks_by_region
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ select
state,
total_species_count

from dagster_hybrid_demo.main.top_ducks_by_state
from main.top_ducks_by_state
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ select
region,
species_count

from dagster_hybrid_demo.main.top_ducks_by_year
from main.top_ducks_by_year

0 comments on commit e5f480a

Please sign in to comment.