From b435e55624f4cfa3eeae05c67982aca14836d1ca Mon Sep 17 00:00:00 2001 From: Gilles Doge Date: Tue, 18 Apr 2023 13:21:38 +0200 Subject: [PATCH 1/3] [docs] Add documentation about only_tables and skip options --- website/docs/getting-started/configuration.md | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/website/docs/getting-started/configuration.md b/website/docs/getting-started/configuration.md index ca2f95b0..c1d69063 100644 --- a/website/docs/getting-started/configuration.md +++ b/website/docs/getting-started/configuration.md @@ -78,6 +78,54 @@ Any datastore compatible with the S3 protocol is a valid datastore. ::: +## Select tables from the source + +You have the possibility to select which tables you want in the dump. + +There are two options to achieve this: + 1. By specifying a list of tables you want (include strategy) + 2. By skipping a list of tables you don't want (exclude strategy) + +### Include strategy (`only_tables`) + +Add a key named `only_tables` under the `source`: + +```yaml +source: + connection_uri: postgres://root:password@localhost:5432/root + only_tables: # optional - dumps only specified tables. + - database: public + table: orders + - database: public + table: customers +``` + +### Exclude strategy (`skip_tables`) + +Add a key named `skip` under the `source`: + +```yaml +source: + connection_uri: postgres://root:password@localhost:5432/root + skip: # optional - exclude from the dump the specified tables. + - database: public + table: us_states + - database: public + table: order_details +``` + +This will exclude from the dump the tables `us_states` and `order_details`. + +:::warning + +This will exclude the table schema AND the table data. +::: + +:::warning + +Currently only PostgreSQL and MySQL sources support this feature. +::: + ## Example Here is a configuration file including some transformations and different options like the database subset. From 97e14fba5b9ee3e6f7695731080b17eb81acec17 Mon Sep 17 00:00:00 2001 From: Gilles Doge Date: Tue, 18 Apr 2023 13:35:09 +0200 Subject: [PATCH 2/3] fix a typo --- website/docs/getting-started/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/getting-started/configuration.md b/website/docs/getting-started/configuration.md index c1d69063..5964032d 100644 --- a/website/docs/getting-started/configuration.md +++ b/website/docs/getting-started/configuration.md @@ -100,7 +100,7 @@ source: table: customers ``` -### Exclude strategy (`skip_tables`) +### Exclude strategy (`skip`) Add a key named `skip` under the `source`: From 4d12bf3e829f5ae4bad766ec81f4cd730fbd024d Mon Sep 17 00:00:00 2001 From: Gilles Doge Date: Tue, 18 Apr 2023 13:35:33 +0200 Subject: [PATCH 3/3] Add an example of skip tables in the full example config --- website/docs/getting-started/configuration.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/website/docs/getting-started/configuration.md b/website/docs/getting-started/configuration.md index 5964032d..3ec73ae7 100644 --- a/website/docs/getting-started/configuration.md +++ b/website/docs/getting-started/configuration.md @@ -166,6 +166,12 @@ source: table: orders - database: public table: customers + # skip: # optional - exclude from the dump the specified tables. + # - database: public + # table: us_states + # - database: public + # table: order_details + datastore: aws: bucket: $BUCKET_NAME