Delete dynamic partition with API call from daemon/code deployment k8s pod

[ssillaots-boku]

I've created a job where I can delete dynamic partitions.

But that got me wondering if there is a way to do that straight from daemon/code deployment (inside the k8s pod itself) with an API call?

[garethbrickman]

You can delete dynamic partitions programmatically using the DagsterInstance.delete_dynamic_partition method. This method allows you to delete a partition for the specified DynamicPartitionsDefinition. If the partition does not exist, it exits silently.

Here's how you can access the DagsterInstance and use the delete_dynamic_partition method:

from dagster import DagsterInstance

# Get the current Dagster instance
instance = DagsterInstance.get()

# Delete a dynamic partition
instance.delete_dynamic_partition("my_partitions_def_name", "partition_key_to_delete")

This code can be executed within the Kubernetes pod that is running your Dagster daemon or code deployment. You would need to ensure that the code has access to the Dagster instance, which is typically configured through environment variables or a dagster.yaml file within the pod.

Keep in mind that this action should be taken with caution, as deleting partitions can affect the downstream dependencies and consumers of the data associated with those partitions.

[ssillaots-boku]

Thanks for the response!
I was rather thinking if it's doable from command line but seems like it isn't (e.g curl -X DELETE ...).

[garethbrickman]

Yeah I also don't think it's doable using the GraphQL API at this time, but you can load the GraphQL playground to explore the schema and docs available in there!