How do I compute the cold start of a scheduled job?

[rob-apella]

Hello,

We want to publish a metric to DataDog which is the cold start of a scheduled job. For us, this is defined as the difference between the scheduled timestamp (e.g., UTC midnight for a daily job), and the actual job start time (I think this would be the RUN_START event).

What is the best way to compute this in Python given a run ID?

Thanks!
Rob

[jamiedemaria]

Hi @rob-apella when a job is scheduled, we apply a tag with the scheduled execution time. Between that and the RUN_START event you should have all the info you need! From the run_id you can get a DagsterRun object that contains the tags

something like

@op 
def compute_time_diff(context):
    scheduled_time = context.instance.get_run_by_id(run_id).tags.get(".dagster/scheduled_execution_time")
    run_start_time = datetime.datetime.fromtimestamp(
        context.instance.get_run_record_by_id(run_id).start_time
    ).astimezone(datetime.timezone.utc)
    context.log.info(f"SCHEDULED TIME: {scheduled_time}")
    context.log.info(f"START TIME: {run_start_time}")

[rob-apella]

amazing! I will give this a whirl and circle back with any questions. thanks so much

[rob-apella]

@jamiedemaria this seems to work well, thank you. two more quick questions:

• is there any way to distinguish a run kicked off by a schedule, vs. a run kicked off via the "Test Schedule -> Launch Pad -> Launch Run" in the UI?
• is there a corresponding way to compute this for a sensor? I think that would be the difference between the sensor tick time, and the RUN_START event start time

[jamiedemaria]

hi! there currently is not a way to determine this. It's a good idea though, so I opened a github issue to track #21055