Using ridgeplot for sktime and skpro distributional predictions?

[fkiraly]

For a while I have now been thinking about what a good plotting modality would be for fully distributional predictions, i.e., the output of predict_proba in sktime or skpro.

The challnge is that you have a (marginal) distribution for each entry in a pandas-like table, which seems hard to visualize. I've experimented with panels (matplotlib.subplots) but I wasn't quit happy with the result.

Now, by accident (just curious clicking), I've discovered ridgeplot.

What would you think of using the look & feel of ridgeplot as a plotting function in BaseDistribution? Where rows are the rows of the data-frame like stucture, and mayb there are also columns (but I am happy with the single-variable case too)

The main difference is that the distribution does not need to be estimated via KDE, you already have it in a form where you can access pdf, cdf, etc, completely, and you have the quantile function too which helps with selecting x-axis range.

Plotting cdf and other distribution defining functions would also be neat, of course pdf (if exists), or cdf (for survival) are already great.

Imagined usage, sth like

fcst = BuildSth(Complex(params), more_params)
fcst.fit(y, fh=range(1, 10)
y_dist = fcst.predict_proba()

y_dist.plot() # default is pdf for continuous distirbutions
y_dist.plot("cdf")

Dependencies-wise, one could imagine ridgeplot as a plotting softdep like matplotlib or seaborn, of skpro and therefore indirectly of sktime.

What do you think?

[tpvasconcelos]

Hey @fkiraly happy to see you here! Give me some time to take a look at this one as I'm not super familiar with that class of forecasters from sktime.

The main difference is that the distribution does not need to be estimated via KDE, you already have it in a form where you can access pdf, cdf, etc, completely, and you have the quantile function too which helps with selecting x-axis range.

If I understood correctly, this is completely fine when it comes to interacting with ridgeplot, since we also accept x-y traces as input -- bypassing the KDE step altogether

[fkiraly]

as I'm not super familiar with that class of forecasters from sktime.

Most popular forecasters have a probabilistic prediction mode, and they can be filtered by the tag capability:pred_int.

Check it out in the forecasting tutorial, the main tutorial of skpro also explains the tabular (non-time-series) interfaces arorund it.

If I understood correctly, this is completely fine when it comes to interacting with ridgeplot, since we also accept x-y traces as input -- bypassing the KDE step altogether

Yes - in the conceptual space of sktime / skpro, the KDE step is an estimator, of type "distribution from sample". In the desired plot, we would go directly from distribution to plot, without th first step of going to distribution from sample.

[fkiraly]

any opinions yet?

[fkiraly]

?

[tpvasconcelos]

Hey @fkiraly. I've been trying to make some time to look into this issue but unfortunately haven't been able to make the time yet. Is this something that is high on your priority list or something that is currently blocking your work? If so, I can try prioritising it. If not, let me know as well and I will take a look at it with more time later on. Thanks!

[fkiraly]

no urgency, just sth that imo would be very nice - whenever you find time.

If in the meanwhile I can help with sth ridgeplot, let me know.
How about this? #116

[tpvasconcelos]

@fkiraly Just trying to get some early feedback from you. I just tried to map this output from sktime's 01b_forecasting_proba.ipynb notebook:

into a ridgeline plot of PDFs (for the same forecast horizon):

👆 this is what I was able to hack together while trying to figure out how the BaseDistribution interface works.

Let me know if this is anything like what you had envisioned, otherwise, could you share a more concrete example I can play around with. A sketch would help.

[fkiraly]

It should become as natural as the left-to-right orientation of time-series plots.

Perhaps, I would put against that that I feel strongly about top-bottom in non-temporal context, as it is the "table" direction; and I would agree with bottom-up being better in a time series context.

The pain point this causes is that the table-like distributions in skpro are the same object used in both contexts, and the sktime pd.DataFrame-s progress from top to bottom in temporal order.

we also should consider the case where the row dimension is not time, simply a sample index, like in sktime. E.g., a user might like to compare ridgeplots for group analysis or similar.

I need to see an example of this.

All examples in the skpro intro notebook are non-temporal, see there. No plots though, but that should suffice to get a picture, hopefully?

This is just the default behaviour from the ridgeplot library. The convention here could be anything we won't. It it's not currently supported by the ridgeplot library, I could try adding support.

Would the colour shift be possible, from example number 5 below? That is, the colour at every point on the x-axis is the same above it.

[tpvasconcelos]

Would the colour shift be possible, from example number 5 below? That is, the colour at every point on the x-axis is the same above it.

I would love to be able to do that as well but unfortunately Plotly does not have an option for filling curves with color gradients. See plotly/plotly.py#3026 and plotly/plotly.js#1918

[tpvasconcelos]

All examples in the skpro intro notebook are non-temporal, see there. No plots though, but that should suffice to get a picture, hopefully?

Sounds good but maybe we can leave this as an exploration option for later on? Keeping my free time in mind, I think it would be best to start with a narrower scope.

I'll try refining my current implementation and example and:

• Add trace lines for a given list of quantiles (e.g. 5/25/50/75/95).
• Add support for panel data (more than one target for y)
• Showcase some examples for the different ways we could colour the different traces.

[fkiraly]

Something like this?

yes, that looks brilliant!

[fkiraly]

All examples in the skpro intro notebook are non-temporal, see there. No plots though, but that should suffice to get a picture, hopefully?

Sounds good but maybe we can leave this as an exploration option for later on? Keeping my free time in mind, I think it would be best to start with a narrower scope.

Sure - just noting, that the skpro.distributions module contains a carbon copy of the sktime one, it is a refactor target. So we need to be mindful that anything that works for temporally ordered collections for distributions also makes sense for unordered or grouped collections.

[tpvasconcelos]

A few examples and references that use this method to visualise probabilistic forecasts

This includes both univariable and panel series examples.

Granted that this is not a very significant sample but just for reference:

• 2/5 use the bottom-up convention 👆
• 3/5 use the top-to-bottom convention 👇

---

👉 https://seita.nl/2019/10/22/how-to-plot-probabilistic-forecasts/

---

👉 https://link.springer.com/article/10.1007/s11222-023-10212-8

---

👉 https://www.reddit.com/r/dataisbeautiful/comments/o765tz/oc_vaccination_rollout_by_age_belgium_ridgeline/

People seemed to have liked this way of visualising probabilistic forecasts. Here the non-OP top comment:

---

---

👉 https://wilkelab.org/ggridges/articles/introduction.html#varying-fill-colors-along-the-x-axis

---

[tpvasconcelos]

the gradient in the last plot looks more natural than homogenous colour left/right.

I would love to be able to do that as well but unfortunately Plotly does not have an option for filling curves with color gradients. See plotly/plotly.py#3026 and plotly/plotly.js#1918

[tpvasconcelos]

there are good reasons for both top/bottom and bottom-up. I think this should be an option.

Agreed!

[fkiraly]

I would love to be able to do that as well but unfortunately Plotly does not have an option for filling curves with color gradients.

I'm not that well-experienced in plotting, so I can't say what is easy or not. I agree with your general rationale - which I seem to be reading between your lines - that we should prioritize low-hanging fruit, of which there are plenty.

(example 5 is R, that's why they can do it?)

[fkiraly]

(random idea - not sure if it works in plotly: (a) filling with single colour or "transparent" placeholder; (b) layering with a large rectangle that has a gradient, where the colour in (a) acts as a "hole" in the plot)

[tpvasconcelos]

(example 5 is R, that's why they can do it?)

Yes example 5 uses ggridges

[fkiraly]

Some design questions:

• how should this be accessed? BaseDistribution.plot, with params?
• ridgeplot would have to become a managed soft dependency, e.g., in a plot softdep set
• the statsmodels dependency is a bit steep and not necessary for plots where you have access to pdf, cdf, etc. How do we manage that? Can we isolate it in ridgeplot proper? How about making the KDE backend a choice, and moving statsmodels to softdep in ridgeplot proper?

[fkiraly]

what's next, @tpvasconcelos?

[fkiraly]

@tpvasconcelos, here's a PR with a plotting interface, for probabiliy distributions in skpro.
sktime/skpro#275

I was thinking about - and have designed it in that way - to allow different backends for plotting.

So, if used in a time series context, the distributions could automatically be switched to temporal plotting mode, why the panel mode makes more sense for tabular datasets where distributions are in no particular order.