Add a temporal validation period to synthetic control and interrupted time series experiments

[drbenvincent]

• Closes Add validation period to synthetic control and interrupted time series #364
• Updates one of the synthetic control and one of the interrupted time series notebooks with quick demos of using the validation period
• This also corrects an error I found in the tests where test_its was testing synthetic control rather than interrupted time series

TODO

• Implement for synthetic control
• Implement for interrupted time series
• Calculate Bayesian $R^2$ value separately for training period and validation period
• Report Bayesian $R^2$ (including validation period where relevant) in the summary method instead of the plot title
• Streamline the implementation - do we really need separate classes, or can we just add a intervention_time kwarg and add in additional logic to the existing classes?
• Ensure class diagram is updated before requesting a review
• Check docstrings are ok and doctests pass
• Add test coverage
• Test for ValueError when validation_time >= treatment_time

---

📚 Documentation preview 📚: https://causalpy--367.org.readthedocs.build/en/367/

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[drbenvincent]

Good progress so far. But we are missing separate Bayesian $R^2$ for training and validation phases.

Synthetic control figure:

Interrupted time series figure:

[codecov]

Codecov Report

Attention: Patch coverage is 90.47619% with 4 lines in your changes missing coverage. Please review.

Project coverage is 85.81%. Comparing base (f6fd97c) to head (444e363).

Files	Patch %	Lines
causalpy/pymc_experiments.py	83.33%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #367      +/-   ##
==========================================
+ Coverage   85.60%   85.81%   +0.20%     
==========================================
  Files          22       22              
  Lines        1716     1748      +32     
==========================================
+ Hits         1469     1500      +31     
- Misses        247      248       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[drbenvincent]

Based on one of @cetagostini 's PR's (#368), I'm wondering if we should add a small feature to calculate a ROPE based on the validation period. Something a bit like this:

Any thoughts/comments welcome. I'm not convinced this is a good idea yet - especially because once we add in actual time series models then the credible interval will increase as we forecast further into the future.

[cetagostini]

Do you think we could replace on validation score by bayesian tail prob instead of R2? So, the interpretation here is about how much the real mean during the validation diverge from the posterior mean.

[drbenvincent]

Sure. Just looking into it so that I get it right - I just need the high level algorithm because I've not heard that much about it. Google search for "bayesian tail probability" shows very few hits. Is this a widely used approach? Doesn't matter if not, as long as it does what we want :)