Unexpected Behavior From grain() Method

[SMSinclair]

Hi there, big fan of this package.

I have a question about how the grain() method from the Triangle class works. I created a cumulative triangle with monthly origin and monthly grain and set the .is_cumulative attribute to True. Then I used .grain('OQDQ') to roll it up, but it didn't sum the diagonal that I thought it should. I've included a picture for illustration, and I'm wondering if it might have to do with the fact that it's labeling the first development period as 1 instead of 0?

[jbogaardt]

Hi @SMSinclair , Thanks for the question, our first through github discussions!

This is unusual behavior. The grain method is looking at the latest valuation, in this case 2019-05 (associated with origin 2018-12 at development month 6) and is using that as the baseline for its quarter groupings. Given it is May, it is selecting months ending in 2, 5, 8, 11. The fact that it is a completed (is_full) triangle is tripping it up possibly as well as it being a partial years worth of observations.

Limiting it to the more typical upper diagonals seems to produce the desired result. Here's a reprex of what I mean:

import pandas as pd
import chainladder as cl

data = pd.DataFrame(
    {'origin': ['2018-07','2018-08','2018-09','2018-10','2018-11','2018-12','2018-07','2018-08','2018-09','2018-10','2018-11','2018-12','2018-07','2018-08','2018-09','2018-10','2018-11','2018-12','2018-07','2018-08','2018-09','2018-10','2018-11','2018-12','2018-07','2018-08','2018-09','2018-10','2018-11','2018-12','2018-07','2018-08','2018-09','2018-10','2018-11','2018-12'],
     'valuation': ['2018-07','2018-08','2018-09','2018-10','2018-11','2018-12','2018-08','2018-09','2018-10','2018-11','2018-12','2019-01','2018-09','2018-10','2018-11','2018-12','2019-01','2019-02','2018-10','2018-11','2018-12','2019-01','2019-02','2019-03','2018-11','2018-12','2019-01','2019-02','2019-03','2019-04','2018-12','2019-01','2019-02','2019-03','2019-04','2019-05'],
     'value': [1,4,0,1,2,0,2,6,1,3,3,1,3,7,3,5,4,3,5,8,5,7,5,5,7,9,7,9,6,7,8,10,8,10,7,8]})

tri = cl.Triangle(
    data=data, origin='origin', development='valuation', 
    columns='value', cumulative=True).dropna()

tri[tri.valuation<'2019'].grain('OQDQ')

The problem with the full triangle is that we are censoring all origins to age 6 (valuation 2019-05), but we don't have observations for every origin at that valuation and it chokes. Regardless of how it works currently , grain + full triangle combination probably needs deeper review in the package.

[CarinaWY]

Also a big fan of this package.
Since we are using the latest valuation date (here, 2019-05) as the baseline for its quarter groupings which may cause incorrect quarterly calculation, is there any way that we can use the starting origin date (here, 2018-07) as the baseline to do quarter groupings?

[jbogaardt]

Thanks for the suggestion @CarinaWY. That might work but it might be tripped up by asymmetric triangles such as 'OYDM'. Definitely need to think on it more.

[SMSinclair]

@jbogaardt ,

Thanks!

My original post was misleading, that was actually the top left corner of an incomplete cumulative triangle with 36 accident months in the origin ranging from 2018-05 to 2021-04.

Anyway, I mocked up an incomplete cumulative triangle with 12 accident months and dummy values to test on:

import pandas as pd
import chainladder as cl

data = pd.DataFrame(
    {'origin': ['2018-01','2018-02','2018-03','2018-04','2018-05','2018-06',
                '2018-07','2018-08','2018-09','2018-10','2018-11','2018-12',
                '2018-01','2018-02','2018-03','2018-04','2018-05','2018-06',
                '2018-07','2018-08','2018-09','2018-10','2018-11','2018-01',
                '2018-02','2018-03','2018-04','2018-05','2018-06','2018-07',
                '2018-08','2018-09','2018-10','2018-01','2018-02','2018-03',
                '2018-04','2018-05','2018-06','2018-07','2018-08','2018-09',
                '2018-01','2018-02','2018-03','2018-04','2018-05','2018-06',
                '2018-07','2018-08','2018-01','2018-02','2018-03','2018-04',
                '2018-05','2018-06','2018-07','2018-01','2018-02','2018-03',
                '2018-04','2018-05','2018-06','2018-01','2018-02','2018-03',
                '2018-04','2018-05','2018-01','2018-02','2018-03','2018-04',
                '2018-01','2018-02','2018-03','2018-01','2018-02','2018-01'],
     'valuation': ['2018-01','2018-02','2018-03','2018-04','2018-05','2018-06',
                   '2018-07','2018-08','2018-09','2018-10','2018-11','2018-12',
                   '2018-02','2018-03','2018-04','2018-05','2018-06','2018-07',
                   '2018-08','2018-09','2018-10','2018-11','2018-12','2018-03',
                   '2018-04','2018-05','2018-06','2018-07','2018-08','2018-09',
                   '2018-10','2018-11','2018-12','2018-04','2018-05','2018-06',
                   '2018-07','2018-08','2018-09','2018-10','2018-11','2018-12',
                   '2018-05','2018-06','2018-07','2018-08','2018-09','2018-10',
                   '2018-11','2018-12','2018-06','2018-07','2018-08','2018-09',
                   '2018-10','2018-11','2018-12','2018-07','2018-08','2018-09',
                   '2018-10','2018-11','2018-12','2018-08','2018-09','2018-10',
                   '2018-11','2018-12','2018-09','2018-10','2018-11','2018-12',
                   '2018-10','2018-11','2018-12','2018-11','2018-12','2018-12'], 
     'value': [1,2,1,2,2,1,1,2,1,2,2,1,2,3,3,3,3,2,3,3,3,3,2,3,4,5,5,4,4,5,5,5,
               4,5,7,6,7,5,6,7,8,6,6,8,7,9,8,9,8,9,7,9,8,10,9,10,10,8,10,9,11,
               11,11,9,11,10,11,12,10,12,11,13,11,13,12,12,14,13]})

tri = cl.Triangle(
    data=data, origin='origin', development='valuation', 
    columns='value', cumulative=True).dropna()
tri.grain('OQDQ')

Then it occurred to me that I was not using the latest version of chainladder, I was still on 0.7.9. So I ran the code in 0.7.9 and 0.8.4 and found that the newest version was working as expected:

So it looks like I'm good to go!