Unexpected behavior when using the n_periods and drop_high/drop_low parameters at the same time

[cdietrich215]

First off, thanks a ton for all of the hard work on this package. Its been a huge help for me and my company.

When using the cl.Development() method with n_periods, I would expect the drop_high/drop_low parameters to drop the highest/lowest parameters after the earlier periods have been dropped from each development period. Instead it appears that the highest and lowest periods are dropped first, and then the earlier diagonals get dropped. This can frequently cause cases where only the low (or high) ldf gets dropped, skewing results.

Is there a way you'd recommend accomplishing this in the package's current state, or is this functionality yet to be built in?

For an example try running: cl.Development(drop_high = 1, drop_low=1, preserve = 1, n_periods = 5).fit_transform(cl.load_sample('raa')).age_to_age.heatmap()

[jbogaardt]

Hi @cdietrich215 . Very nice to hear that the library has been helpful for you and your company.

I agree with you that it is more intuitive for high/low to drop after censoring for n_periods, and you're right, it doesn't work that way currently. This hasn't even been on my radar, so thanks for bringing it up. It is, unfortunately, functionality that needs to be built. I'm converting this dicussion to an issue so that it be used for code development.

[kennethshsu]

Hi @cdietrich215, thanks for the feedback!

I just implemented the change. Thanks for the recommendation!

I also updated the docstring, hopefully this is clear:

class Development(DevelopmentBase):
    ...
    Parameters
    ...
    drop_high: bool, int, list of bools, or list of ints (default = None)
        Drops highest (by rank) link ratio(s) from LDF calculation
        If a boolean variable is passed, drop_high is set to 1, dropping only the 
        highest value
        Note that drop_high is performed after consideration of n_periods (if used)

Do you think there will ever be a case where we want to perform drop_high and drop_low before considering n_periods? I didn't want to introduce another hyper-parameter.

[cdietrich215]

Hi @kennethshsu. That's great, thank you for all the work on this.

I personally can't think of a case where someone would want to drop_high or drop_low before considering n_periods, and this would lead to the issue of skewing results as described above. So I think a hyper-parameter here would be unnecessary. I think most users will expect this to drop the highest/lowest age-to-age factors within the n_periods they are examining.

It would probably be a fringe case if someone wanted to do that, and they could probably get what they wanted by just using the drop or drop_above parameters anyways.

[kennethshsu]

Ok sounds good. We'll keep it strictly that drop_high and drop_low are performed after n_periods for now!

Please reach out here if you run into any other problems.