"mean of empty slice" in spearman correlation calculation

[timokau]

During the tests, numpy complains about a "mean of empty slice". That happens because the calculation of the spearman correlation filters the labels it applies to as follows:

cs-ranking/csrank/metrics_np.py

Line 24 in ba03234

if len(np.unique(r2)) == len(r2):

And then averages its results:

cs-ranking/csrank/metrics_np.py

Line 29 in ba03234

return np.nanmean(np.array(rho))

Which may be empty (or consist of only NaNs) due to the previous filter. What is the intention behind that filter?

CC @prithagupta

[kiudee]

The filter is applied to remove instances for which there are ties in the prediction. Ties are problematic in the calculation of Spearman correlation and can cause a non-minor bias. But I also think that the current state of the code could be improved - at the very least the user should get a warning.

Here is a paper discussing several methods on how to deal with ties: https://www.tandfonline.com/doi/full/10.1080/02664763.2015.1043870

[prithagupta]

@kiudee @timokau even the script version takes ties into consideration. But we need to check that implementation on how they do it. As far as I remember we removed it because it was not correct or efficient ways of evaluating spearman correlation.