Add feature set permutation test

[mdml]

Contributions

I adapted Vasilis's code for a feature permutation test, merging it into the permutation_test.py script and adding it to our pipeline.

By default, we now perform the following permutation tests:

1. Permute outcomes.
2. Permute "blood" (circulating) features.
3. Permute "tumor" features.
4. Permute "clinical" features.

Testing

I tested the updated implementation in two ways.

First, I verified that the right features were indeed being permuted, given a particular feature "class."

Second, I ran the entire pipeline with N=100 permutations, but only setting tolerance to 1E-5 and max iterations to 10000 (see the new configs/test.yml file). See the results below, which look reasonable to me. (Also note that these are just the TSV tables that our pipeline now outputs by default.)

Model summary

Excluded Feature Classes	No. features	Variance explained	MSE	RMSE	MAE
None	36	0.78918	0.42035	0.42035	0.17669
Tumor	22	0.227	0.80491	0.80491	0.64788
Blood	26	0.0846	0.87592	0.87592	0.76723
Clinical	24	-1.02539	1.3029	1.3029	1.69755

Permutation test summary

Test type	Feature class	Score	N	Mean Permuted Score	P-value
Feature	Clinical	0.17669	100	1.314	0.0099
Feature	Tumor	0.17669	100	0.72782	0.0099
Feature	Blood	0.17669	100	0.89115	0.0099
Outcome		0.17669	100	1.26388	0.0099

Review

@lmackey: note that I am assigning you to be the sole reviewer since Vasilis has not yet joined this GitHub repo.