Mass-selecting nodes with low prediction scores

[gracegliu]

In my single-animal videos, I will occasionally get a handful of frames where only some of my nodes are not visible (ie, mouse rearing, front paws not visible). In these instances, SLEAP will sometimes mislabel those non-visible nodes to parts of the field, ie, reflections in the acrylic chamber and so on, with very low confidence. However, the prediction scores for the visible nodes (back paws) are still high and those are correctly labeled. Is it possible to mass-select all nodes with low prediction scores and set them as invisible without deleting the instance?

[talmo]

Hi @gracegliu,

Sure thing -- we don't have this built-in, but I think we can write up a quick example of this.

Do you mind sharing your SLP file?

Cheers,

Talmo

[gracegliu]

Thanks so much! I have submitted my file.

[talmo]

Thanks @gracegliu! Here's a notebook that does what you requested. This uses sleap-io to load the labels file, which we then use to visualize the distribution of point scores, pick a cutoff, and set all points with scores below that value to not be visible, before saving the resulting labels back out.

You can run this yourself in Google Colab, or locally if you install sleap-io. You can then open the resulting file in SLEAP as you would normally.

For reference, here's the bit of code that minimally does this:

import sleap_io as sio

labels = sio.load_file("help.1799.slp")
cutoff = 0.4

for lf in labels:
    for inst in lf:
        for point in inst.points.values():
            if point.score < cutoff:
                point.visible = False

labels.save("filtered_labels.slp")

Let me know if you have any questions!

Talmo

[maia-g]

Hey,
I found this discussion and have been trying to adjust the script to work for my own data as I am hoping to do the exact same thing with the cutoff values. I am also fairly new to Python.

I successfully mounted my drive, set the working directory, and ran the labels = sio.load_file with my .slp file. However, I keep running into this error with the numpy commands when trying to build the plot.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-42-29e20f2d9d39> in <cell line: 3>()
      3 for lf in labels:
      4     for inst in lf:
----> 5         scores = inst.numpy(scores=True)[:, 2]
      6         node_names = inst.skeleton.node_names
      7         for node_name, score in zip(node_names, scores):

TypeError: Instance.numpy() got an unexpected keyword argument 'scores'

I then tried to just skip the plot creation and run the cutoff code, but got a similar error once again (see below). I have tried to look up the issue and troubleshoot on my own, but have been unsuccessful so far. Any tips would be appreciated!

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-40-6d6d30e03cfc> in <cell line: 3>()
      4     for inst in lf:
      5         for point in inst.points.values():
----> 6             if point.score < cutoff:
      7                 point.visible = False

AttributeError: 'Point' object has no attribute 'score'

Thanks in advance,
Maia

[maia-g]

Update, I see that SLEAP actually now has the option to remove instances below a score, but I would still like to create the plot to visualize the score spread anyway.

Thanks!

[talmo]

Hi @maia-g,

Right, I think the notebook was assuming that the instances are all predicted instances, and I'm guessing you might have some that are user-labeled instances (which don't have a score attribute).

I've updated the notebook to take that into account and you should be able to use it to generate the distribution now (it just ignores user labeled instances).

Let us know if you have any questions!

Cheers,

Talmo

[gracegliu]

Hi Talmo, Thanks so much! I will give that a shot. Best, Grace

…

________________________________ From: Talmo Pereira ***@***.***> Sent: Thursday, June 6, 2024 5:27 PM To: talmolab/sleap ***@***.***> Cc: Liu, Grace Geyu ***@***.***>; Mention ***@***.***> Subject: Re: [talmolab/sleap] Mass-selecting nodes with low prediction scores (Discussion #1799) Thanks @gracegliu<https://github.com/gracegliu>! Here's a notebook<https://colab.research.google.com/drive/1jKSK6aAG_qptPZIEOfL9Ojxv4SVoxpHa?usp=sharing> that does what you requested. This uses sleap-io<https://io.sleap.ai/> to load the labels file, which we then use to visualize the distribution of point scores, pick a cutoff, and set all points with scores below that value to not be visible, before saving the resulting labels back out. You can run this yourself in Google Colab, or locally if you install sleap-io. You can then open the resulting file in SLEAP as you would normally. Let me know if you have any questions! Talmo — Reply to this email directly, view it on GitHub<#1799 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BARQUZKY5637Y5CZI7FRURLZGDH33AVCNFSM6AAAAABI5FM6WKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMOJUGIYDO>. You are receiving this because you were mentioned.Message ID: ***@***.***>