How to train a image dataset like COCO, which includes json format annotations

[Young-eng]

Hi~ I meet a problem about the training, now I have a image dataset which have been annotated in COCO format, in json file, including train.json and val.json, the dataset is a image dataset, so how can I train it with Sleap ? Are videos necessary for this workflow with Sleap ? I don't know how to start training using my image dataset. Thanks

[talmo]

Hi @Young-eng,

You can import COCO datasets into SLEAP from the GUI by going to File --> Import.

Since you have splits, what you could do is then save the SLEAP label to .slp files for the training and validation splits separately.

Once you've done that, you can either merge them into a single SLEAP labels file (File --> Merge), or if you want to keep the split consistent, you can train via the sleap-train CLI with a syntax like:

sleap-train --val labels val.slp training_config.json train.slp

But I prefer to actually specify the different splits through the training_config.json itself, which has fields for specifying the paths to each split in the data --> labels section:

sleap/sleap/training_profiles/baseline_large_rf.bottomup.json

Lines 4 to 5 in 2d24296

	"training_labels": null,
	"validation_labels": null,

Let us know if you have any questions!

Cheers,

Talmo

[Young-eng]

I have uploaded the training configs including centroid and centered config. I think I know the answer about the split of train and val data when no validation path is assigned. There is a parameter named validation fraction. val data would be sampled from train data in a given proportion.

Another problem I met was crush when the input image size was large, e.g $1900 \times 3500$, even though I set the batch size to 1, adjust the input scale and filter_rate, etc. The usage of RAM is over 100G while the memory of GPU is surplus and not full utilized when training centered model. Are there any methods which can resize the input images to a small size for training?

Thanks.

[talmo]

Hi @Young-eng,

You can also disable data preloading if you have a large dataset:

sleap/sleap/training_profiles/baseline_large_rf.bottomup.json

Line 62 in 834c68a

"preload_data": true,

This normally will speed up the data loading pipeline during training quite a bit, but it'll blow up your RAM if your data/images are too large. Try turning that off and seeing if it helps with the memory usage.

Cheers,

Talmo

[Young-eng]

I did that, I tried to set "preload_data" to false. But It still crushed at the last stage during the training of centered model.

Thanks so much!

[talmo]

Hmmm, strange! Are you sure those are set in the centered instance model configuration in addition to the centroid model one?

A couple more settings to try:

        "online_shuffling": true,
        "shuffle_buffer_size": 4,
        "prefetch": false,

The shuffling preloads a lot of images into a buffer for order randomization, but it doesn't matter as much if you have a lot of images. Decreasing the buffer size helps, but you might be able to just turn it off entirely.

You can also turn off prefetching. I'm not sure about the impact on RAM, but it might help a bit too since it'll disable more preloading optimizations.

Cheers,

Talmo

[Young-eng]

Here are parameters in centered model:

It failed to compute metrics. Error like this, but I need the metric. I did not encounter this problem when training other datasets with smaller image size. It is strange :(

[Young-eng]

I solved this problem by resizing original datasets to smaller size, then import transformed datasets for training.