Modifying labels.json

[theeanas]

How did you modify the labels file to only include the training instances that you used? Because you stated that you only used train0, ..., train3.

Thanks in advance!

[hpanwar08]

You can write a simple python script to create a new label file based on the original label file and training data in 0,1,2,3

[theeanas]

That's what I tried doing, here's the snippet:

train0 = [f for f in listdir("./publaynet/train/") if isfile(join("./publaynet/train/", f)) and f.endswith(".jpg")]
labels_file = open("publaynet/train.json", "r")
all_labels = json.load(labels_file)

all_labels['images'] = [img for img in all_labels['images'] if img['file_name'] in train0] # This line works fine
ids = [img['id'] for img in all_labels['images']]
all_labels['annotations'] = [ann for ann in all_labels['annotations'] if ann['image_id'] in ids]  # doesn't work

This fixes all_labels['images'], but for some reason, it doesn't work on all_labels['annotations']. That's very strange for me.
Note: train0 has the images file names that I want to train on.

[hpanwar08]

Code seems fine to me, what's the output of your last line?

[theeanas]

Same thing, it's like the assignment never happens. Also, the last line takes a very long time to finish execution, around 2 hrs on colab servers.

[hpanwar08]

You can speed up by making ids as set instead of list.