Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EOFError when training #43

Open
jhaggle opened this issue Feb 14, 2020 · 0 comments
Open

EOFError when training #43

jhaggle opened this issue Feb 14, 2020 · 0 comments

Comments

@jhaggle
Copy link

jhaggle commented Feb 14, 2020

Starting the training I get the EOFError shown below.
What am I doing wrong?

(keras2) johan@johan-VirtualBox:~/Documents/school_projects/temp/EAST-master$ python3 train.py --gpu_list=1 --input_size=512 --batch_size=12 --nb_workers=6 --training_data_path=/home/johan/Documents/school_projects/temp/train_data40 --validation_data_path=/home/johan/Documents/school_projects/temp/train_data40 --checkpoint_path=/home/johan/Documents/school_projects/temp/EAST-master/temp --max_epochs=3
/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:471: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:472: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:473: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:474: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:475: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
Using TensorFlow backend.
Number of validation images : 40
Training with 1 GPU
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.


Layer (type) Output Shape Param # Connected to

input_image (InputLayer) (None, None, None, 3 0


conv1_pad (ZeroPadding2D) (None, None, None, 3 0 input_image[0][0]


conv1 (Conv2D) (None, None, None, 6 9472 conv1_pad[0][0]


bn_conv1 (BatchNormalization) (None, None, None, 6 256 conv1[0][0]


activation_1 (Activation) (None, None, None, 6 0 bn_conv1[0][0]


max_pooling2d_1 (MaxPooling2D) (None, None, None, 6 0 activation_1[0][0]


res2a_branch2a (Conv2D) (None, None, None, 6 4160 max_pooling2d_1[0][0]


bn2a_branch2a (BatchNormalizati (None, None, None, 6 256 res2a_branch2a[0][0]


activation_2 (Activation) (None, None, None, 6 0 bn2a_branch2a[0][0]


res2a_branch2b (Conv2D) (None, None, None, 6 36928 activation_2[0][0]


bn2a_branch2b (BatchNormalizati (None, None, None, 6 256 res2a_branch2b[0][0]


activation_3 (Activation) (None, None, None, 6 0 bn2a_branch2b[0][0]


res2a_branch2c (Conv2D) (None, None, None, 2 16640 activation_3[0][0]


res2a_branch1 (Conv2D) (None, None, None, 2 16640 max_pooling2d_1[0][0]


bn2a_branch2c (BatchNormalizati (None, None, None, 2 1024 res2a_branch2c[0][0]


bn2a_branch1 (BatchNormalizatio (None, None, None, 2 1024 res2a_branch1[0][0]


add_1 (Add) (None, None, None, 2 0 bn2a_branch2c[0][0]
bn2a_branch1[0][0]


activation_4 (Activation) (None, None, None, 2 0 add_1[0][0]


res2b_branch2a (Conv2D) (None, None, None, 6 16448 activation_4[0][0]


bn2b_branch2a (BatchNormalizati (None, None, None, 6 256 res2b_branch2a[0][0]


activation_5 (Activation) (None, None, None, 6 0 bn2b_branch2a[0][0]


res2b_branch2b (Conv2D) (None, None, None, 6 36928 activation_5[0][0]


bn2b_branch2b (BatchNormalizati (None, None, None, 6 256 res2b_branch2b[0][0]


activation_6 (Activation) (None, None, None, 6 0 bn2b_branch2b[0][0]


res2b_branch2c (Conv2D) (None, None, None, 2 16640 activation_6[0][0]


bn2b_branch2c (BatchNormalizati (None, None, None, 2 1024 res2b_branch2c[0][0]


add_2 (Add) (None, None, None, 2 0 bn2b_branch2c[0][0]
activation_4[0][0]


activation_7 (Activation) (None, None, None, 2 0 add_2[0][0]


res2c_branch2a (Conv2D) (None, None, None, 6 16448 activation_7[0][0]


bn2c_branch2a (BatchNormalizati (None, None, None, 6 256 res2c_branch2a[0][0]


activation_8 (Activation) (None, None, None, 6 0 bn2c_branch2a[0][0]


res2c_branch2b (Conv2D) (None, None, None, 6 36928 activation_8[0][0]


bn2c_branch2b (BatchNormalizati (None, None, None, 6 256 res2c_branch2b[0][0]


activation_9 (Activation) (None, None, None, 6 0 bn2c_branch2b[0][0]


res2c_branch2c (Conv2D) (None, None, None, 2 16640 activation_9[0][0]


bn2c_branch2c (BatchNormalizati (None, None, None, 2 1024 res2c_branch2c[0][0]


add_3 (Add) (None, None, None, 2 0 bn2c_branch2c[0][0]
activation_7[0][0]


activation_10 (Activation) (None, None, None, 2 0 add_3[0][0]


res3a_branch2a (Conv2D) (None, None, None, 1 32896 activation_10[0][0]


bn3a_branch2a (BatchNormalizati (None, None, None, 1 512 res3a_branch2a[0][0]


activation_11 (Activation) (None, None, None, 1 0 bn3a_branch2a[0][0]


res3a_branch2b (Conv2D) (None, None, None, 1 147584 activation_11[0][0]


bn3a_branch2b (BatchNormalizati (None, None, None, 1 512 res3a_branch2b[0][0]


activation_12 (Activation) (None, None, None, 1 0 bn3a_branch2b[0][0]


res3a_branch2c (Conv2D) (None, None, None, 5 66048 activation_12[0][0]


res3a_branch1 (Conv2D) (None, None, None, 5 131584 activation_10[0][0]


bn3a_branch2c (BatchNormalizati (None, None, None, 5 2048 res3a_branch2c[0][0]


bn3a_branch1 (BatchNormalizatio (None, None, None, 5 2048 res3a_branch1[0][0]


add_4 (Add) (None, None, None, 5 0 bn3a_branch2c[0][0]
bn3a_branch1[0][0]


activation_13 (Activation) (None, None, None, 5 0 add_4[0][0]


res3b_branch2a (Conv2D) (None, None, None, 1 65664 activation_13[0][0]


bn3b_branch2a (BatchNormalizati (None, None, None, 1 512 res3b_branch2a[0][0]


activation_14 (Activation) (None, None, None, 1 0 bn3b_branch2a[0][0]


res3b_branch2b (Conv2D) (None, None, None, 1 147584 activation_14[0][0]


bn3b_branch2b (BatchNormalizati (None, None, None, 1 512 res3b_branch2b[0][0]


activation_15 (Activation) (None, None, None, 1 0 bn3b_branch2b[0][0]


res3b_branch2c (Conv2D) (None, None, None, 5 66048 activation_15[0][0]


bn3b_branch2c (BatchNormalizati (None, None, None, 5 2048 res3b_branch2c[0][0]


add_5 (Add) (None, None, None, 5 0 bn3b_branch2c[0][0]
activation_13[0][0]


activation_16 (Activation) (None, None, None, 5 0 add_5[0][0]


res3c_branch2a (Conv2D) (None, None, None, 1 65664 activation_16[0][0]


bn3c_branch2a (BatchNormalizati (None, None, None, 1 512 res3c_branch2a[0][0]


activation_17 (Activation) (None, None, None, 1 0 bn3c_branch2a[0][0]


res3c_branch2b (Conv2D) (None, None, None, 1 147584 activation_17[0][0]


bn3c_branch2b (BatchNormalizati (None, None, None, 1 512 res3c_branch2b[0][0]


activation_18 (Activation) (None, None, None, 1 0 bn3c_branch2b[0][0]


res3c_branch2c (Conv2D) (None, None, None, 5 66048 activation_18[0][0]


bn3c_branch2c (BatchNormalizati (None, None, None, 5 2048 res3c_branch2c[0][0]


add_6 (Add) (None, None, None, 5 0 bn3c_branch2c[0][0]
activation_16[0][0]


activation_19 (Activation) (None, None, None, 5 0 add_6[0][0]


res3d_branch2a (Conv2D) (None, None, None, 1 65664 activation_19[0][0]


bn3d_branch2a (BatchNormalizati (None, None, None, 1 512 res3d_branch2a[0][0]


activation_20 (Activation) (None, None, None, 1 0 bn3d_branch2a[0][0]


res3d_branch2b (Conv2D) (None, None, None, 1 147584 activation_20[0][0]


bn3d_branch2b (BatchNormalizati (None, None, None, 1 512 res3d_branch2b[0][0]


activation_21 (Activation) (None, None, None, 1 0 bn3d_branch2b[0][0]


res3d_branch2c (Conv2D) (None, None, None, 5 66048 activation_21[0][0]


bn3d_branch2c (BatchNormalizati (None, None, None, 5 2048 res3d_branch2c[0][0]


add_7 (Add) (None, None, None, 5 0 bn3d_branch2c[0][0]
activation_19[0][0]


activation_22 (Activation) (None, None, None, 5 0 add_7[0][0]


res4a_branch2a (Conv2D) (None, None, None, 2 131328 activation_22[0][0]


bn4a_branch2a (BatchNormalizati (None, None, None, 2 1024 res4a_branch2a[0][0]


activation_23 (Activation) (None, None, None, 2 0 bn4a_branch2a[0][0]


res4a_branch2b (Conv2D) (None, None, None, 2 590080 activation_23[0][0]


bn4a_branch2b (BatchNormalizati (None, None, None, 2 1024 res4a_branch2b[0][0]


activation_24 (Activation) (None, None, None, 2 0 bn4a_branch2b[0][0]


res4a_branch2c (Conv2D) (None, None, None, 1 263168 activation_24[0][0]


res4a_branch1 (Conv2D) (None, None, None, 1 525312 activation_22[0][0]


bn4a_branch2c (BatchNormalizati (None, None, None, 1 4096 res4a_branch2c[0][0]


bn4a_branch1 (BatchNormalizatio (None, None, None, 1 4096 res4a_branch1[0][0]


add_8 (Add) (None, None, None, 1 0 bn4a_branch2c[0][0]
bn4a_branch1[0][0]


activation_25 (Activation) (None, None, None, 1 0 add_8[0][0]


res4b_branch2a (Conv2D) (None, None, None, 2 262400 activation_25[0][0]


bn4b_branch2a (BatchNormalizati (None, None, None, 2 1024 res4b_branch2a[0][0]


activation_26 (Activation) (None, None, None, 2 0 bn4b_branch2a[0][0]


res4b_branch2b (Conv2D) (None, None, None, 2 590080 activation_26[0][0]


bn4b_branch2b (BatchNormalizati (None, None, None, 2 1024 res4b_branch2b[0][0]


activation_27 (Activation) (None, None, None, 2 0 bn4b_branch2b[0][0]


res4b_branch2c (Conv2D) (None, None, None, 1 263168 activation_27[0][0]


bn4b_branch2c (BatchNormalizati (None, None, None, 1 4096 res4b_branch2c[0][0]


add_9 (Add) (None, None, None, 1 0 bn4b_branch2c[0][0]
activation_25[0][0]


activation_28 (Activation) (None, None, None, 1 0 add_9[0][0]


res4c_branch2a (Conv2D) (None, None, None, 2 262400 activation_28[0][0]


bn4c_branch2a (BatchNormalizati (None, None, None, 2 1024 res4c_branch2a[0][0]


activation_29 (Activation) (None, None, None, 2 0 bn4c_branch2a[0][0]


res4c_branch2b (Conv2D) (None, None, None, 2 590080 activation_29[0][0]


bn4c_branch2b (BatchNormalizati (None, None, None, 2 1024 res4c_branch2b[0][0]


activation_30 (Activation) (None, None, None, 2 0 bn4c_branch2b[0][0]


res4c_branch2c (Conv2D) (None, None, None, 1 263168 activation_30[0][0]


bn4c_branch2c (BatchNormalizati (None, None, None, 1 4096 res4c_branch2c[0][0]


add_10 (Add) (None, None, None, 1 0 bn4c_branch2c[0][0]
activation_28[0][0]


activation_31 (Activation) (None, None, None, 1 0 add_10[0][0]


res4d_branch2a (Conv2D) (None, None, None, 2 262400 activation_31[0][0]


bn4d_branch2a (BatchNormalizati (None, None, None, 2 1024 res4d_branch2a[0][0]


activation_32 (Activation) (None, None, None, 2 0 bn4d_branch2a[0][0]


res4d_branch2b (Conv2D) (None, None, None, 2 590080 activation_32[0][0]


bn4d_branch2b (BatchNormalizati (None, None, None, 2 1024 res4d_branch2b[0][0]


activation_33 (Activation) (None, None, None, 2 0 bn4d_branch2b[0][0]


res4d_branch2c (Conv2D) (None, None, None, 1 263168 activation_33[0][0]


bn4d_branch2c (BatchNormalizati (None, None, None, 1 4096 res4d_branch2c[0][0]


add_11 (Add) (None, None, None, 1 0 bn4d_branch2c[0][0]
activation_31[0][0]


activation_34 (Activation) (None, None, None, 1 0 add_11[0][0]


res4e_branch2a (Conv2D) (None, None, None, 2 262400 activation_34[0][0]


bn4e_branch2a (BatchNormalizati (None, None, None, 2 1024 res4e_branch2a[0][0]


activation_35 (Activation) (None, None, None, 2 0 bn4e_branch2a[0][0]


res4e_branch2b (Conv2D) (None, None, None, 2 590080 activation_35[0][0]


bn4e_branch2b (BatchNormalizati (None, None, None, 2 1024 res4e_branch2b[0][0]


activation_36 (Activation) (None, None, None, 2 0 bn4e_branch2b[0][0]


res4e_branch2c (Conv2D) (None, None, None, 1 263168 activation_36[0][0]


bn4e_branch2c (BatchNormalizati (None, None, None, 1 4096 res4e_branch2c[0][0]


add_12 (Add) (None, None, None, 1 0 bn4e_branch2c[0][0]
activation_34[0][0]


activation_37 (Activation) (None, None, None, 1 0 add_12[0][0]


res4f_branch2a (Conv2D) (None, None, None, 2 262400 activation_37[0][0]


bn4f_branch2a (BatchNormalizati (None, None, None, 2 1024 res4f_branch2a[0][0]


activation_38 (Activation) (None, None, None, 2 0 bn4f_branch2a[0][0]


res4f_branch2b (Conv2D) (None, None, None, 2 590080 activation_38[0][0]


bn4f_branch2b (BatchNormalizati (None, None, None, 2 1024 res4f_branch2b[0][0]


activation_39 (Activation) (None, None, None, 2 0 bn4f_branch2b[0][0]


res4f_branch2c (Conv2D) (None, None, None, 1 263168 activation_39[0][0]


bn4f_branch2c (BatchNormalizati (None, None, None, 1 4096 res4f_branch2c[0][0]


add_13 (Add) (None, None, None, 1 0 bn4f_branch2c[0][0]
activation_37[0][0]


activation_40 (Activation) (None, None, None, 1 0 add_13[0][0]


res5a_branch2a (Conv2D) (None, None, None, 5 524800 activation_40[0][0]


bn5a_branch2a (BatchNormalizati (None, None, None, 5 2048 res5a_branch2a[0][0]


activation_41 (Activation) (None, None, None, 5 0 bn5a_branch2a[0][0]


res5a_branch2b (Conv2D) (None, None, None, 5 2359808 activation_41[0][0]


bn5a_branch2b (BatchNormalizati (None, None, None, 5 2048 res5a_branch2b[0][0]


activation_42 (Activation) (None, None, None, 5 0 bn5a_branch2b[0][0]


res5a_branch2c (Conv2D) (None, None, None, 2 1050624 activation_42[0][0]


res5a_branch1 (Conv2D) (None, None, None, 2 2099200 activation_40[0][0]


bn5a_branch2c (BatchNormalizati (None, None, None, 2 8192 res5a_branch2c[0][0]


bn5a_branch1 (BatchNormalizatio (None, None, None, 2 8192 res5a_branch1[0][0]


add_14 (Add) (None, None, None, 2 0 bn5a_branch2c[0][0]
bn5a_branch1[0][0]


activation_43 (Activation) (None, None, None, 2 0 add_14[0][0]


res5b_branch2a (Conv2D) (None, None, None, 5 1049088 activation_43[0][0]


bn5b_branch2a (BatchNormalizati (None, None, None, 5 2048 res5b_branch2a[0][0]


activation_44 (Activation) (None, None, None, 5 0 bn5b_branch2a[0][0]


res5b_branch2b (Conv2D) (None, None, None, 5 2359808 activation_44[0][0]


bn5b_branch2b (BatchNormalizati (None, None, None, 5 2048 res5b_branch2b[0][0]


activation_45 (Activation) (None, None, None, 5 0 bn5b_branch2b[0][0]


res5b_branch2c (Conv2D) (None, None, None, 2 1050624 activation_45[0][0]


bn5b_branch2c (BatchNormalizati (None, None, None, 2 8192 res5b_branch2c[0][0]


add_15 (Add) (None, None, None, 2 0 bn5b_branch2c[0][0]
activation_43[0][0]


activation_46 (Activation) (None, None, None, 2 0 add_15[0][0]


res5c_branch2a (Conv2D) (None, None, None, 5 1049088 activation_46[0][0]


bn5c_branch2a (BatchNormalizati (None, None, None, 5 2048 res5c_branch2a[0][0]


activation_47 (Activation) (None, None, None, 5 0 bn5c_branch2a[0][0]


res5c_branch2b (Conv2D) (None, None, None, 5 2359808 activation_47[0][0]


bn5c_branch2b (BatchNormalizati (None, None, None, 5 2048 res5c_branch2b[0][0]


activation_48 (Activation) (None, None, None, 5 0 bn5c_branch2b[0][0]


res5c_branch2c (Conv2D) (None, None, None, 2 1050624 activation_48[0][0]


bn5c_branch2c (BatchNormalizati (None, None, None, 2 8192 res5c_branch2c[0][0]


add_16 (Add) (None, None, None, 2 0 bn5c_branch2c[0][0]
activation_46[0][0]


activation_49 (Activation) (None, None, None, 2 0 add_16[0][0]


resize_1 (Lambda) (None, None, None, 2 0 activation_49[0][0]


concatenate_1 (Concatenate) (None, None, None, 3 0 resize_1[0][0]
activation_40[0][0]


conv2d_1 (Conv2D) (None, None, None, 1 393344 concatenate_1[0][0]


batch_normalization_1 (BatchNor (None, None, None, 1 512 conv2d_1[0][0]


activation_50 (Activation) (None, None, None, 1 0 batch_normalization_1[0][0]


conv2d_2 (Conv2D) (None, None, None, 1 147584 activation_50[0][0]


batch_normalization_2 (BatchNor (None, None, None, 1 512 conv2d_2[0][0]


activation_51 (Activation) (None, None, None, 1 0 batch_normalization_2[0][0]


resize_2 (Lambda) (None, None, None, 1 0 activation_51[0][0]


concatenate_2 (Concatenate) (None, None, None, 6 0 resize_2[0][0]
activation_22[0][0]


conv2d_3 (Conv2D) (None, None, None, 6 41024 concatenate_2[0][0]


batch_normalization_3 (BatchNor (None, None, None, 6 256 conv2d_3[0][0]


activation_52 (Activation) (None, None, None, 6 0 batch_normalization_3[0][0]


conv2d_4 (Conv2D) (None, None, None, 6 36928 activation_52[0][0]


batch_normalization_4 (BatchNor (None, None, None, 6 256 conv2d_4[0][0]


activation_53 (Activation) (None, None, None, 6 0 batch_normalization_4[0][0]


resize_3 (Lambda) (None, None, None, 6 0 activation_53[0][0]


zero_padding2d_1 (ZeroPadding2D (None, None, None, 2 0 activation_10[0][0]


concatenate_3 (Concatenate) (None, None, None, 3 0 resize_3[0][0]
zero_padding2d_1[0][0]


conv2d_5 (Conv2D) (None, None, None, 3 10272 concatenate_3[0][0]


batch_normalization_5 (BatchNor (None, None, None, 3 128 conv2d_5[0][0]


activation_54 (Activation) (None, None, None, 3 0 batch_normalization_5[0][0]


conv2d_6 (Conv2D) (None, None, None, 3 9248 activation_54[0][0]


batch_normalization_6 (BatchNor (None, None, None, 3 128 conv2d_6[0][0]


activation_55 (Activation) (None, None, None, 3 0 batch_normalization_6[0][0]


conv2d_7 (Conv2D) (None, None, None, 3 9248 activation_55[0][0]


batch_normalization_7 (BatchNor (None, None, None, 3 128 conv2d_7[0][0]


activation_56 (Activation) (None, None, None, 3 0 batch_normalization_7[0][0]


rbox_geo_map (Conv2D) (None, None, None, 4 132 activation_56[0][0]


rbox_angle_map (Conv2D) (None, None, None, 1 33 activation_56[0][0]


lambda_1 (Lambda) (None, None, None, 4 0 rbox_geo_map[0][0]


lambda_2 (Lambda) (None, None, None, 1 0 rbox_angle_map[0][0]


pred_score_map (Conv2D) (None, None, None, 1 33 activation_56[0][0]


pred_geo_map (Concatenate) (None, None, None, 5 0 lambda_1[0][0]
lambda_2[0][0]

Total params: 24,237,478
Trainable params: 24,183,398
Non-trainable params: 54,080


/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/engine/training.py:2087: UserWarning: Using a generator with use_multiprocessing=True and multiple workers may duplicate your data. Please consider using thekeras.utils.Sequence class. UserWarning('Using a generator with use_multiprocessing=True`'
Traceback (most recent call last):
Traceback (most recent call last):
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task
self.queue.put((True, generator_output))
File "", line 2, in put
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task
self.queue.put((True, generator_output))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
EOFError
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task
self.queue.put((True, generator_output))
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-8:
Process Process-13:
Process Process-10:
Traceback (most recent call last):
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task
self.queue.put((True, generator_output))
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-11:
Traceback (most recent call last):
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task
self.queue.put((True, generator_output))
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-12:
EOFError
EOFError

During handling of the above exception, another exception occurred:

EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
Traceback (most recent call last):

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task
self.queue.put((False, e))
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task
self.queue.put((False, e))
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task
self.queue.put((False, e))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
BrokenPipeError: [Errno 32] Broken pipe
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
EOFError
Traceback (most recent call last):
EOFError

During handling of the above exception, another exception occurred:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 676, in _data_generator_task
self.queue.qsize() < self.max_queue_size):
Traceback (most recent call last):
File "", line 2, in qsize
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
BrokenPipeError: [Errno 32] Broken pipe
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task
self.queue.put((False, e))
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task
self.queue.put((False, e))
File "", line 2, in put
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
BrokenPipeError: [Errno 32] Broken pipe
Process Process-9:
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task
self.queue.put((False, e))
File "", line 2, in put
File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
40 training images in /home/johan/Documents/school_projects/temp/train_data40
40 training images in /home/johan/Documents/school_projects/temp/train_data40
40 training images in /home/johan/Documents/school_projects/temp/train_data40
40 training images in /home/johan/Documents/school_projects/temp/train_data40
40 training images in /home/johan/Documents/school_projects/temp/train_data40
Epoch 1/3
40 training images in /home/johan/Documents/school_projects/temp/train_data40
Killed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant