ValueError when running run_connectivity_sampling.py

[giorgiopiras]

Hi @oozdenizci and thanks for your work on this repo!
Unfortunately, I am currently facing an issue when running run_connectivity_sampling.py that I am struggling to solve. The launched command is:
python -u run_connectivity_sampling.py --data cifar10 --model resnet18 --n_classes 10 -s -pc 0.05 --w_decay 1e-3 --objective "at"
The error happens while the training process is about to start apparently, as no log of ongoing epochs appear.
The line of code from which the error stems from is:
optimizer.apply_gradients(zip(grads, model.param_list(trainable=True)))

Error Log

Traceback (most recent call last):
  File "run_connectivity_sampling.py", line 176, in <module>
    main()
  File "run_connectivity_sampling.py", line 129, in main
    train_step(batch_xs_adv, batch_ys)
  File "run_connectivity_sampling.py", line 100, in train_step
    optimizer.apply_gradients(zip(grads, model.param_list(trainable=True)))
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow_addons/optimizers/weight_decay_optimizers.py", line 154, in apply_gradients
    return super().apply_gradients(grads_and_vars, name=name, **kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 504, in apply_gradients
    return distribute_ctx.get_replica_context().merge_call(
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py", line 2420, in merge_call
    return self._merge_call(merge_fn, args, kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py", line 2427, in _merge_call
    return merge_fn(self._strategy, *args, **kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 282, in wrapper
    return func(*args, **kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 591, in _distributed_apply
    update_ops.extend(distribution.extended.update(
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py", line 2013, in update
    return self._update(var, fn, args, kwargs, group)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py", line 2659, in _update
    return self._update_non_slot(var, fn, (var,) + tuple(args), kwargs, group)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py", line 2665, in _update_non_slot
    result = fn(*args, **kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 282, in wrapper
    return func(*args, **kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 567, in apply_grad_to_update_var
    update_op = self._resource_apply_dense(grad, var, **apply_kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow_addons/optimizers/weight_decay_optimizers.py", line 175, in _resource_apply_dense
    with tf.control_dependencies([self._decay_weights_op(var)]):
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow_addons/optimizers/weight_decay_optimizers.py", line 159, in _decay_weights_op
    self._get_hyper("weight_decay", var.dtype) * var, self._use_locking
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 1072, in _run_op
    return tensor_oper(a.value(), *args, **kwargs)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py", line 1009, in r_binary_op_wrapper
    x = ops.convert_to_tensor(x, dtype=y.dtype.base_dtype, name="x")
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1341, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 321, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 261, in constant
    return _constant_impl(value, dtype, shape, name, verify_shape=False,
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 270, in _constant_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "/home/gpiras/anaconda3/envs/bcsp/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 96, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Attempt to convert a value (<tensorflow.python.keras.optimizer_v2.learning_rate_schedule.PiecewiseConstantDecay object at 0x7f72b1ee8f10>) with an unsupported type (<class 'tensorflow.python.keras.optimizer_v2.learning_rate_schedule.PiecewiseConstantDecay'>) to a Tensor.

My env

The env should be compliant with the requirements. I'll list here the main packages:

cudatoolkit               10.1.243             h6bb024c_0    anaconda
cudnn                     7.6.5                cuda10.1_0    anaconda
foolbox                   3.3.1              pyh44b312d_1    conda-forge
keras                     2.4.3              pyhd8ed1ab_0    conda-forge
keras-preprocessing       1.1.2              pyhd3eb1b0_0    anaconda
numpy                     1.23.5                   pypi_0    pypi
pickle-mixin              1.0.2                    pypi_0    pypi
pip                       23.3             py38h06a4308_0  
python                    3.8.18               h955ad1f_0  
pyyaml                    5.3              py38h516909a_0    conda-forge
tensorboard               2.2.0                    pypi_0    pypi
tensorboard-data-server   0.7.0            py38h52d8a92_0    anaconda
tensorboard-plugin-wit    1.8.1            py38h06a4308_0    anaconda
tensorflow                2.2.0           gpu_py38hb782248_0    anaconda
tensorflow-addons         0.10.0                   pypi_0    pypi
tensorflow-base           2.2.0           gpu_py38h83e3d50_0    anaconda
tensorflow-estimator      2.2.0                    pypi_0    pypi
tensorflow-gpu            2.2.0                h0d30ee6_0    anaconda
tensorflow-probability    0.10.0                   pypi_0    pypi

Any chance you could help me? Thanks