Building ´constraint_candidate_connection_flow_lb´ take too much time compared to ´constraint_candidate_connection_flow_ub´

[datejada]

Describe the bug
When running the benchmark with the investment option, building the ´constraint_candidate_connection_flow_lb´ takes 30 times the amount of creating the ´constraint_candidate_connection_flow_ub´ constraint. They are, in principle, similar constraints, so the time to build them should not be that different.

To Reproduce

1. Configure the last part of the file benchmark.jl as follows:

# url_in_basic, url_out_basic =
# setup(number_of_weeks=8, n_count=100, add_meshed_network=true, add_investment=false, add_rolling=false)
url_in_invest, url_out_invest = setup(number_of_weeks=4, n_count=10, add_investment=true, add_rolling=false)
# url_in_roll, url_out_roll =
#     setup(number_of_weeks=8, n_count=10, add_meshed_network=false, add_investment=false, add_rolling=true)

# SUITE["main", "run_spineopt", "basic"] =
#     @benchmarkable run_spineopt($url_in_basic, $url_out_basic; log_level=3, optimize=false) samples = 3 evals = 1 seconds =
#         Inf
SUITE["main", "run_spineopt", "investment"] =
    @benchmarkable run_spineopt($url_in_invest, $url_out_invest; log_level=3, optimize=false) samples = 1 evals = 1 seconds =
        Inf
# SUITE["main", "run_spineopt", "roll"] =
#     @benchmarkable run_spineopt($url_in_roll, $url_out_roll; log_level=3, optimize=true) samples = 3 evals = 1 seconds =
#         Inf

2. Open a Julia terminal in the SpineOpt package and run the following commands:

using Pkg
Pkg.activate("benchmark")
pkg"dev ."
Pkg.instantiate()
include("../benchmark/benchmarks.jl")
results = run(SUITE, verbose=true)

Expected behavior
These commands will run the benchmark case study 2 times. You will see on the SpineOpt log in the terminal that the lb constraints is the one take most of the time for each run, being almost 30 times the time to build the ub constraint.

We should expect a building time for the lb similar to the ub constraint.

Desktop (please complete the following information):

• Which SpineOpt version?
  SpineOpt version 0.8.2
  SpineInterface version 0.14.2
• Which toolbox version? No use of Toolbox
• OS: Windows

[datejada]

Full Julia log from SpineOpt:

Running SpineOpt...
[SpineOpt version 0.8.2 (git hash: 63337b52886bbac5e4678740e912e1c6df6040a9)]
[SpineInterface version 0.14.2 (git hash: N/A)]
Execution started at 2024-06-25T11:19:18.926
Reading input data from sqlite://...
Initializing base data structure from db... 24.752404 seconds (17.68 M allocations: 1.161 GiB, 1.66% gc time.28% compilation time: <1% of which was recompilation)
Preprocessing base data structure... 27.062737 seconds (16.49 M allocations: 1.100 GiB, 1.48% gc time, 99.67mpilation time)
Checking base data structure...  1.182295 seconds (483.95 k allocations: 33.229 MiB, 99.66% compilation time
Creating instance temporal structure...[ Info: the last time slice of temporal block two_year has been cut tt within the block
  6.247015 seconds (5.02 M allocations: 338.999 MiB, 2.57% gc time, 98.92% compilation time: <1% of which wacompilation)
Creating instance stochastic structure...  4.964215 seconds (3.80 M allocations: 259.783 MiB, 1.81% gc time,74% compilation time)
Adding instance variables...
- [variable_binary_gas_connection_flow]  2.041904 seconds (1.17 M allocations: 78.354 MiB, 1.51% gc time, 99 compilation time)
- [variable_connection_flow]  7.973441 seconds (15.67 M allocations: 1.620 GiB, 10.40% gc time, 75.50% compion time: <1% of which was recompilation)
- [variable_connection_intact_flow]  2.798814 seconds (11.80 M allocations: 1.321 GiB, 30.04% gc time, 24.14mpilation time)
mpilation time)
- [variable_unit_flow]  6.265331 seconds (9.81 M allocations: 1.033 GiB, 3.75% gc time, 84.67% compilation time)- [variable_unit_flow_op]  0.559807 seconds (517.29 k allocations: 35.185 MiB, 6.27% gc time, 99.77% compilation time)
- [variable_unit_flow_op_active]  0.315133 seconds (283.35 k allocations: 19.144 MiB, 99.64% compilation time)
- [variable_units_invested]  1.805284 seconds (1.13 M allocations: 76.359 MiB, 99.78% compilation time)
- [variable_units_invested_available]  0.425938 seconds (227.47 k allocations: 15.586 MiB, 99.63% compilation time)
- [variable_units_mothballed]  0.073880 seconds (32.71 k allocations: 2.422 MiB, 98.44% compilation time)
- [variable_units_on]  1.397784 seconds (2.14 M allocations: 164.215 MiB, 3.21% gc time, 86.29% compilation time)
- [variable_units_out_of_service]  0.603712 seconds (493.97 k allocations: 33.226 MiB, 99.77% compilation time)
- [variable_units_returned_to_service]  0.098755 seconds (61.75 k allocations: 4.175 MiB, 99.49% compilation time)
- [variable_units_shut_down]  0.708097 seconds (1.61 M allocations: 126.858 MiB, 5.66% gc time, 75.63% compilation time)
- [variable_units_started_up]  0.269307 seconds (1.29 M allocations: 105.125 MiB, 18.08% gc time, 14.46% compilation time)
- [variable_units_taken_out_of_service]  0.048831 seconds (70.45 k allocations: 4.746 MiB, 99.16% compilation time)
- [variable_user_constraint_slack_neg]  0.514642 seconds (449.28 k allocations: 30.493 MiB, 99.88% compilation time)
- [variable_user_constraint_slack_pos]  0.005367 seconds (3.79 k allocations: 332.875 KiB, 95.82% compilation time)
 37.233201 seconds (58.07 M allocations: 5.512 GiB, 6.20% gc time, 83.59% compilation time: <1% of which was recompilation)
Adding instance expressions...
- [expression_capacity_margin]  0.405245 seconds (627.71 k allocations: 40.417 MiB, 99.90% compilation time)
  0.406151 seconds (627.76 k allocations: 40.421 MiB, 99.68% compilation time)
Adding instance constraints...
- [constraint_candidate_connection_flow_lb]1827.741170 seconds (12.61 G allocations: 1.508 TiB, 21.62% gc time, 0.28% compilation time)
- [constraint_candidate_connection_flow_ub] 60.497981 seconds (7.70 M allocations: 801.898 MiB, 96.95% gc time, 1.67% compilation time)
- [constraint_compression_ratio]  0.622695 seconds (414.93 k allocations: 29.092 MiB, 99.96% compilation time)
- [constraint_connection_flow_capacity] 11.167465 seconds (33.94 M allocations: 3.495 GiB, 15.67% gc time, 43.71% compilation time)
- [constraint_connection_flow_gas_capacity]  0.536702 seconds (270.80 k allocations: 19.035 MiB, 99.96% compilation time)
- [constraint_connection_flow_intact_flow]  0.637649 seconds (472.47 k allocations: 31.274 MiB, 99.94% compilation time)
- [constraint_connection_flow_lodf]  0.842208 seconds (759.01 k allocations: 49.385 MiB, 99.79% compilation time)
- [constraint_connection_intact_flow_capacity] 11.420445 seconds (39.91 M allocations: 4.400 GiB, 10.73% gc time, 16.57% compilation time)
- [constraint_connection_intact_flow_ptdf]  0.671740 seconds (407.77 k allocations: 27.378 MiB, 99.95% compilation time)
- [constraint_connection_lifetime]  0.527264 seconds (218.72 k allocations: 15.292 MiB, 99.96% compilation time)
- [constraint_connection_unitary_gas_flow]  0.146138 seconds (166.65 k allocations: 11.196 MiB, 99.86% compilation time)
- [constraint_connections_invested_available]  0.792733 seconds (422.37 k allocations: 29.056 MiB, 99.84% compilation time)
- [constraint_connections_invested_transition]  3.605853 seconds (1.52 M allocations: 104.184 MiB, 99.79% compilation time)
- [constraint_cyclic_node_state]  1.019721 seconds (299.71 k allocations: 20.757 MiB, 23.91% gc time, 99.95% compilation time)
- [constraint_fix_node_pressure_point]  0.559549 seconds (315.63 k allocations: 22.074 MiB, 99.96% compilation time)
- [constraint_fix_ratio_in_in_unit_flow]  0.644149 seconds (604.80 k allocations: 41.864 MiB, 99.97% compilation time)
- [constraint_fix_ratio_in_out_unit_flow]  0.029856 seconds (552 allocations: 38.672 KiB, 99.54% compilation time)
- [constraint_fix_ratio_out_in_connection_flow]  6.804034 seconds (13.66 M allocations: 1.170 GiB, 5.08% gc time, 66.75% compilation time)
- [constraint_fix_ratio_out_in_unit_flow]  0.048229 seconds (3.65 k allocations: 290.469 KiB, 98.94% compilation time)
- [constraint_fix_ratio_out_out_unit_flow]  0.034188 seconds (552 allocations: 38.422 KiB, 99.62% compilation time)
- [constraint_investment_group_equal_investments]  0.638100 seconds (290.67 k allocations: 19.381 MiB, 99.95% compilation time)
- [constraint_investment_group_maximum_capacity_invested_available]  0.568133 seconds (380.23 k allocations: 26.388 MiB, 99.97% compilation time)
- [constraint_investment_group_maximum_entities_invested_available]  0.556297 seconds (345.98 k allocations: 23.705 MiB, 99.97% compilation time)
- [constraint_investment_group_minimum_capacity_invested_available]  0.506468 seconds (203.13 k allocations: 14.430 MiB, 99.97% compilation time)
- [constraint_investment_group_minimum_entities_invested_available]  0.503289 seconds (203.13 k allocations: 14.414 MiB, 99.97% compilation time)
- [constraint_max_node_pressure]  1.266021 seconds (755.32 k allocations: 53.160 MiB, 99.93% compilation time)
- [constraint_max_node_voltage_angle]  0.414510 seconds (181.79 k allocations: 12.447 MiB, 99.89% compilation time)
- [constraint_max_ratio_in_in_unit_flow]  0.139868 seconds (33.22 k allocations: 2.287 MiB, 99.84% compilation time)
- [constraint_max_ratio_in_out_unit_flow]  0.029256 seconds (552 allocations: 38.672 KiB, 99.49% compilation time)
- [constraint_max_ratio_out_in_connection_flow]  0.135107 seconds (98.28 k allocations: 6.532 MiB, 99.84% compilation time)
- [constraint_max_ratio_out_in_unit_flow]  0.029812 seconds (552 allocations: 38.672 KiB, 99.47% compilation time)
- [constraint_max_ratio_out_out_unit_flow]  0.013252 seconds (552 allocations: 38.422 KiB, 99.25% compilation time)
- [constraint_max_total_cumulated_unit_flow_from_node]  0.551774 seconds (385.57 k allocations: 27.153 MiB, 99.96% compilation time)
- [constraint_max_total_cumulated_unit_flow_to_node]  0.027335 seconds (505 allocations: 33.688 KiB, 99.43% compilation time)
- [constraint_min_capacity_margin]  0.205596 seconds (125.84 k allocations: 8.674 MiB, 99.92% compilation time)
- [constraint_min_down_time]  9.350478 seconds (15.08 M allocations: 1.269 GiB, 2.80% gc time, 76.42% compilation time)
- [constraint_min_node_pressure]  0.312055 seconds (111.38 k allocations: 7.786 MiB, 99.89% compilation time)
- [constraint_min_node_voltage_angle]  0.284758 seconds (111.41 k allocations: 7.792 MiB, 99.87% compilation time)
- [constraint_min_ratio_in_in_unit_flow]  0.120674 seconds (33.23 k allocations: 2.290 MiB, 99.85% compilation time)
- [constraint_min_ratio_in_out_unit_flow]  0.012670 seconds (552 allocations: 38.672 KiB, 99.17% compilation time)
- [constraint_min_ratio_out_in_connection_flow]  0.139118 seconds (47.45 k allocations: 3.284 MiB, 99.83% compilation time)
- [constraint_min_ratio_out_in_unit_flow]  0.013027 seconds (552 allocations: 38.672 KiB, 99.29% compilation time)
- [constraint_min_ratio_out_out_unit_flow]  0.035871 seconds (555 allocations: 43.000 KiB, 99.65% compilation time)
- [constraint_min_scheduled_outage_duration]  0.548676 seconds (299.54 k allocations: 21.180 MiB, 99.97% compilation time)
- [constraint_min_total_cumulated_unit_flow_from_node]  0.111979 seconds (47.27 k allocations: 3.274 MiB, 99.84% compilation time)
- [constraint_min_total_cumulated_unit_flow_to_node]  0.012323 seconds (505 allocations: 33.688 KiB, 99.15% compilation time)
- [constraint_min_up_time]  3.526574 seconds (9.96 M allocations: 895.693 MiB, 6.97% gc time, 58.83% compilation time)
- [constraint_minimum_operating_point]  7.085160 seconds (8.02 M allocations: 812.299 MiB, 3.57% gc time, 82.76% compilation time)
- [constraint_nodal_balance]  4.388330 seconds (9.44 M allocations: 844.749 MiB, 5.65% gc time, 71.84% compilation time)
- [constraint_node_injection]  6.395699 seconds (12.32 M allocations: 1.089 GiB, 3.88% gc time, 74.49% compilation time)
- [constraint_node_state_capacity]  3.609460 seconds (8.57 M allocations: 684.001 MiB, 71.12% compilation time)
- [constraint_node_voltage_angle]  0.849589 seconds (357.00 k allocations: 24.915 MiB, 99.87% compilation time)
- [constraint_non_spinning_reserves_lower_bound]  0.690712 seconds (415.24 k allocations: 28.466 MiB, 99.95% compilation time)
- [constraint_non_spinning_reserves_shut_down_upper_bound]  0.448404 seconds (369.45 k allocations: 24.528 MiB, 99.86% compilation time)
- [constraint_non_spinning_reserves_start_up_upper_bound]  0.029192 seconds (1.04 k allocations: 74.781 KiB, 99.17% compilation time)
- [constraint_operating_point_bounds]  0.584250 seconds (436.48 k allocations: 29.642 MiB, 99.88% compilation time)
- [constraint_operating_point_rank]  0.316418 seconds (186.36 k allocations: 12.952 MiB, 99.86% compilation time)
- [constraint_ramp_down]  8.386314 seconds (14.36 M allocations: 1.455 GiB, 6.45% gc time, 75.12% compilation time)
- [constraint_ramp_up]  5.845309 seconds (13.02 M allocations: 1.368 GiB, 4.94% gc time, 63.33% compilation time)
- [constraint_ratio_out_in_connection_intact_flow]  0.403084 seconds (223.07 k allocations: 14.662 MiB, 99.91% compilation time)
- [constraint_storage_lifetime]  0.527086 seconds (192.19 k allocations: 13.466 MiB, 99.96% compilation time)
- [constraint_storage_line_pack]  0.539318 seconds (231.74 k allocations: 16.151 MiB, 99.95% compilation time)
- [constraint_storages_invested_available]  1.228179 seconds (469.67 k allocations: 32.248 MiB, 99.86% compilation time)
- [constraint_storages_invested_transition]  1.551908 seconds (531.93 k allocations: 36.712 MiB, 99.76% compilation time)
- [constraint_unit_flow_capacity]  4.906600 seconds (5.60 M allocations: 483.644 MiB, 6.19% gc time, 83.38% compilation time)
- [constraint_unit_flow_op_bounds]  0.751016 seconds (318.37 k allocations: 22.522 MiB, 99.88% compilation time)
- [constraint_unit_flow_op_rank]  0.738031 seconds (295.21 k allocations: 20.593 MiB, 99.88% compilation time)
- [constraint_unit_flow_op_sum]  0.504199 seconds (281.02 k allocations: 19.969 MiB, 99.96% compilation time)
- [constraint_unit_lifetime]  0.486118 seconds (191.70 k allocations: 13.428 MiB, 99.96% compilation time)
- [constraint_unit_pw_heat_rate]  0.518854 seconds (329.62 k allocations: 23.303 MiB, 99.96% compilation time)
- [constraint_unit_state_transition]  2.957965 seconds (3.42 M allocations: 319.275 MiB, 26.28% gc time, 61.66% compilation time)
- [constraint_units_available]  2.807443 seconds (6.95 M allocations: 558.618 MiB, 74.69% compilation time)
- [constraint_units_invested_available]  0.864375 seconds (286.84 k allocations: 20.031 MiB, 99.83% compilation time)
- [constraint_units_invested_transition]  1.850761 seconds (617.24 k allocations: 42.276 MiB, 99.80% compilation time)
- [constraint_units_out_of_service_contiguity]  0.487235 seconds (174.12 k allocations: 12.263 MiB, 99.96% compilation time)
- [constraint_units_out_of_service_transition]  0.342763 seconds (200.47 k allocations: 13.729 MiB, 99.87% compilation time)
- [constraint_user_constraint]  0.624254 seconds (906.76 k allocations: 60.182 MiB, 99.95% compilation time)
2015.730125 seconds (12.86 G allocations: 1.530 TiB, 22.88% gc time, 4.78% compilation time)
Setting instance objective...┌ Warning: no objective terms defined
└ @ SpineOpt C:\github-repos\SpineOpt-master\SpineOpt.jl\src\run_spineopt_standard.jl:242
  6.244791 seconds (2.69 M allocations: 181.892 MiB, 99.88% compilation time)
Execution complete. Started at 2024-06-25T11:19:18.926, ended at 2024-06-25T11:55:01.455, elapsed time: 35 minutes, 42 seconds, 529 milliseconds

[manuelma]

We should expect a building time for the lb similar to the ub constraint.

Why? I don't think these constraints do similar things, they just have similar names. If you look at the formulation you can see how much more complex constraint_candidate_connection_flow_lb is.

[datejada]

After the developers' meeting, the main actions about this issue:

• Check the input parameters and disable the constraint if it is not needed (e.g., there are PTDFs defined)
• If the user wants/needs the constraint, then try to improve the building time as much as possible