Perform topological operations based on switches

[SanPen]

Description

At the moment, Grid2Op runs with a bus-branch calculation engine. This forces the topological changes to be made by changing the connection points of the elements of the grid. It would be very advantageous to perform the topological actions just by operating on the status of a subset of the grid switches.

Solution I'd like

In the apply_action function of the back-end, I'd like to receive a vector of 0/1 encoding the status of the retained switches to act on the topology.

Alternatives

A simple solution for this, in order to keep using the current bus-branch engine is to model the retained switches as impedances (say x_pu=1e-20)

Of course the alternative is to model the switches as switches and use a calculation engine that does topological reductions before running power flows.

Additional context

This would require to modify the current grids to add more buses and the corresponding switches. therefore there would be no more bus changing on the devices, but rather, everything would be done via switching as it is in the SCADA systems.

[BDonnot]

Hello,

Thanks for this issue :-)

This is indeed a requirement if one wants to work with more advanced powerflow than the one used by default (PandaPowerBackend) or a port to it in c++ (LightSimBackend) that handles things without actual link to real life. And I think it would be an interesting steps toward using grid2op with "close to operational" solutions "in real life" or "near real life".

Lightsim2grid for example has no switches, but its data model is not related to a "bus branch" model either. It is something else, an intermediate representation of things that does not require a topological reduction before running powerflow and does not use (from its interface) the "bus branch" model. Actually it's really similar to the "switches" solution that you describe for the ieee standard cases (the one we use today).

Actually, you can say there is 2 switches for each element (side of powerline / transformer, load, generator, shunt, storage units etc.). And only one (at most) is connected at each step.
For example, if you have the loop "of the backend action":

loads_bus = action.get_loads_bus()
for load_id, new_bus in loads_bus:
         # new_bus is -1 => both switches are opened
         # new_bus is 1 => switch that connects this load to busbar1 (at its substation) is closed, the other one is closed
         # new_bus is 2 => switch that connects this load to busbar2 (at its substation) is closed, the other one is closed

This is the meaning of the "1 / 2" in grid2op actions. It is rigorously equivalent to the opened / closed switches you propose in this way provided that you can "map" a load_id in grid2op and the corresponding switches.

This model is actually a simplification of the "switches" model because you do not need to compute which switches are opened or closed. The agent would tell "I want this" (for today's competition it's always feasible, for later there would be a function to know if this "target topology" is feasible or not) regardless of the

We did it this way because there is no (to my knowledge) full description of the underlying substations for matpower cases. So we supposed that:

• there are 2 busbars on each substation
• all elements (load, gen, storage units, side of powerline / transformer etc.) can be connected to either busbars (one switches per element per busbars)

With a "real" substation configuration I think i'll keep a similar point of view: in grid2op you will specify on which busbars each element is connected. Though you'll be able to do it either using the actual form or with the switches.

On the "side" of the backend of course the next implementation (provided that detailed information about the substation are given) will allow to output a switches state that matches the topology in grid2op (though it will require a "routine calculation" so slow down the backend)

Would that be suitable for you ?

[BDonnot]

I'll try a first implementation here https://github.com/BDonnot/Grid2Op/tree/dev-switches

[BDonnot]

Hello,

I finished the POC of the implementation and pushed the dev branch in this repo:
https://github.com/rte-france/Grid2Op/tree/dev-switches

Basically, if you call the correct function (see backend_action.get_all_switches() ) with the backend action you received in apply_action you get 2 arguments:

busbar_connector_state, switches_state = bkaction.get_all_switches()

and you can do whatever you want with this afterwards.

Also i would more than recommend that you get a look at the example here https://github.com/rte-france/Grid2Op/tree/dev-switches/examples/backend_integration.

[SanPen]

Excellent news.

I have been working on the GridCal editor to (easily) produce grids that make sense for this scheme.

Also, I've been working in reading the pandapower format in the backends repository: https://github.com/SanPen/Grid2OpBackends

Currently, I'm having an issue right after load, because I removed all the pandapower stuff from the Newton and GridCal backends implementations since the api's are way simpler. To fix this, I might need your help... but first I want to understand better how the backends works.

[BDonnot]

You don't "have to" read grid files in pandapower format (which is sometimes rather... obscure i would say).

For simple test, if you already have them, you can assume that the ieee14 is loaded and read this file from gridcal / newton.

Another intermediate solution would be (during the development phases) to install pandapower and to read the grid from the pandapower (and delegate to them the joy to parse their json format).

For example this could look like

def load_grid(self, path=None, filename=None):
        import pandapower as pp        
        if path is None and filename is None:
            raise RuntimeError(
                "You must provide at least one of path or file to load a powergrid."
            )
        if path is None:
            full_path = filename
        elif filename is None:
            full_path = path
        else:
            full_path = os.path.join(path, filename)
        if not os.path.exists(full_path):
            raise RuntimeError('There is no powergrid at "{}"'.format(full_path))

        with warnings.catch_warnings():
            # remove deprecationg warnings for old version of pandapower
            warnings.filterwarnings("ignore", category=DeprecationWarning)
            warnings.filterwarnings("ignore", category=FutureWarning)
            tmp_grid = pp.from_json(full_path)

and then you read the attribute from the tmp_grid

Their API is rather user friendly:

tmp_grid.bus # pandas dataframe of all the buses
tmp_grid.line  # all powerflines
tmp_grid.trafo   # all the transformer
tmp_grid.load
tmp_grid.gen

[SanPen]

Hi,

I've just finished the loading of the json files (see here)

The problem that I get is this:

Any idea?

[BDonnot]

Yes I think it's because these attributes should be set by the backend (and not the other way around).

Grid2op is agnostic of everything power system related. It tells the backend "load this grid and tell me what is in there" (with the "load_grid" function). So actually it's the responsibility of the backend to set these attributes (see an example https://github.com/rte-france/Grid2Op/blob/master/examples/backend_integration/Step1_loading.py for a perfectly functional and cleared from all the optimization of the pandapower backend that makes the code hard to read and understand)

[BDonnot]

If you want an easier things to look is here:
https://grid2op.readthedocs.io/en/latest/createbackend.html#grid-description

def load_grid(self, path=None, filename=None):
    # simply handles different way of inputing the data
    if path is None and filename is None:
        raise RuntimeError("You must provide at least one of path or file to load a powergrid.")
    if path is None:
        full_path = filename
    elif filename is None:
        full_path = path
    else:
        full_path = os.path.join(path, filename)
    if not os.path.exists(full_path):
        raise RuntimeError("There is no powergrid at \"{}\"".format(full_path))

    # load the grid in your favorite format:
    self._grid = ... # all the stuff you do to parse the json format

    # and now initialize the attributes (see list bellow)
    self.n_line = ...  # number of lines in the grid should be read from self._grid (it's powerline + transformer)
    self.n_gen = ...  # number of generators in the grid should be read from self._grid
    self.n_load = ...  # number of generators in the grid should be read from self._grid
    self.n_sub = ...  # number of generators in the grid should be read from self._grid

    # other attributes should be read from self._grid (see table below for a full list of the attributes)
    self.load_to_subid = ...
    self.gen_to_subid = ...
    self.line_or_to_subid = ...
    self.line_ex_to_subid = ...

    # and finish the initialization with a call to this function
    self._compute_pos_big_topo()

    # the initial thermal limit
    self.thermal_limit_a = ...

[SanPen]

I understand, so the pandapower loading of the structures has to remain, and the new backend must be apart, duplicating the grid data potentially.
Is this correct?

Or, can I load the pandapower json and then fill the structures from the new data? (I'd prefer this, even if it is more difficult)

[BDonnot]

I understand, so the pandapower loading of the structures has to remain, and the new backend must be apart, duplicating the grid data potentially.

In the short term, to speed things up yes. But you can totally make your own "pandapower json parser" and intialize a gridcal / netwon grid from this. This is totally feasible but would (in my opinion) take a bit more time.

Or, can I load the pandapower json and then fill the structures from the new data? (I'd prefer this, even if it is more difficult)

Yes you can do that too.

Basically, a sketch of the load_grid function could look like :

Relying on pandapower to read the data

def load_grid(self, path=None, filename=None):
    ###### pandapower part
    if path is None and filename is None:
        raise RuntimeError("You must provide at least one of path or file to load a powergrid.")
    if path is None:
        full_path = filename
    elif filename is None:
        full_path = path
    else:
        full_path = os.path.join(path, filename)
    if not os.path.exists(full_path):
        raise RuntimeError("There is no powergrid at \"{}\"".format(full_path))


        with warnings.catch_warnings():
            # remove deprecationg warnings for old version of pandapower
            warnings.filterwarnings("ignore", category=DeprecationWarning)
            warnings.filterwarnings("ignore", category=FutureWarning)
            tmp_grid = pp.from_json(full_path)  # this grid will not stay in memory !
    #######################

    # now use tmp_grid to initialize your grid
        for i, row in tmp_grid.bus.iterrows():
            bus = dev.Bus(idtag='',
                          code='',
                          name=str(row['name']),
                          active=True,
                          vnom=row['vn_kv'],
                          vmin=row['min_vm_pu'],
                          vmax=row['max_vm_pu'])
            bus_dict[i] = bus
            self._grid.add_bus(bus)


        for i, row in tmp_grid.load.iterrows():
            bus = bus_dict[row['bus']]
            self._grid.add_load(bus, dev.Load(idtag='',
                                              code='',
                                              name=str(row['name']),
                                              active=True,
                                              P=row['p_mw'] * row['scaling'],
                                              Q=row['q_mvar'] * row['scaling']))

    # etc. etc.

Or alternatively

You can make a parser of pandapower json (like you started) and continue like this (I will not copy paste the code but you can totally do that too)

[SanPen]

I believe I have the load function done.

Now it fails at the generators_infofunctions, etc. I assume those are run after a power flow is performed, correct?

[BDonnot]

Yes exactly. A clear implementation of this function is provided here:
https://github.com/rte-france/Grid2Op/blob/master/examples/backend_integration/Step3_modify_gen.py

It is expected to return 3 numpy array:

• prod_p : active generation in MW (generator convention so >= power is injected to the node)
• prod_q : reactive generation in MVAr (generator convention so >= power is injected to the node)
• prod_v : voltage magnitude in kV to the bus to which the generator is connected

[BDonnot]

Oh and a quick remark (I just looked at your code) : grid2op does not make any difference between lines and transformers. So self.n_line should be nb_line + nb_trafo (as shown in the example with pandapower)

[SanPen]

yes, no problem with that as we use the branch nomenclature. Indeed lines + transformers + ...

[BDonnot]

TODO for this feature:

• being able to compute the position of the switches given the "topo_vect" vector
• being able to compute the topo_vect / shunt_bus from the the position of the switches
• being able to act directly on the switches (regular grid2op action)
• documentation
• tests
• example on how to use this feature (would be in beta at first release)

Then :

• integration with gym_compat module
• test
• doc