import pyoptsparse hangs (on local machine) when using a vpn

[cfe316]

Description

This feels like an odd one.
I'm running python on my local machine.
When I'm not connected to a certain VPN, I can import pyoptsparse. When I am connected, it hangs on the import command.
This happens both in a python repl and a jupyter notebook (using the same kernel).

The import command stops hanging and succeeds a few moments after disconnecting the vpn.

This seems very odd to me, as I don't know why there would be any network-calling code in pyoptsparse (other than perhaps to check license for certain optimizers?)

Steps to reproduce issue

1. Have pyoptsparse installed successfully - I can run Openmdao problems using IPOPT, for example.
2. Turn on the Pulse Secure VPN so I connect to my lab's servers
3. Open a new, local python repl or notebook
4. import pyoptsparse

Current behavior

it hangs until I disconnect from the VPN

Expected behavior

it imports

Code versions

• Operating System: pop!_os
• Python: 3.10.2
• OpenMPI: n/a
• CGNS: idk what this is
• PETSc: n/a
• Compiler: not relevant?
• This repository: 2.9.2

[marcomangano]

Hello Jacob. This sounds very surprising to me as, to the best of my knowledge, pyoptsparse does not rely on network connections in any part of the code base and the import works fine on any offline machine. The proprietary optimizers we support do not check for the license online.

How are you setting up your python environment? Are you using bare-metal Python or do you rely on Conda or other environment/package managers?
Does this happen only with pyOptSparse?
Have you installed any of the extra dependencies, using pip install .[optview] or [testing] ?

One approach to try and bisect the origin of the issue could be to comment out entries from pyoptsparse/__init__.py and see if the issue stems from any specific submodule - although I doubt this would actually lead to something.
Unfortunately we are not able to replicate the issue, feel free to share any additional info you deem relevant.

[cfe316]

Hi Marco,

I've tried doing some bisecting and other searching, though not exhaustively.

Here's what I'm seeing

All these work while NOT on the vpn

>>> import pyoptsparse
>>> import openmdao
>>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> import sklearn

...
while on the vpn

>>> import sklearn ✓
>>> import openmdao ✓
>>> import matplotlib.pyplot as plt ✓
>>> import numpy as np ✓
>>> import pyoptsparse ✗

Try commenting out lines in __init__.py
-------
turn off all everything
yes (works on vpn)

turn off everything exept History
yes

turn off everything except History and Variable
yes

turned on all optimizers
no

turn off SNOPT to CONMIN
no

turn off everything except PSQP
no

turn off everything except History, Variable, Gradient
no

turn off everything except History, Variable, Solution
no

turn off everything except History, Variable, SLSQP
no

Went through imports in pyOpt_optimizer
----------------

from collections import OrderedDict
import copy ✓
import datetime ✓
from enum import Enum ✓
import os ✓
import shutil ✓
import tempfile ✓
import time ✓
from typing import Any, Callable, Dict, List, Optional, Union ✓

from baseclasses import BaseSolver ✗

...

import baseclasses ✗

So far it looks like an issue with baseclasses.

[cfe316]

When not connected to wifi at all, it works - but whichever of these two is loaded first is subject to a 4 or 5-second delay

>>> import baseclasses < 5 second delay >
>>> import pyoptsparse < instant >

>>> import pyoptsparse < 5 second delay >
>>> import baseclasses < instant >

This is strong evidence that the issue is coming from baseclasses. I can reopen the issue under that repo if you like.

[marcomangano]

Nice detective work! No need to open a new issue right now imho, we can keep the conversation here until we narrow down the problem more specifically.

I cannot replicate the VPN issue but I also observe a 1-2 sec delay when baseclasses is imported. Could you try and do a similar test with the baseclasses imports? I cannot pinpoint any suspicious file or sub-import at the moment.

[ewu63]

To add on to what @marcomangano said, can you also include the following information?

• how you're installing the package (pyOptSparse, baseclasses etc.)
• which distribution of Python you're using (system Python from pop, anaconda, etc.)

[cfe316]

Thank you for the encouragement! I may have some more time to troubleshoot baseclasses later.

How installed

• I believe I installed pyoptsparse using build_pyoptsparse

According to pipdeptree I have baseclasses (only) because of pyoptsparse.

pyoptsparse==2.10.1
├── mdolab-baseclasses [required: >=1.3.1, installed: 1.8.0]

which distribution

I used pyenv to install a fresh python 3.10. It's not the system python. No anaconda in sight.
I then use the pyenv-virtualenv plugin to manage a 'virtualenv' for this installation. I may have other virtualenvs or python installs which also contain mdolab-baseclasses, but these should all be independent from one another thanks to pyenv.

[marcomangano]

I use the same pyenv-virtualenv setup on my machine so that part should be good.

I don't have experience with build_pyoptsparse but I don't think the issues stem from there. It looks like they pinned pyoptsparse to version 2.9.2 though, so maybe you changed something or installed pyoptsparse directly? Anyway, since you are not using conda build_pyoptsparse should have installed IPOPT on your machine and added the correct variable to your environment, so a regular pip install from the pyoptsparse root folder should link to IPOPT properly in future installations. According to what you posted above, it does not look like that a clean install would fix the issue.

Anyway, keep us posted about the baseclasses import debugging when you have time to deal with it!

[cfe316]

Continued debugging:

Within baseclasses, it seems to hang (when on the vpn) on from .testing import BaseRegTest, getTol but not anything else.
This is also the import that causes a few seconds delay when not on the vpn.

[cfe316]

.testing also has from mpi4py import MPI.

This line, alone (outside of pyoptsparse or baseclasses) causes the same behavior: delay when not on VPN. Hangs forever when on vpn. So I think it must be deeper in there.

[ewu63]

Do you have MPI/mpi4py installed into the environment? If so can you list the versions and which MPI implementation you are using? Lastly, can you try the suggestion here for disabling MPI? I suspect it won't work since baseclasses does not respect this env var, but perhaps worth a try.

[cfe316]

Thanks nwu. Just saw the suggestion to disable it, but you're right that doesn't work.
I'll start investigating my mpi installation. Thanks again.