Mixed precision #37
Mixed precision #37
Conversation
Due to lbfgs optim it is required to change the logic of closure.
| import time | ||
|
|
||
| os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE' | ||
|
|
There was a problem hiding this comment.
тут видимо нужно добавить
sys.path.append('../')
sys.path.pop()
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( file ), '..')))
чтобы скрипт запускался.
| @@ -125,9 +125,9 @@ | |||
|
|
|||
|
|
|||
There was a problem hiding this comment.
тут если поставить параметры use_cache=True, save_always=True, mixed_precision=True, то при втором запуске (уже с кэшем) выскакивает ошибка.
examples/example_wave_physics.py
Outdated
| @@ -182,27 +182,27 @@ def wave_experiment(grid_res, CACHE): | |||
|
|
|||
| start = time.time() | |||
|
|
|||
| equation = Equation(grid, wave_eq, bconds).set_strategy('NN') | |||
| equation = Equation(grid, wave_eq, bconds).set_strategy('autograd') | |||
|
|
|||
| img_dir = os.path.join(os.path.dirname(__file__), 'wave_example_physics_img') | |||
|
|
|||
| if not (os.path.isdir(img_dir)): | |||
| os.mkdir(img_dir) | |||
|
|
|||
|
|
|||
There was a problem hiding this comment.
тут ошибка аналогичная как и в скрипте example_wave_periodic.py. При втором запуске с параметрами use_cache=True, save_always=True, mixed_precision=True.
tedeous/solver.py
Outdated
There was a problem hiding this comment.
Две функции closure и closure_cuda в рамках отдного метода избыточны. Но тут в любом случае нужно разгружать метод solve. Это уже дело рефакторинга самого класса Solver.
* Now cache is working with AMP * Slight refactor cache module
Removed scaler state dict in cache module.
Added PyTorch automatic mixed precision for both CPU/GPU + example. LBFGS is not currently supported due to PyTorch limitations.
This mode allows you to quickly obtain a solution while sacrificing accuracy. An additional method can be tensor decomposition of the model parameters. Currently, experiments are being conducted with tensor decomposition (tensor-train).
PR closes the issue #27.