Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate multi objective optimization #692

Open
bpkroth opened this issue Feb 28, 2024 · 0 comments
Open

Integrate multi objective optimization #692

bpkroth opened this issue Feb 28, 2024 · 0 comments
Assignees
@motus
Labels
enhancement New feature or request

Comments

@bpkroth
Copy link
Contributor

bpkroth commented Feb 28, 2024

Both FLAML and SMAC support multi objective optimization now.

We should extend mlos_core and mlos_bench to handle this.
@bpkroth bpkroth added the enhancement New feature or request label Feb 28, 2024
@motus motus self-assigned this Mar 22, 2024
This was referenced Mar 22, 2024
Merged
Merged
bpkroth pushed a commit that referenced this issue Apr 29, 2024
@motus
Pass multiple metrics into the optimizer (#723)
58464b9 
We will need this for multi-objective optimization as well as for
training the optimizers that can take muti-dimensional input

* [x] Make `Storage.Experiment.load()` return multiple scores
* [x] Fix unit tests to check for loading multi-dimensional scores from
the DB
* [x] Make `Optimizer.register()` and `.bulk_register()` take
multi-dimensional trial scores
* [x] Fix the Optimizer unit tests to check for registering
multi-dimensional scores
* [x] Check the Scheduler and optimization loop unit tests to see if we
need to adjust the types etc.

**NOTE:** In this PR, we _do not_ change mlos_core: we will still pass a
single scalar into it and do not change the API on mlos_core side. We
will change mlos_core in the subsequent PR to minimize the diff.

Part of #692
bpkroth added a commit that referenced this issue Apr 30, 2024
@motus @bpkroth
Allow multiple objectives in the optimizer config (#725)
fbdc116 
* [x] Update JSON schema to use `"optimizer_target": {"score": "min"}`
format
* [x] Update base Optimizer class to use the new config format (throw
`NotImplementedError` if > 1 target)
* [x] Modify all unit tests to use the new format

Part of #692

Merge after #723

---------

Co-authored-by: Brian Kroth <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@motus motus

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@motus @bpkroth