Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor check dialect to support multiple devices via hal.tensor.device. #17738

Open
4 tasks
benvanik opened this issue Jun 25, 2024 · 0 comments
Open
4 tasks
Assignees
Labels
cleanup 🧹 compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm) infrastructure Relating to build systems, CI, or testing

Comments

@benvanik
Copy link
Collaborator

Currently the dialect predates external modules and does some weird stuff to support conversion through flow/stream/hal down to the point it becomes VM calls. Of particular issue is that the calls require a !hal.device but that's only introduced during HAL conversion at which point we've already discarded the affinities that would make it possible to infer. As a workaround #17482 forces check tests to use device 0 when lowering into HAL.

Instead we should probably think of the check dialect as an input dialect and convert it to an extern util.func/util.call with HAL ops early on. The extern call would take HAL buffer views and a HAL device that we resolve via hal.tensor.export and some new hal.tensor.device (?) op that allows for retrieving the device at input time. This would also allow custom modules to get device handles as required.

The hal.tensor.device op could take one or more tensors and let us fail if any of the tensors end up with differing affinities. In ConvertToStream the affinity analysis would allow it to be turned into a stream.context.resolve.

TODO:

  • add hal.tensor.device op
  • add stream conversion of hal.tensor.device to stream.context.resolve
  • add pass (maybe via dialect interface) at input time to convert check dialect to calls
  • remove check dialect conversion for HAL/VM
@benvanik benvanik added infrastructure Relating to build systems, CI, or testing compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm) cleanup 🧹 labels Jun 25, 2024
@benvanik benvanik self-assigned this Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup 🧹 compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm) infrastructure Relating to build systems, CI, or testing
Projects
None yet
Development

No branches or pull requests

1 participant