Allocation on device RTX 4070

[Milor123]

Guys, is not possible use it in my RTX 4070 with 12GB VRAM?
I am in linux with 4070 dedicated only for AI and my IGPU renders my desktop, I have 48gb of ram

I would like use it, I've tried bfloat and float but nothings works

[yunlianwangluo]

same, :(
maybe we should try better hardware...

[cardosofelipe]

12GB of VRAM are not enough for this model. I an running on a RTX 3090 24GB and I basically have 80% of VRAM consumption only by running IDM-VTON. This is for inference. For training, my GPU is not even enough.

[txhno]

It is a problem with this implementation, If you use the original repository and the diffusers library you can run the FP4 alternative which even runs with 8GB VRAM.