Loading IP Adapter FaceId plus model using community example

[ManasInd]

Hi, I have been trying out the IP Adapter Face Id community example, added via #6276.
The readme was very helpful, and I could load the ip-adapter-faceid_sd15.bin weights and was able to get some output images.

Below is the code which worked for me,

`
pipeline = DiffusionPipeline.from_pretrained(
base_model_id,
torch_dtype=torch_data_type,
local_files_only=use_local_model,
custom_pipeline="ip_adapter_face_id",
feature_extractor=None,
safety_checker=None,
image_encoder=image_encoder,
)

pipeline.load_ip_adapter_face_id("/dev/models/", "ip-adapter-faceid-_sd15.bin")

`

However, when I am trying to load ip-adapter-faceid-plus_sd15.bin, then I get the below errors:

I found in some references online that ip-adapter-faceid-plus_sd15.bin requires image encoder so I modified the code as well but no luck :

`

image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "h94/IP-Adapter", 
    subfolder="models/image_encoder",
    torch_dtype=torch.float16,
)
    
pipeline = DiffusionPipeline.from_pretrained(
    base_model_id,
    torch_dtype=torch_data_type,
    local_files_only=use_local_model,
    custom_pipeline="ip_adapter_face_id",
    feature_extractor=None,
    safety_checker=None,
    image_encoder=image_encoder,
)

pipeline.load_ip_adapter_face_id("/dev/models/", "ip-adapter-faceid-plus_sd15.bin")`

Requesting any inputs to help me figure out what I may be doing wrong here!

[fabiorigano]

Hi @ManasInd, I am the contributor of IP Adapter Face ID.
The community example only works with the ip-adapter-faceid_sd15.bin weights, because the plus version uses a different image projection model (see the original implementation https://github.com/tencent-ailab/IP-Adapter/blob/9315694d639258e9eef59a5e8aba21b58c0d5e0d/ip_adapter/ip_adapter_faceid.py#L85 for reference).

[ManasInd]

Thanks @fabiorigano for the details. This helps! Otherwise, I was wondering if I am doing something wrong in my code.