How do I create a mask but not optimize it?

[zdoek001]

How to create a black and white mask of a face

[dosubot]

Hey @zdoek001! I'm a bot here to help you out with any bugs, questions, or contributions you have. Let's tackle this issue together while we wait for a human maintainer.

To create a black and white mask of a face without optimizing it, you can use the mediapipe_face_mesh function. This function processes an image to detect face landmarks and creates a mask where the face area is white and the rest is black.

Here is the relevant code:

def mediapipe_face_mesh(
    image: Image.Image, confidence: float = 0.3
) -> PredictOutput[int]:
    import mediapipe as mp

    mp_face_mesh = mp.solutions.face_mesh
    draw_util = mp.solutions.drawing_utils
    drawing_styles = mp.solutions.drawing_styles

    w, h = image.size

    with mp_face_mesh.FaceMesh(
        static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
    ) as face_mesh:
        arr = np.array(image)
        pred = face_mesh.process(arr)

        if pred.multi_face_landmarks is None:
            return PredictOutput()

        preview = arr.copy()
        masks = []

        for landmarks in pred.multi_face_landmarks:
            draw_util.draw_landmarks(
                image=preview,
                landmark_list=landmarks,
                connections=mp_face_mesh.FACEMESH_TESSELATION,
                landmark_drawing_spec=None,
                connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
            )

            points = np.array(
                [[land.x * w, land.y * h] for land in landmarks.landmark], dtype=int
            )
            outline = cv2.convexHull(points).reshape(-1).tolist()

            mask = Image.new("L", image.size, "black")
            draw = ImageDraw.Draw(mask)
            draw.polygon(outline, fill="white")
            masks.append(mask)

        bboxes = create_bbox_from_mask(masks, image.size)
        preview = Image.fromarray(preview)
        return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)

Additionally, you can use the mask_preprocess function from the adetailer library to preprocess the mask. Here is an example:

from PIL import Image
from adetailer.mask import mask_preprocess

# Preprocess the mask
processed_masks = mask_preprocess(masks, kernel=0, x_offset=0, y_offset=0, merge_invert='NONE')

This function will generate a black and white mask of the face detected in the image, and you can further manipulate the mask using additional utility functions if required ^[1][2].

To continue talking to Dosu, mention @dosu.

[zdoek001]

mediapipe_face_mesh

@dosu
Is there an API?

[dosubot]

Yes, there is an API available for the mediapipe_face_mesh function. This function processes an image to detect face landmarks and creates a mask where the face area is white and the rest is black.

Here is the relevant code:

def mediapipe_face_mesh(
    image: Image.Image, confidence: float = 0.3
) -> PredictOutput[int]:
    import mediapipe as mp

    mp_face_mesh = mp.solutions.face_mesh
    draw_util = mp.solutions.drawing_utils
    drawing_styles = mp.solutions.drawing_styles

    w, h = image.size

    with mp_face_mesh.FaceMesh(
        static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
    ) as face_mesh:
        arr = np.array(image)
        pred = face_mesh.process(arr)

        if pred.multi_face_landmarks is None:
            return PredictOutput()

        preview = arr.copy()
        masks = []

        for landmarks in pred.multi_face_landmarks:
            draw_util.draw_landmarks(
                image=preview,
                landmark_list=landmarks,
                connections=mp_face_mesh.FACEMESH_TESSELATION,
                landmark_drawing_spec=None,
                connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
            )

            points = np.array(
                [[land.x * w, land.y * h] for land in landmarks.landmark], dtype=int
            )
            outline = cv2.convexHull(points).reshape(-1).tolist()

            mask = Image.new("L", image.size, "black")
            draw = ImageDraw.Draw(mask)
            draw.polygon(outline, fill="white")
            masks.append(mask)

        bboxes = create_bbox_from_mask(masks, image.size)
        preview = Image.fromarray(preview)
        return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)

This function is part of the adetailer repository and can be used to generate face masks from images ^[1].

To continue talking to Dosu, mention @dosu.

[Bing-su]

What are you trying to get the mask image for?

[zdoek001]

What are you trying to get the mask image for?

Once I have the mask image, I will put it in the mask of the img2img. When it is generated, the facial area is excluded so that it can be processed later in other developments
Is there any way to obtain this mask? Via API。@Bing-su

[Bing-su]

Can't you just draw the mask image directly in img2img inpaint? I don't see the need for a mask.

[zdoek001]

Can't you just draw the mask image directly in img2img inpaint? I don't see the need for a mask.

If I'm using the Web UI, that can be drawn, but I'm using the Web UI API for the request, as a step in the program, so the implementation of the mask generation needs to be automated

[Bing-su]

The WebUI has this feature so you can get inpaint masks, but it's probably not possible with the API.

[zdoek001]

! [chrome_xJgSkCvP8z] (https://private-user-images.githubusercontent.com/37621276/339330273-6f9b3991-5d6b-4815-b200-ede0336efb97.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTgyNzYyNTcsIm5iZiI6MTcxODI3NTk1NywicGF0aCI6Ii8zNzYyMTI3Ni8zMzkzMzAyNzMtNmY5YjM5OTEtNWQ2Yi00ODE1LWIyMDAtZWRlMDMzNmVmYjk3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjEzVDEwNTIzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE3NTJkZWI3MzE2Nzg2YjUwNjllODAwY2I1ZDNlOTMwMmE5NzFjZTUxOWYwOThmYzY4Y2ZjODUyOGU3ZjEwZGUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.Lc1pCMfV4zJmKd5wZ5vU3y9vhr9PTSFxsiGk_gTh8MI) The WebUI has this feature so you can get inpaint masks, but it's probably not possible with the API.

Ok, thanks for the help, is this button in adetailer, please? Or what location? Didn't find it

[Bing-su]

On Saving images/grids