Is it possible to have overlapping segmentation?

[a-parida12]

I am using pydicom-seg to create some dicomseg overlays for chest XRay dicom.
The segmentations are multiclass and overlapping in nature. Currently, I pass a 3D NumPy array to make the segmentation.
But given the reference source image is 2D and a single slice. How doespydicom-seg handle 3D segmentation (in the 3rd dimension I have segmentation of a particular class in a layer)?

Is there a better way to handle the multiple overlapping segmentation?

[razorx89]

Thank you for reporting. In your case it would be a multi-label scenario, where one voxel can have multiple labels. Multi-class is usually defined as a single label per voxel. Unfortunately, pydicom-seg currently does not support writing of multi-label segmentations. The second issue is 2D data. I do not have any reliable reference segmentation for 2D as a guidance, which is working in multiple viewers, yet.

[a-parida12]

You are right. My bad with the terminology, it is indeed a multilabel problem.

This is something I wrote up which seems to work when I read/write using pydicom. What do you think are some pitfalls here?

from pydicom.dataset import FileDataset
import pydicom_seg
import SimpleITK as sitk
from config import meta_json_path


def store_dcmseg(
    source_image: FileDataset, seg_img: np.ndarray, instance_number: int
) -> FileDataset:
    # template created  based on http://qiicr.org/dcmqi/#/seg
    template = pydicom_seg.template.from_dcmqi_metainfo(meta_json_path)

    # check and correct  imagePositionPatient
    patient_position_modified = False
    try:
        source_image.ImagePositionPatient
    except Exception:
        patient_position_modified = True
        source_image.ImagePositionPatient = [0, 0, 0]

    # convert image to sitk format
    seg_img_itk = sitk.GetImageFromArray(seg_img.astype(np.uint8))
    # expecting shape Cx HxW for seg_img; there are C binary labels for C class if it is present or absent
    writer = pydicom_seg.MultiClassWriter(
        template=template,
        inplane_cropping=False,  # Crop image slices to the minimum bounding box on
        # x and y axes. Maybe not supported by other frameworks.
        skip_empty_slices=True,  # Don't encode slices with only zeros
        skip_missing_segment=True,  # If a segment definition is missing in the
        # template, then raise an error instead of
        # skipping it.
    )

    dcmseg = writer.write(seg_img_itk, source_images=[source_image])
    if patient_position_modified:
          dcmseg.ImagePositionPatient = None
    dcmseg.AcquisitionTime = source_image.AcquisitionTime
    return dcmseg

But I am unsure if it works on a generic viewer. Do you have recommendations on a viewer that I could test on?
I have tried it on webviewer(an internal tool) based on the cornerstone and cornerstone tools and it works fine on it.

[razorx89]

Sure, this is a workaround for how pydicom-seg expects the data. But I doubt that the generated SEG file will work in e.g. 3D Slicer or OHIF Viewer. The problem lies in some of the indexing attributes during generation. When writing the frames, an indexing scheme is applied which encodes for 3D volumes and not for 2D images.