From fe331aea81a7e2bba1260219699100e62ceec5a5 Mon Sep 17 00:00:00 2001 From: CI/CD Bot Date: Thu, 21 Mar 2024 15:56:53 +0000 Subject: [PATCH] bla --- docs/md/Comfy/Nodes/BasicScheduler.md | 63 +++ docs/md/Comfy/Nodes/CLIPLoader.md | 50 ++ docs/md/Comfy/Nodes/CLIPMergeSimple.md | 55 ++ docs/md/Comfy/Nodes/CLIPSave.md | 83 +++ docs/md/Comfy/Nodes/CLIPSetLastLayer.md | 46 ++ docs/md/Comfy/Nodes/CLIPTextEncode.md | 44 ++ .../Comfy/Nodes/CLIPTextEncodeControlnet.md | 54 ++ docs/md/Comfy/Nodes/CLIPTextEncodeSDXL.md | 88 +++ .../Comfy/Nodes/CLIPTextEncodeSDXLRefiner.md | 61 +++ docs/md/Comfy/Nodes/CLIPVisionEncode.md | 45 ++ docs/md/Comfy/Nodes/CLIPVisionLoader.md | 41 ++ docs/md/Comfy/Nodes/Canny.md | 52 ++ docs/md/Comfy/Nodes/CheckpointLoader.md | 53 ++ docs/md/Comfy/Nodes/CheckpointLoaderSimple.md | 49 ++ docs/md/Comfy/Nodes/CheckpointSave.md | 56 ++ docs/md/Comfy/Nodes/ConditioningAverage.md | 72 +++ docs/md/Comfy/Nodes/ConditioningCombine.md | 42 ++ docs/md/Comfy/Nodes/ConditioningConcat.md | 58 ++ docs/md/Comfy/Nodes/ConditioningSetArea.md | 71 +++ .../Nodes/ConditioningSetAreaPercentage.md | 71 +++ .../Nodes/ConditioningSetAreaStrength.md | 49 ++ docs/md/Comfy/Nodes/ConditioningSetMask.md | 67 +++ .../Nodes/ConditioningSetTimestepRange.md | 56 ++ docs/md/Comfy/Nodes/ConditioningZeroOut.md | 45 ++ docs/md/Comfy/Nodes/ControlNetApply.md | 67 +++ .../md/Comfy/Nodes/ControlNetApplyAdvanced.md | 96 ++++ docs/md/Comfy/Nodes/ControlNetLoader.md | 41 ++ docs/md/Comfy/Nodes/CropMask.md | 66 +++ docs/md/Comfy/Nodes/DiffControlNetLoader.md | 46 ++ docs/md/Comfy/Nodes/DiffusersLoader.md | 60 ++ docs/md/Comfy/Nodes/DualCLIPLoader.md | 46 ++ docs/md/Comfy/Nodes/EmptyImage.md | 60 ++ docs/md/Comfy/Nodes/EmptyLatentImage.md | 52 ++ docs/md/Comfy/Nodes/ExponentialScheduler.md | 52 ++ docs/md/Comfy/Nodes/FeatherMask.md | 87 +++ docs/md/Comfy/Nodes/FlipSigmas.md | 44 ++ docs/md/Comfy/Nodes/FreeU.md | 81 +++ docs/md/Comfy/Nodes/FreeU_V2.md | 88 +++ docs/md/Comfy/Nodes/GLIGENLoader.md | 41 ++ docs/md/Comfy/Nodes/GLIGENTextBoxApply.md | 85 +++ docs/md/Comfy/Nodes/GrowMask.md | 69 +++ docs/md/Comfy/Nodes/HyperTile.md | 100 ++++ docs/md/Comfy/Nodes/HypernetworkLoader.md | 55 ++ docs/md/Comfy/Nodes/ImageBatch.md | 47 ++ docs/md/Comfy/Nodes/ImageBlend.md | 94 ++++ docs/md/Comfy/Nodes/ImageBlur.md | 79 +++ docs/md/Comfy/Nodes/ImageColorToMask.md | 51 ++ docs/md/Comfy/Nodes/ImageCompositeMasked.md | 72 +++ docs/md/Comfy/Nodes/ImageCrop.md | 64 +++ docs/md/Comfy/Nodes/ImageFromBatch.md | 53 ++ docs/md/Comfy/Nodes/ImageInvert.md | 41 ++ .../Comfy/Nodes/ImageOnlyCheckpointLoader.md | 49 ++ .../md/Comfy/Nodes/ImageOnlyCheckpointSave.md | 50 ++ docs/md/Comfy/Nodes/ImagePadForOutpaint.md | 113 ++++ docs/md/Comfy/Nodes/ImageQuantize.md | 105 ++++ docs/md/Comfy/Nodes/ImageScale.md | 72 +++ docs/md/Comfy/Nodes/ImageScaleBy.md | 54 ++ .../md/Comfy/Nodes/ImageScaleToTotalPixels.md | 60 ++ docs/md/Comfy/Nodes/ImageSharpen.md | 94 ++++ docs/md/Comfy/Nodes/ImageToMask.md | 50 ++ docs/md/Comfy/Nodes/ImageUpscaleWithModel.md | 66 +++ .../Comfy/Nodes/InpaintModelConditioning.md | 100 ++++ docs/md/Comfy/Nodes/InvertMask.md | 45 ++ docs/md/Comfy/Nodes/JoinImageWithAlpha.md | 55 ++ docs/md/Comfy/Nodes/KSampler.md | 87 +++ docs/md/Comfy/Nodes/KSamplerAdvanced.md | 108 ++++ docs/md/Comfy/Nodes/KSamplerSelect.md | 42 ++ docs/md/Comfy/Nodes/KarrasScheduler.md | 57 ++ docs/md/Comfy/Nodes/LatentAdd.md | 50 ++ docs/md/Comfy/Nodes/LatentBatch.md | 52 ++ .../md/Comfy/Nodes/LatentBatchSeedBehavior.md | 53 ++ docs/md/Comfy/Nodes/LatentBlend.md | 75 +++ docs/md/Comfy/Nodes/LatentComposite.md | 84 +++ docs/md/Comfy/Nodes/LatentCompositeMasked.md | 74 +++ docs/md/Comfy/Nodes/LatentCrop.md | 75 +++ docs/md/Comfy/Nodes/LatentFlip.md | 50 ++ docs/md/Comfy/Nodes/LatentFromBatch.md | 66 +++ docs/md/Comfy/Nodes/LatentInterpolate.md | 68 +++ docs/md/Comfy/Nodes/LatentMultiply.md | 49 ++ docs/md/Comfy/Nodes/LatentRotate.md | 54 ++ docs/md/Comfy/Nodes/LatentSubtract.md | 50 ++ docs/md/Comfy/Nodes/LatentUpscale.md | 76 +++ docs/md/Comfy/Nodes/LatentUpscaleBy.md | 53 ++ docs/md/Comfy/Nodes/LoadImage.md | 87 +++ docs/md/Comfy/Nodes/LoadImageMask.md | 79 +++ docs/md/Comfy/Nodes/LoadLatent.md | 61 +++ docs/md/Comfy/Nodes/LoraLoader.md | 84 +++ docs/md/Comfy/Nodes/LoraLoaderModelOnly.md | 47 ++ docs/md/Comfy/Nodes/MaskComposite.md | 89 +++ docs/md/Comfy/Nodes/MaskToImage.md | 44 ++ docs/md/Comfy/Nodes/ModelMergeAdd.md | 48 ++ docs/md/Comfy/Nodes/ModelMergeBlocks.md | 74 +++ docs/md/Comfy/Nodes/ModelMergeSimple.md | 53 ++ docs/md/Comfy/Nodes/ModelMergeSubtract.md | 53 ++ .../Comfy/Nodes/ModelSamplingContinuousEDM.md | 68 +++ docs/md/Comfy/Nodes/ModelSamplingDiscrete.md | 71 +++ .../Comfy/Nodes/ModelSamplingStableCascade.md | 56 ++ docs/md/Comfy/Nodes/PatchModelAddDownscale.md | 96 ++++ docs/md/Comfy/Nodes/PerpNeg.md | 74 +++ docs/md/Comfy/Nodes/PhotoMakerEncode.md | 82 +++ docs/md/Comfy/Nodes/PhotoMakerLoader.md | 45 ++ .../Comfy/Nodes/PolyexponentialScheduler.md | 57 ++ .../Comfy/Nodes/PorterDuffImageComposite.md | 98 ++++ docs/md/Comfy/Nodes/PreviewImage.md | 37 ++ docs/md/Comfy/Nodes/RebatchImages.md | 58 ++ docs/md/Comfy/Nodes/RebatchLatents.md | 129 +++++ docs/md/Comfy/Nodes/RepeatImageBatch.md | 45 ++ docs/md/Comfy/Nodes/RepeatLatentBatch.md | 56 ++ docs/md/Comfy/Nodes/RescaleCFG.md | 69 +++ docs/md/Comfy/Nodes/SDTurboScheduler.md | 56 ++ .../Comfy/Nodes/SD_4XUpscale_Conditioning.md | 86 +++ .../Comfy/Nodes/SVD_img2vid_Conditioning.md | 95 ++++ docs/md/Comfy/Nodes/SamplerCustom.md | 109 ++++ docs/md/Comfy/Nodes/SamplerDPMPP_2M_SDE.md | 61 +++ docs/md/Comfy/Nodes/SamplerDPMPP_SDE.md | 61 +++ docs/md/Comfy/Nodes/SaveAnimatedPNG.md | 89 +++ docs/md/Comfy/Nodes/SaveAnimatedWEBP.md | 110 ++++ docs/md/Comfy/Nodes/SaveImage.md | 78 +++ docs/md/Comfy/Nodes/SaveLatent.md | 77 +++ docs/md/Comfy/Nodes/SelfAttentionGuidance.md | 102 ++++ docs/md/Comfy/Nodes/SetLatentNoiseMask.md | 46 ++ docs/md/Comfy/Nodes/SolidMask.md | 55 ++ docs/md/Comfy/Nodes/SplitImageWithAlpha.md | 49 ++ docs/md/Comfy/Nodes/SplitSigmas.md | 48 ++ .../Nodes/StableCascade_EmptyLatentImage.md | 65 +++ .../StableCascade_StageB_Conditioning.md | 51 ++ .../Nodes/StableCascade_StageC_VAEEncode.md | 67 +++ .../Comfy/Nodes/StableZero123_Conditioning.md | 91 +++ .../StableZero123_Conditioning_Batched.md | 108 ++++ docs/md/Comfy/Nodes/StyleModelApply.md | 54 ++ docs/md/Comfy/Nodes/StyleModelLoader.md | 41 ++ docs/md/Comfy/Nodes/TomePatchModel.md | 56 ++ docs/md/Comfy/Nodes/UNETLoader.md | 41 ++ docs/md/Comfy/Nodes/UpscaleModelLoader.md | 44 ++ docs/md/Comfy/Nodes/VAEDecode.md | 42 ++ docs/md/Comfy/Nodes/VAEDecodeTiled.md | 48 ++ docs/md/Comfy/Nodes/VAEEncode.md | 43 ++ docs/md/Comfy/Nodes/VAEEncodeForInpaint.md | 77 +++ docs/md/Comfy/Nodes/VAEEncodeTiled.md | 49 ++ docs/md/Comfy/Nodes/VAELoader.md | 91 +++ docs/md/Comfy/Nodes/VAESave.md | 61 +++ docs/md/Comfy/Nodes/VPScheduler.md | 57 ++ docs/md/Comfy/Nodes/VideoLinearCFGGuidance.md | 54 ++ docs/md/Comfy/Nodes/unCLIPCheckpointLoader.md | 53 ++ docs/md/Comfy/Nodes/unCLIPConditioning.md | 67 +++ docs/md/Comfy/index.md | 3 + .../ComfyMath/Nodes/CM_BoolBinaryOperation.md | 52 ++ docs/md/ComfyMath/Nodes/CM_BoolToInt.md | 38 ++ .../ComfyMath/Nodes/CM_BoolUnaryOperation.md | 44 ++ docs/md/ComfyMath/Nodes/CM_BreakoutVec2.md | 38 ++ docs/md/ComfyMath/Nodes/CM_BreakoutVec3.md | 38 ++ docs/md/ComfyMath/Nodes/CM_BreakoutVec4.md | 38 ++ docs/md/ComfyMath/Nodes/CM_ComposeVec2.md | 47 ++ docs/md/ComfyMath/Nodes/CM_ComposeVec3.md | 52 ++ docs/md/ComfyMath/Nodes/CM_ComposeVec4.md | 57 ++ .../Nodes/CM_FloatBinaryCondition.md | 52 ++ .../Nodes/CM_FloatBinaryOperation.md | 52 ++ docs/md/ComfyMath/Nodes/CM_FloatToInt.md | 38 ++ docs/md/ComfyMath/Nodes/CM_FloatToNumber.md | 38 ++ .../ComfyMath/Nodes/CM_FloatUnaryCondition.md | 47 ++ .../ComfyMath/Nodes/CM_FloatUnaryOperation.md | 47 ++ .../ComfyMath/Nodes/CM_IntBinaryCondition.md | 52 ++ .../ComfyMath/Nodes/CM_IntBinaryOperation.md | 52 ++ docs/md/ComfyMath/Nodes/CM_IntToBool.md | 38 ++ docs/md/ComfyMath/Nodes/CM_IntToFloat.md | 38 ++ docs/md/ComfyMath/Nodes/CM_IntToNumber.md | 38 ++ .../ComfyMath/Nodes/CM_IntUnaryCondition.md | 44 ++ .../ComfyMath/Nodes/CM_IntUnaryOperation.md | 44 ++ .../Nodes/CM_NearestSDXLResolution.md | 61 +++ .../Nodes/CM_NumberBinaryCondition.md | 52 ++ .../Nodes/CM_NumberBinaryOperation.md | 52 ++ docs/md/ComfyMath/Nodes/CM_NumberToFloat.md | 38 ++ docs/md/ComfyMath/Nodes/CM_NumberToInt.md | 38 ++ .../Nodes/CM_NumberUnaryCondition.md | 47 ++ .../Nodes/CM_NumberUnaryOperation.md | 47 ++ docs/md/ComfyMath/Nodes/CM_SDXLResolution.md | 46 ++ .../ComfyMath/Nodes/CM_Vec2BinaryCondition.md | 52 ++ .../ComfyMath/Nodes/CM_Vec2BinaryOperation.md | 54 ++ .../ComfyMath/Nodes/CM_Vec2ScalarOperation.md | 52 ++ .../Nodes/CM_Vec2ToScalarBinaryOperation.md | 52 ++ .../Nodes/CM_Vec2ToScalarUnaryOperation.md | 47 ++ .../ComfyMath/Nodes/CM_Vec2UnaryCondition.md | 47 ++ .../ComfyMath/Nodes/CM_Vec2UnaryOperation.md | 47 ++ .../ComfyMath/Nodes/CM_Vec3BinaryCondition.md | 52 ++ .../ComfyMath/Nodes/CM_Vec3BinaryOperation.md | 54 ++ .../ComfyMath/Nodes/CM_Vec3ScalarOperation.md | 52 ++ .../Nodes/CM_Vec3ToScalarBinaryOperation.md | 52 ++ .../Nodes/CM_Vec3ToScalarUnaryOperation.md | 47 ++ .../ComfyMath/Nodes/CM_Vec3UnaryCondition.md | 47 ++ .../ComfyMath/Nodes/CM_Vec3UnaryOperation.md | 47 ++ .../ComfyMath/Nodes/CM_Vec4BinaryCondition.md | 52 ++ .../ComfyMath/Nodes/CM_Vec4BinaryOperation.md | 54 ++ .../ComfyMath/Nodes/CM_Vec4ScalarOperation.md | 52 ++ .../Nodes/CM_Vec4ToScalarBinaryOperation.md | 52 ++ .../Nodes/CM_Vec4ToScalarUnaryOperation.md | 47 ++ .../ComfyMath/Nodes/CM_Vec4UnaryCondition.md | 47 ++ .../ComfyMath/Nodes/CM_Vec4UnaryOperation.md | 47 ++ docs/md/ComfyMath/index.md | 3 + .../Nodes/BNK_AddCLIPSDXLParams.md | 81 +++ .../Nodes/BNK_AddCLIPSDXLRParams.md | 63 +++ .../Nodes/BNK_CLIPTextEncodeAdvanced.md | 58 ++ .../Nodes/BNK_CLIPTextEncodeSDXLAdvanced.md | 68 +++ docs/md/ComfyUI_ADV_CLIP_emb/index.md | 3 + .../Nodes/IPAdapterApply.md | 242 ++++++++ .../Nodes/IPAdapterApplyEncoded.md | 78 +++ .../Nodes/IPAdapterApplyFaceID.md | 103 ++++ .../Nodes/IPAdapterBatchEmbeds.md | 45 ++ .../Nodes/IPAdapterEncoder.md | 140 +++++ .../Nodes/IPAdapterLoadEmbeds.md | 43 ++ .../Nodes/IPAdapterModelLoader.md | 54 ++ .../Nodes/IPAdapterSaveEmbeds.md | 52 ++ .../Nodes/InsightFaceLoader.md | 54 ++ .../Nodes/PrepImageForClipVision.md | 59 ++ .../Nodes/PrepImageForInsightFace.md | 65 +++ docs/md/ComfyUI_IPAdapter_plus/index.md | 3 + docs/md/SavePNG/Nodes/PNG_Save.md | 104 ++++ docs/md/SavePNG/index.md | 3 + docs/md/WAS_Extras/Nodes/BLVAEEncode.md | 155 ++++++ .../md/WAS_Extras/Nodes/CLIPTextEncodeList.md | 81 +++ .../Nodes/CLIPTextEncodeSequence2.md | 120 ++++ docs/md/WAS_Extras/Nodes/ConditioningBlend.md | 79 +++ docs/md/WAS_Extras/Nodes/DebugInput.md | 50 ++ docs/md/WAS_Extras/Nodes/KSamplerSeq.md | 260 +++++++++ docs/md/WAS_Extras/Nodes/KSamplerSeq2.md | 323 +++++++++++ .../Nodes/VAEEncodeForInpaint (WAS).md | 86 +++ docs/md/WAS_Extras/Nodes/VividSharpen.md | 68 +++ docs/md/WAS_Extras/index.md | 3 + ...ndingDinoModelLoader (segment anything).md | 42 ++ ...undingDinoSAMSegment (segment anything).md | 94 ++++ .../Nodes/InvertMask (segment anything).md | 42 ++ .../Nodes/IsMaskEmpty.md | 43 ++ .../SAMModelLoader (segment anything).md | 42 ++ docs/md/comfyui_segment_anything/index.md | 3 + docs/md/index.md | 4 + docs/md/llmnodes/Nodes/DocumentNode.md | 74 +++ docs/md/llmnodes/Nodes/DocumentPack.md | 225 ++++++++ docs/md/llmnodes/Nodes/LoadOpenAIModel.md | 58 ++ docs/md/llmnodes/index.md | 3 + .../Nodes/PlaiFlowPlayerAppFooter.md | 50 ++ .../Nodes/PlaiFlowPlayerAppInfo.md | 51 ++ .../Nodes/PlaiFlowPlayerInput.md | 191 +++++++ .../Nodes/PlaiFlowPlayerModelList.md | 48 ++ .../Nodes/PlaiFlowPlayerOutput.md | 94 ++++ .../Nodes/PlaiFlowPlayerSAMSplit.md | 52 ++ docs/md/plaiflowplayer_blocks/index.md | 3 + mkdocs.yml | 517 +++++++++--------- 246 files changed, 15741 insertions(+), 248 deletions(-) create mode 100644 docs/md/Comfy/Nodes/BasicScheduler.md create mode 100644 docs/md/Comfy/Nodes/CLIPLoader.md create mode 100644 docs/md/Comfy/Nodes/CLIPMergeSimple.md create mode 100644 docs/md/Comfy/Nodes/CLIPSave.md create mode 100644 docs/md/Comfy/Nodes/CLIPSetLastLayer.md create mode 100644 docs/md/Comfy/Nodes/CLIPTextEncode.md create mode 100644 docs/md/Comfy/Nodes/CLIPTextEncodeControlnet.md create mode 100644 docs/md/Comfy/Nodes/CLIPTextEncodeSDXL.md create mode 100644 docs/md/Comfy/Nodes/CLIPTextEncodeSDXLRefiner.md create mode 100644 docs/md/Comfy/Nodes/CLIPVisionEncode.md create mode 100644 docs/md/Comfy/Nodes/CLIPVisionLoader.md create mode 100644 docs/md/Comfy/Nodes/Canny.md create mode 100644 docs/md/Comfy/Nodes/CheckpointLoader.md create mode 100644 docs/md/Comfy/Nodes/CheckpointLoaderSimple.md create mode 100644 docs/md/Comfy/Nodes/CheckpointSave.md create mode 100644 docs/md/Comfy/Nodes/ConditioningAverage.md create mode 100644 docs/md/Comfy/Nodes/ConditioningCombine.md create mode 100644 docs/md/Comfy/Nodes/ConditioningConcat.md create mode 100644 docs/md/Comfy/Nodes/ConditioningSetArea.md create mode 100644 docs/md/Comfy/Nodes/ConditioningSetAreaPercentage.md create mode 100644 docs/md/Comfy/Nodes/ConditioningSetAreaStrength.md create mode 100644 docs/md/Comfy/Nodes/ConditioningSetMask.md create mode 100644 docs/md/Comfy/Nodes/ConditioningSetTimestepRange.md create mode 100644 docs/md/Comfy/Nodes/ConditioningZeroOut.md create mode 100644 docs/md/Comfy/Nodes/ControlNetApply.md create mode 100644 docs/md/Comfy/Nodes/ControlNetApplyAdvanced.md create mode 100644 docs/md/Comfy/Nodes/ControlNetLoader.md create mode 100644 docs/md/Comfy/Nodes/CropMask.md create mode 100644 docs/md/Comfy/Nodes/DiffControlNetLoader.md create mode 100644 docs/md/Comfy/Nodes/DiffusersLoader.md create mode 100644 docs/md/Comfy/Nodes/DualCLIPLoader.md create mode 100644 docs/md/Comfy/Nodes/EmptyImage.md create mode 100644 docs/md/Comfy/Nodes/EmptyLatentImage.md create mode 100644 docs/md/Comfy/Nodes/ExponentialScheduler.md create mode 100644 docs/md/Comfy/Nodes/FeatherMask.md create mode 100644 docs/md/Comfy/Nodes/FlipSigmas.md create mode 100644 docs/md/Comfy/Nodes/FreeU.md create mode 100644 docs/md/Comfy/Nodes/FreeU_V2.md create mode 100644 docs/md/Comfy/Nodes/GLIGENLoader.md create mode 100644 docs/md/Comfy/Nodes/GLIGENTextBoxApply.md create mode 100644 docs/md/Comfy/Nodes/GrowMask.md create mode 100644 docs/md/Comfy/Nodes/HyperTile.md create mode 100644 docs/md/Comfy/Nodes/HypernetworkLoader.md create mode 100644 docs/md/Comfy/Nodes/ImageBatch.md create mode 100644 docs/md/Comfy/Nodes/ImageBlend.md create mode 100644 docs/md/Comfy/Nodes/ImageBlur.md create mode 100644 docs/md/Comfy/Nodes/ImageColorToMask.md create mode 100644 docs/md/Comfy/Nodes/ImageCompositeMasked.md create mode 100644 docs/md/Comfy/Nodes/ImageCrop.md create mode 100644 docs/md/Comfy/Nodes/ImageFromBatch.md create mode 100644 docs/md/Comfy/Nodes/ImageInvert.md create mode 100644 docs/md/Comfy/Nodes/ImageOnlyCheckpointLoader.md create mode 100644 docs/md/Comfy/Nodes/ImageOnlyCheckpointSave.md create mode 100644 docs/md/Comfy/Nodes/ImagePadForOutpaint.md create mode 100644 docs/md/Comfy/Nodes/ImageQuantize.md create mode 100644 docs/md/Comfy/Nodes/ImageScale.md create mode 100644 docs/md/Comfy/Nodes/ImageScaleBy.md create mode 100644 docs/md/Comfy/Nodes/ImageScaleToTotalPixels.md create mode 100644 docs/md/Comfy/Nodes/ImageSharpen.md create mode 100644 docs/md/Comfy/Nodes/ImageToMask.md create mode 100644 docs/md/Comfy/Nodes/ImageUpscaleWithModel.md create mode 100644 docs/md/Comfy/Nodes/InpaintModelConditioning.md create mode 100644 docs/md/Comfy/Nodes/InvertMask.md create mode 100644 docs/md/Comfy/Nodes/JoinImageWithAlpha.md create mode 100644 docs/md/Comfy/Nodes/KSampler.md create mode 100644 docs/md/Comfy/Nodes/KSamplerAdvanced.md create mode 100644 docs/md/Comfy/Nodes/KSamplerSelect.md create mode 100644 docs/md/Comfy/Nodes/KarrasScheduler.md create mode 100644 docs/md/Comfy/Nodes/LatentAdd.md create mode 100644 docs/md/Comfy/Nodes/LatentBatch.md create mode 100644 docs/md/Comfy/Nodes/LatentBatchSeedBehavior.md create mode 100644 docs/md/Comfy/Nodes/LatentBlend.md create mode 100644 docs/md/Comfy/Nodes/LatentComposite.md create mode 100644 docs/md/Comfy/Nodes/LatentCompositeMasked.md create mode 100644 docs/md/Comfy/Nodes/LatentCrop.md create mode 100644 docs/md/Comfy/Nodes/LatentFlip.md create mode 100644 docs/md/Comfy/Nodes/LatentFromBatch.md create mode 100644 docs/md/Comfy/Nodes/LatentInterpolate.md create mode 100644 docs/md/Comfy/Nodes/LatentMultiply.md create mode 100644 docs/md/Comfy/Nodes/LatentRotate.md create mode 100644 docs/md/Comfy/Nodes/LatentSubtract.md create mode 100644 docs/md/Comfy/Nodes/LatentUpscale.md create mode 100644 docs/md/Comfy/Nodes/LatentUpscaleBy.md create mode 100644 docs/md/Comfy/Nodes/LoadImage.md create mode 100644 docs/md/Comfy/Nodes/LoadImageMask.md create mode 100644 docs/md/Comfy/Nodes/LoadLatent.md create mode 100644 docs/md/Comfy/Nodes/LoraLoader.md create mode 100644 docs/md/Comfy/Nodes/LoraLoaderModelOnly.md create mode 100644 docs/md/Comfy/Nodes/MaskComposite.md create mode 100644 docs/md/Comfy/Nodes/MaskToImage.md create mode 100644 docs/md/Comfy/Nodes/ModelMergeAdd.md create mode 100644 docs/md/Comfy/Nodes/ModelMergeBlocks.md create mode 100644 docs/md/Comfy/Nodes/ModelMergeSimple.md create mode 100644 docs/md/Comfy/Nodes/ModelMergeSubtract.md create mode 100644 docs/md/Comfy/Nodes/ModelSamplingContinuousEDM.md create mode 100644 docs/md/Comfy/Nodes/ModelSamplingDiscrete.md create mode 100644 docs/md/Comfy/Nodes/ModelSamplingStableCascade.md create mode 100644 docs/md/Comfy/Nodes/PatchModelAddDownscale.md create mode 100644 docs/md/Comfy/Nodes/PerpNeg.md create mode 100644 docs/md/Comfy/Nodes/PhotoMakerEncode.md create mode 100644 docs/md/Comfy/Nodes/PhotoMakerLoader.md create mode 100644 docs/md/Comfy/Nodes/PolyexponentialScheduler.md create mode 100644 docs/md/Comfy/Nodes/PorterDuffImageComposite.md create mode 100644 docs/md/Comfy/Nodes/PreviewImage.md create mode 100644 docs/md/Comfy/Nodes/RebatchImages.md create mode 100644 docs/md/Comfy/Nodes/RebatchLatents.md create mode 100644 docs/md/Comfy/Nodes/RepeatImageBatch.md create mode 100644 docs/md/Comfy/Nodes/RepeatLatentBatch.md create mode 100644 docs/md/Comfy/Nodes/RescaleCFG.md create mode 100644 docs/md/Comfy/Nodes/SDTurboScheduler.md create mode 100644 docs/md/Comfy/Nodes/SD_4XUpscale_Conditioning.md create mode 100644 docs/md/Comfy/Nodes/SVD_img2vid_Conditioning.md create mode 100644 docs/md/Comfy/Nodes/SamplerCustom.md create mode 100644 docs/md/Comfy/Nodes/SamplerDPMPP_2M_SDE.md create mode 100644 docs/md/Comfy/Nodes/SamplerDPMPP_SDE.md create mode 100644 docs/md/Comfy/Nodes/SaveAnimatedPNG.md create mode 100644 docs/md/Comfy/Nodes/SaveAnimatedWEBP.md create mode 100644 docs/md/Comfy/Nodes/SaveImage.md create mode 100644 docs/md/Comfy/Nodes/SaveLatent.md create mode 100644 docs/md/Comfy/Nodes/SelfAttentionGuidance.md create mode 100644 docs/md/Comfy/Nodes/SetLatentNoiseMask.md create mode 100644 docs/md/Comfy/Nodes/SolidMask.md create mode 100644 docs/md/Comfy/Nodes/SplitImageWithAlpha.md create mode 100644 docs/md/Comfy/Nodes/SplitSigmas.md create mode 100644 docs/md/Comfy/Nodes/StableCascade_EmptyLatentImage.md create mode 100644 docs/md/Comfy/Nodes/StableCascade_StageB_Conditioning.md create mode 100644 docs/md/Comfy/Nodes/StableCascade_StageC_VAEEncode.md create mode 100644 docs/md/Comfy/Nodes/StableZero123_Conditioning.md create mode 100644 docs/md/Comfy/Nodes/StableZero123_Conditioning_Batched.md create mode 100644 docs/md/Comfy/Nodes/StyleModelApply.md create mode 100644 docs/md/Comfy/Nodes/StyleModelLoader.md create mode 100644 docs/md/Comfy/Nodes/TomePatchModel.md create mode 100644 docs/md/Comfy/Nodes/UNETLoader.md create mode 100644 docs/md/Comfy/Nodes/UpscaleModelLoader.md create mode 100644 docs/md/Comfy/Nodes/VAEDecode.md create mode 100644 docs/md/Comfy/Nodes/VAEDecodeTiled.md create mode 100644 docs/md/Comfy/Nodes/VAEEncode.md create mode 100644 docs/md/Comfy/Nodes/VAEEncodeForInpaint.md create mode 100644 docs/md/Comfy/Nodes/VAEEncodeTiled.md create mode 100644 docs/md/Comfy/Nodes/VAELoader.md create mode 100644 docs/md/Comfy/Nodes/VAESave.md create mode 100644 docs/md/Comfy/Nodes/VPScheduler.md create mode 100644 docs/md/Comfy/Nodes/VideoLinearCFGGuidance.md create mode 100644 docs/md/Comfy/Nodes/unCLIPCheckpointLoader.md create mode 100644 docs/md/Comfy/Nodes/unCLIPConditioning.md create mode 100644 docs/md/Comfy/index.md create mode 100644 docs/md/ComfyMath/Nodes/CM_BoolBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_BoolToInt.md create mode 100644 docs/md/ComfyMath/Nodes/CM_BoolUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_BreakoutVec2.md create mode 100644 docs/md/ComfyMath/Nodes/CM_BreakoutVec3.md create mode 100644 docs/md/ComfyMath/Nodes/CM_BreakoutVec4.md create mode 100644 docs/md/ComfyMath/Nodes/CM_ComposeVec2.md create mode 100644 docs/md/ComfyMath/Nodes/CM_ComposeVec3.md create mode 100644 docs/md/ComfyMath/Nodes/CM_ComposeVec4.md create mode 100644 docs/md/ComfyMath/Nodes/CM_FloatBinaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_FloatBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_FloatToInt.md create mode 100644 docs/md/ComfyMath/Nodes/CM_FloatToNumber.md create mode 100644 docs/md/ComfyMath/Nodes/CM_FloatUnaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_FloatUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntBinaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntToBool.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntToFloat.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntToNumber.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntUnaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_IntUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NearestSDXLResolution.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NumberBinaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NumberBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NumberToFloat.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NumberToInt.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NumberUnaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_NumberUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_SDXLResolution.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2BinaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2BinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2ScalarOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2ToScalarBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2ToScalarUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2UnaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec2UnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3BinaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3BinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3ScalarOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3ToScalarBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3ToScalarUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3UnaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec3UnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4BinaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4BinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4ScalarOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4ToScalarBinaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4ToScalarUnaryOperation.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4UnaryCondition.md create mode 100644 docs/md/ComfyMath/Nodes/CM_Vec4UnaryOperation.md create mode 100644 docs/md/ComfyMath/index.md create mode 100644 docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLParams.md create mode 100644 docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLRParams.md create mode 100644 docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeAdvanced.md create mode 100644 docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeSDXLAdvanced.md create mode 100644 docs/md/ComfyUI_ADV_CLIP_emb/index.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApply.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyEncoded.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyFaceID.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterBatchEmbeds.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterEncoder.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterLoadEmbeds.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterModelLoader.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterSaveEmbeds.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/InsightFaceLoader.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForClipVision.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForInsightFace.md create mode 100644 docs/md/ComfyUI_IPAdapter_plus/index.md create mode 100644 docs/md/SavePNG/Nodes/PNG_Save.md create mode 100644 docs/md/SavePNG/index.md create mode 100644 docs/md/WAS_Extras/Nodes/BLVAEEncode.md create mode 100644 docs/md/WAS_Extras/Nodes/CLIPTextEncodeList.md create mode 100644 docs/md/WAS_Extras/Nodes/CLIPTextEncodeSequence2.md create mode 100644 docs/md/WAS_Extras/Nodes/ConditioningBlend.md create mode 100644 docs/md/WAS_Extras/Nodes/DebugInput.md create mode 100644 docs/md/WAS_Extras/Nodes/KSamplerSeq.md create mode 100644 docs/md/WAS_Extras/Nodes/KSamplerSeq2.md create mode 100644 docs/md/WAS_Extras/Nodes/VAEEncodeForInpaint (WAS).md create mode 100644 docs/md/WAS_Extras/Nodes/VividSharpen.md create mode 100644 docs/md/WAS_Extras/index.md create mode 100644 docs/md/comfyui_segment_anything/Nodes/GroundingDinoModelLoader (segment anything).md create mode 100644 docs/md/comfyui_segment_anything/Nodes/GroundingDinoSAMSegment (segment anything).md create mode 100644 docs/md/comfyui_segment_anything/Nodes/InvertMask (segment anything).md create mode 100644 docs/md/comfyui_segment_anything/Nodes/IsMaskEmpty.md create mode 100644 docs/md/comfyui_segment_anything/Nodes/SAMModelLoader (segment anything).md create mode 100644 docs/md/comfyui_segment_anything/index.md create mode 100644 docs/md/index.md create mode 100644 docs/md/llmnodes/Nodes/DocumentNode.md create mode 100644 docs/md/llmnodes/Nodes/DocumentPack.md create mode 100644 docs/md/llmnodes/Nodes/LoadOpenAIModel.md create mode 100644 docs/md/llmnodes/index.md create mode 100644 docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppFooter.md create mode 100644 docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppInfo.md create mode 100644 docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerInput.md create mode 100644 docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerModelList.md create mode 100644 docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerOutput.md create mode 100644 docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerSAMSplit.md create mode 100644 docs/md/plaiflowplayer_blocks/index.md diff --git a/docs/md/Comfy/Nodes/BasicScheduler.md b/docs/md/Comfy/Nodes/BasicScheduler.md new file mode 100644 index 0000000000..72ad130b1d --- /dev/null +++ b/docs/md/Comfy/Nodes/BasicScheduler.md @@ -0,0 +1,63 @@ +# BasicScheduler +## Documentation +- Class name: `BasicScheduler` +- Category: `sampling/custom_sampling/schedulers` +- Output node: `False` + +The BasicScheduler node calculates a sequence of sigma values for diffusion models based on the provided scheduler, model, steps, and denoise parameters. It adjusts the total number of steps based on the denoise factor and utilizes the model's scheduler to generate the sigma values. +## Input types +### Required +- **`model`** + - The model parameter specifies the diffusion model for which the sigma values are to be calculated. It is crucial for determining the behavior of the diffusion process. + - Python dtype: `comfy.models.DiffusionModel` + - Comfy dtype: `MODEL` +- **`scheduler`** + - The scheduler parameter determines the method used to calculate the sigma values. It affects the diffusion process by altering the noise levels at each step. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`steps`** + - Specifies the number of diffusion steps. It directly influences the granularity of the diffusion process. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`denoise`** + - A factor that adjusts the total number of steps based on its value, affecting the smoothness of the diffusion process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`sigmas`** + - A sequence of sigma values calculated for the diffusion model. These values are essential for controlling the noise level throughout the diffusion process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: `SamplerCustom,SplitSigmas,Reroute` + + +## Source code +```python +class BasicScheduler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "scheduler": (comfy.samplers.SCHEDULER_NAMES, ), + "steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/schedulers" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, model, scheduler, steps, denoise): + total_steps = steps + if denoise < 1.0: + total_steps = int(steps/denoise) + + comfy.model_management.load_models_gpu([model]) + sigmas = comfy.samplers.calculate_sigmas_scheduler(model.model, scheduler, total_steps).cpu() + sigmas = sigmas[-(steps + 1):] + return (sigmas, ) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPLoader.md b/docs/md/Comfy/Nodes/CLIPLoader.md new file mode 100644 index 0000000000..46766cf063 --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPLoader.md @@ -0,0 +1,50 @@ +# Load CLIP +## Documentation +- Class name: `CLIPLoader` +- Category: `advanced/loaders` +- Output node: `False` + +The CLIPLoader node is responsible for loading a CLIP model based on the specified name and type. It supports loading different types of CLIP models, such as 'stable_diffusion' and 'stable_cascade', by fetching the model from a specified path and applying the appropriate CLIP type configuration. +## Input types +### Required +- **`clip_name`** + - Specifies the name of the CLIP model to be loaded. This name is used to locate the model file in a predefined directory structure. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`type`** + - Determines the type of CLIP model to load, allowing for different configurations such as 'stable_diffusion' or 'stable_cascade'. This affects how the model is initialized and configured. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`clip`** + - The loaded CLIP model, ready for use in further processing or analysis. + - Python dtype: `comfy.sd.CLIPModel` + - Comfy dtype: `CLIP` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class CLIPLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_name": (folder_paths.get_filename_list("clip"), ), + "type": (["stable_diffusion", "stable_cascade"], ), + }} + RETURN_TYPES = ("CLIP",) + FUNCTION = "load_clip" + + CATEGORY = "advanced/loaders" + + def load_clip(self, clip_name, type="stable_diffusion"): + clip_type = comfy.sd.CLIPType.STABLE_DIFFUSION + if type == "stable_cascade": + clip_type = comfy.sd.CLIPType.STABLE_CASCADE + + clip_path = folder_paths.get_full_path("clip", clip_name) + clip = comfy.sd.load_clip(ckpt_paths=[clip_path], embedding_directory=folder_paths.get_folder_paths("embeddings"), clip_type=clip_type) + return (clip,) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPMergeSimple.md b/docs/md/Comfy/Nodes/CLIPMergeSimple.md new file mode 100644 index 0000000000..9573912172 --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPMergeSimple.md @@ -0,0 +1,55 @@ +# CLIPMergeSimple +## Documentation +- Class name: `CLIPMergeSimple` +- Category: `advanced/model_merging` +- Output node: `False` + +The `CLIPMergeSimple` node merges two CLIP models based on a specified ratio, excluding specific parameters related to position IDs and logit scale. This operation allows for the combination of features from both models, potentially enhancing their capabilities or creating a new model with a balanced set of characteristics from the input models. +## Input types +### Required +- **`clip1`** + - The first CLIP model to be merged. It serves as the base model for the merging process. + - Python dtype: `comfy.model_base.CLIPModel` + - Comfy dtype: `CLIP` +- **`clip2`** + - The second CLIP model to be merged. Its key patches, except for those related to position IDs and logit scale, are added to the first model based on the specified ratio. + - Python dtype: `comfy.model_base.CLIPModel` + - Comfy dtype: `CLIP` +- **`ratio`** + - Determines the proportion of features from the second model to be merged into the first model. A ratio of 1 means fully adopting the second model's features, while 0 means no adoption. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`clip`** + - The resulting CLIP model after merging the specified models according to the given ratio. + - Python dtype: `comfy.model_base.CLIPModel` + - Comfy dtype: `CLIP` +## Usage tips +- Infra type: `GPU` +- Common nodes: `CR Apply LoRA Stack,CLIPSetLastLayer` + + +## Source code +```python +class CLIPMergeSimple: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip1": ("CLIP",), + "clip2": ("CLIP",), + "ratio": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + RETURN_TYPES = ("CLIP",) + FUNCTION = "merge" + + CATEGORY = "advanced/model_merging" + + def merge(self, clip1, clip2, ratio): + m = clip1.clone() + kp = clip2.get_key_patches() + for k in kp: + if k.endswith(".position_ids") or k.endswith(".logit_scale"): + continue + m.add_patches({k: kp[k]}, 1.0 - ratio, ratio) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPSave.md b/docs/md/Comfy/Nodes/CLIPSave.md new file mode 100644 index 0000000000..6b24c58c2a --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPSave.md @@ -0,0 +1,83 @@ +# CLIPSave +## Documentation +- Class name: `CLIPSave` +- Category: `advanced/model_merging` +- Output node: `True` + +The `CLIPSave` node is designed for saving CLIP models along with additional information such as prompts and extra PNG metadata. It encapsulates the functionality to serialize and store the model's state, making it easier to manage and reuse models across different projects or experiments. +## Input types +### Required +- **`clip`** + - The CLIP model to be saved. This parameter is crucial as it represents the model whose state is being serialized for future use. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`filename_prefix`** + - A prefix for the filename under which the model and its associated data will be saved. This allows for organized storage and easy retrieval of saved models. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class CLIPSave: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip": ("CLIP",), + "filename_prefix": ("STRING", {"default": "clip/ComfyUI"}),}, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},} + RETURN_TYPES = () + FUNCTION = "save" + OUTPUT_NODE = True + + CATEGORY = "advanced/model_merging" + + def save(self, clip, filename_prefix, prompt=None, extra_pnginfo=None): + prompt_info = "" + if prompt is not None: + prompt_info = json.dumps(prompt) + + metadata = {} + if not args.disable_metadata: + metadata["prompt"] = prompt_info + if extra_pnginfo is not None: + for x in extra_pnginfo: + metadata[x] = json.dumps(extra_pnginfo[x]) + + comfy.model_management.load_models_gpu([clip.load_model()]) + clip_sd = clip.get_sd() + + for prefix in ["clip_l.", "clip_g.", ""]: + k = list(filter(lambda a: a.startswith(prefix), clip_sd.keys())) + current_clip_sd = {} + for x in k: + current_clip_sd[x] = clip_sd.pop(x) + if len(current_clip_sd) == 0: + continue + + p = prefix[:-1] + replace_prefix = {} + filename_prefix_ = filename_prefix + if len(p) > 0: + filename_prefix_ = "{}_{}".format(filename_prefix_, p) + replace_prefix[prefix] = "" + replace_prefix["transformer."] = "" + + full_output_folder, filename, counter, subfolder, filename_prefix_ = folder_paths.get_save_image_path(filename_prefix_, self.output_dir) + + output_checkpoint = f"{filename}_{counter:05}_.safetensors" + output_checkpoint = os.path.join(full_output_folder, output_checkpoint) + + current_clip_sd = comfy.utils.state_dict_prefix_replace(current_clip_sd, replace_prefix) + + comfy.utils.save_torch_file(current_clip_sd, output_checkpoint, metadata=metadata) + return {} + +``` diff --git a/docs/md/Comfy/Nodes/CLIPSetLastLayer.md b/docs/md/Comfy/Nodes/CLIPSetLastLayer.md new file mode 100644 index 0000000000..d83578f2ac --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPSetLastLayer.md @@ -0,0 +1,46 @@ +# CLIP Set Last Layer +## Documentation +- Class name: `CLIPSetLastLayer` +- Category: `conditioning` +- Output node: `False` + +The `CLIPSetLastLayer` node is designed to modify a CLIP model by setting its last layer to stop at a specified layer index. This operation is useful for controlling the depth of the model's processing, potentially affecting the model's performance and the characteristics of the generated embeddings. +## Input types +### Required +- **`clip`** + - The `clip` parameter represents the CLIP model to be modified. It is crucial for defining the model on which the layer adjustment will be performed. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`stop_at_clip_layer`** + - The `stop_at_clip_layer` parameter specifies the index of the last layer to be used in the CLIP model. This allows for fine-tuning the depth of the model's processing, which can influence the model's output and performance. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`clip`** + - Returns the modified CLIP model with the last layer set to the specified index. This adjusted model can then be used for further processing or embedding generation. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +## Usage tips +- Infra type: `GPU` +- Common nodes: `CLIPTextEncode,LoraLoader,CR Apply LoRA Stack,Text to Conditioning,Reroute,PromptControlSimple,BatchPromptSchedule,CLIPTextEncodeA1111,FaceDetailer,BNK_CutoffBasePrompt` + + +## Source code +```python +class CLIPSetLastLayer: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip": ("CLIP", ), + "stop_at_clip_layer": ("INT", {"default": -1, "min": -24, "max": -1, "step": 1}), + }} + RETURN_TYPES = ("CLIP",) + FUNCTION = "set_last_layer" + + CATEGORY = "conditioning" + + def set_last_layer(self, clip, stop_at_clip_layer): + clip = clip.clone() + clip.clip_layer(stop_at_clip_layer) + return (clip,) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPTextEncode.md b/docs/md/Comfy/Nodes/CLIPTextEncode.md new file mode 100644 index 0000000000..3c763a89fd --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPTextEncode.md @@ -0,0 +1,44 @@ +# CLIP Text Encode (Prompt) +## Documentation +- Class name: `CLIPTextEncode` +- Category: `conditioning` +- Output node: `False` + +The `CLIPTextEncode` node encodes text inputs using a CLIP model to produce conditioning information. It tokenizes the input text and then encodes these tokens to generate a conditioning vector and a pooled output, which are used for further processing or generation tasks. +## Input types +### Required +- **`text`** + - The text input to be encoded. This is tokenized and encoded to produce the conditioning information. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`clip`** + - The CLIP model used for text tokenization and encoding. It plays a crucial role in transforming the input text into a format suitable for generating conditioning information. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +## Output types +- **`conditioning`** + - The output conditioning information, consisting of a conditioning vector and a pooled output, derived from the encoded text. This information is crucial for guiding the generation process in tasks such as image synthesis. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,ControlNetApplyAdvanced,KSampler //Inspire,SamplerCustom,Reroute,KSamplerAdvanced,ACN_AdvancedControlNetApply,ToBasicPipe,FaceDetailer` + + +## Source code +```python +class CLIPTextEncode: + @classmethod + def INPUT_TYPES(s): + return {"required": {"text": ("STRING", {"multiline": True}), "clip": ("CLIP", )}} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "conditioning" + + def encode(self, clip, text): + tokens = clip.tokenize(text) + cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True) + return ([[cond, {"pooled_output": pooled}]], ) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPTextEncodeControlnet.md b/docs/md/Comfy/Nodes/CLIPTextEncodeControlnet.md new file mode 100644 index 0000000000..75602a0d5b --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPTextEncodeControlnet.md @@ -0,0 +1,54 @@ +# CLIPTextEncodeControlnet +## Documentation +- Class name: `CLIPTextEncodeControlnet` +- Category: `_for_testing/conditioning` +- Output node: `False` + +The `CLIPTextEncodeControlnet` node is designed for encoding text inputs using a CLIP model, specifically for control network applications. It tokenizes the input text, encodes it to obtain both the standard and pooled representations, and then integrates these representations into the provided conditioning data, enhancing it with additional control information. +## Input types +### Required +- **`clip`** + - The CLIP model used for text tokenization and encoding. It's crucial for transforming the input text into a format that can be effectively utilized for further processing and analysis. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`conditioning`** + - A list of conditioning data to which the encoded text representations are added. This process enriches the conditioning with text-derived features, enabling more controlled and nuanced generation processes. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`text`** + - The input text to be encoded. This text is tokenized and encoded by the CLIP model, forming the basis for the control information added to the conditioning data. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`conditioning`** + - The enhanced conditioning data, now including the encoded text representations. This output is ready for use in controlled generation tasks, providing a richer set of features for the generation process. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class CLIPTextEncodeControlnet: + @classmethod + def INPUT_TYPES(s): + return {"required": {"clip": ("CLIP", ), "conditioning": ("CONDITIONING", ), "text": ("STRING", {"multiline": True})}} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "_for_testing/conditioning" + + def encode(self, clip, conditioning, text): + tokens = clip.tokenize(text) + cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True) + c = [] + for t in conditioning: + n = [t[0], t[1].copy()] + n[1]['cross_attn_controlnet'] = cond + n[1]['pooled_output_controlnet'] = pooled + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPTextEncodeSDXL.md b/docs/md/Comfy/Nodes/CLIPTextEncodeSDXL.md new file mode 100644 index 0000000000..89a90d1f92 --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPTextEncodeSDXL.md @@ -0,0 +1,88 @@ +# CLIPTextEncodeSDXL +## Documentation +- Class name: `CLIPTextEncodeSDXL` +- Category: `advanced/conditioning` +- Output node: `False` + +This node is not explicitly defined in the provided context, indicating it might be a part of a larger system or framework not fully detailed here. Given the naming convention and the context of other nodes, it can be inferred that CLIPTextEncodeSDXL likely involves encoding text inputs using a CLIP model specifically tailored or configured for the SDXL architecture. This process would involve transforming textual data into a format that is compatible with subsequent processing or analysis steps, particularly in tasks related to image-text interaction or multimodal learning. +## Input types +### Required +- **`width`** + - Specifies the width of the target image. This dimension is crucial for ensuring that the encoded text is appropriately scaled and aligned with the visual content it is intended to interact with or generate. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Similar to the width parameter, the height specifies the vertical dimension of the target image. It plays a key role in matching the encoded text's representation with the intended visual context, affecting the overall coherence of the generated or modified image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`crop_w`** + - Specifies the width of the crop area. This parameter is used to define a specific region of interest within the image that the text encoding should focus on. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`crop_h`** + - Specifies the height of the crop area. Similar to crop_w, it defines a vertical region of interest within the image for the text encoding process. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`target_width`** + - Defines the desired width of the output image after processing. This parameter can influence the scaling and aspect ratio of the final image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`target_height`** + - Defines the desired height of the output image after processing. It works alongside target_width to determine the final dimensions of the image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`text_g`** + - The global textual input to be encoded. This text is processed by the CLIP model to extract relevant features and semantic information for the entire image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`clip`** + - Represents the CLIP model used for encoding the text. This model is a crucial component of the node, transforming textual data into a format that can be effectively utilized in image-text interaction tasks. + - Python dtype: `object` + - Comfy dtype: `CLIP` +- **`text_l`** + - The local textual input to be encoded. This text provides additional details or focuses on specific areas within the image, complementing the global text input. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`conditioning`** + - The output is a conditioning format that combines the encoded text representation with additional information such as aesthetic score and image dimensions. This enriched representation is used to guide the generation or modification of images in a way that aligns with the encoded textual content. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,KSamplerAdvanced,ControlNetApplyAdvanced,Reroute,ToBasicPipe,KSampler (Efficient),Prompts Everywhere,ACN_AdvancedControlNetApply,FaceDetailer,KRestartSamplerAdv` + + +## Source code +```python +class CLIPTextEncodeSDXL: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "width": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "height": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "crop_w": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION}), + "crop_h": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION}), + "target_width": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "target_height": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "text_g": ("STRING", {"multiline": True, "default": "CLIP_G"}), "clip": ("CLIP", ), + "text_l": ("STRING", {"multiline": True, "default": "CLIP_L"}), "clip": ("CLIP", ), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "advanced/conditioning" + + def encode(self, clip, width, height, crop_w, crop_h, target_width, target_height, text_g, text_l): + tokens = clip.tokenize(text_g) + tokens["l"] = clip.tokenize(text_l)["l"] + if len(tokens["l"]) != len(tokens["g"]): + empty = clip.tokenize("") + while len(tokens["l"]) < len(tokens["g"]): + tokens["l"] += empty["l"] + while len(tokens["l"]) > len(tokens["g"]): + tokens["g"] += empty["g"] + cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True) + return ([[cond, {"pooled_output": pooled, "width": width, "height": height, "crop_w": crop_w, "crop_h": crop_h, "target_width": target_width, "target_height": target_height}]], ) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPTextEncodeSDXLRefiner.md b/docs/md/Comfy/Nodes/CLIPTextEncodeSDXLRefiner.md new file mode 100644 index 0000000000..45bbc87582 --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPTextEncodeSDXLRefiner.md @@ -0,0 +1,61 @@ +# CLIPTextEncodeSDXLRefiner +## Documentation +- Class name: `CLIPTextEncodeSDXLRefiner` +- Category: `advanced/conditioning` +- Output node: `False` + +This node refines the encoding of text inputs using a CLIP model, specifically designed for aesthetic scoring and image dimension adjustments. It tokenizes the input text, encodes it to generate conditioning information, and enriches this with aesthetic scores and image dimensions. +## Input types +### Required +- **`ascore`** + - The aesthetic score to be associated with the encoded text, influencing the visual appeal of generated images. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`width`** + - The desired width of the output image. This parameter helps in adjusting the dimensions of the generated image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - The desired height of the output image, used alongside width to define the image dimensions. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`text`** + - The text input to be encoded. This is the primary content that will be transformed into a conditioning vector. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`clip`** + - The CLIP model used for text tokenization and encoding. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +## Output types +- **`conditioning`** + - The output conditioning vector, enriched with aesthetic scores and image dimensions, ready for further processing in image generation pipelines. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Union[float, int]]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSamplerAdvanced,UltimateSDUpscale` + + +## Source code +```python +class CLIPTextEncodeSDXLRefiner: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "ascore": ("FLOAT", {"default": 6.0, "min": 0.0, "max": 1000.0, "step": 0.01}), + "width": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "height": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "text": ("STRING", {"multiline": True}), "clip": ("CLIP", ), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "advanced/conditioning" + + def encode(self, clip, ascore, width, height, text): + tokens = clip.tokenize(text) + cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True) + return ([[cond, {"pooled_output": pooled, "aesthetic_score": ascore, "width": width,"height": height}]], ) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPVisionEncode.md b/docs/md/Comfy/Nodes/CLIPVisionEncode.md new file mode 100644 index 0000000000..11c6d56a61 --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPVisionEncode.md @@ -0,0 +1,45 @@ +# CLIP Vision Encode +## Documentation +- Class name: `CLIPVisionEncode` +- Category: `conditioning` +- Output node: `False` + +The `CLIPVisionEncode` node is responsible for encoding an image using a CLIP vision model. It takes an image as input and processes it through the CLIP vision model's `encode_image` method to produce an output that represents the encoded features of the image. +## Input types +### Required +- **`clip_vision`** + - The CLIP vision model used for encoding the image. It is crucial for transforming the raw image data into a format that can be effectively utilized for further processing or analysis. + - Python dtype: `CLIPVision` + - Comfy dtype: `CLIP_VISION` +- **`image`** + - The image to be encoded. This input is transformed by the CLIP vision model into an encoded representation, capturing essential features and characteristics of the image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +- **`clip_vision_output`** + - The encoded representation of the image, as produced by the CLIP vision model. This output captures the essential features and characteristics of the image, suitable for further processing or analysis. + - Python dtype: `CLIPVisionOutput` + - Comfy dtype: `CLIP_VISION_OUTPUT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `unCLIPConditioning` + + +## Source code +```python +class CLIPVisionEncode: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_vision": ("CLIP_VISION",), + "image": ("IMAGE",) + }} + RETURN_TYPES = ("CLIP_VISION_OUTPUT",) + FUNCTION = "encode" + + CATEGORY = "conditioning" + + def encode(self, clip_vision, image): + output = clip_vision.encode_image(image) + return (output,) + +``` diff --git a/docs/md/Comfy/Nodes/CLIPVisionLoader.md b/docs/md/Comfy/Nodes/CLIPVisionLoader.md new file mode 100644 index 0000000000..67134feddf --- /dev/null +++ b/docs/md/Comfy/Nodes/CLIPVisionLoader.md @@ -0,0 +1,41 @@ +# Load CLIP Vision +## Documentation +- Class name: `CLIPVisionLoader` +- Category: `loaders` +- Output node: `False` + +The `CLIPVisionLoader` node is responsible for loading a CLIP Vision model from a specified path. It utilizes the `comfy.clip_vision.load` method to retrieve the model, which is then returned for further use. +## Input types +### Required +- **`clip_name`** + - The `clip_name` parameter specifies the name of the CLIP Vision model to be loaded. This name is used to locate the model file within a predefined directory structure. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`clip_vision`** + - This output represents the loaded CLIP Vision model, ready for use in subsequent operations such as encoding images. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP_VISION` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IPAdapterApply,IPAdapterApplyFaceID,IPAdapter,IPAdapterEncoder,CLIPVisionEncode,ToIPAdapterPipe //Inspire,Reroute,AV_IPAdapter` + + +## Source code +```python +class CLIPVisionLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_name": (folder_paths.get_filename_list("clip_vision"), ), + }} + RETURN_TYPES = ("CLIP_VISION",) + FUNCTION = "load_clip" + + CATEGORY = "loaders" + + def load_clip(self, clip_name): + clip_path = folder_paths.get_full_path("clip_vision", clip_name) + clip_vision = comfy.clip_vision.load(clip_path) + return (clip_vision,) + +``` diff --git a/docs/md/Comfy/Nodes/Canny.md b/docs/md/Comfy/Nodes/Canny.md new file mode 100644 index 0000000000..12af262dcd --- /dev/null +++ b/docs/md/Comfy/Nodes/Canny.md @@ -0,0 +1,52 @@ +# Canny +## Documentation +- Class name: `Canny` +- Category: `image/preprocessors` +- Output node: `False` + +The `Canny` node applies the Canny edge detection algorithm to an input image, utilizing thresholds to identify and filter edges. This process involves converting the image to grayscale, applying a Gaussian blur, computing gradients, and performing non-maximum suppression and hysteresis to accurately identify edges. +## Input types +### Required +- **`image`** + - The input image to which the Canny edge detection algorithm will be applied. It's crucial for identifying and filtering edges in the image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`low_threshold`** + - The lower threshold for the hysteresis procedure in edge detection. It helps in filtering out edges that are less likely to be part of the edge structure of the image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`high_threshold`** + - The upper threshold for the hysteresis procedure in edge detection. It is used to identify strong edges in the image, ensuring that only the most significant edges are considered. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The output image after applying the Canny edge detection algorithm. It highlights the edges detected in the input image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ControlNetApply,PreviewImage,Reroute` + + +## Source code +```python +class Canny: + @classmethod + def INPUT_TYPES(s): + return {"required": {"image": ("IMAGE",), + "low_threshold": ("FLOAT", {"default": 0.4, "min": 0.01, "max": 0.99, "step": 0.01}), + "high_threshold": ("FLOAT", {"default": 0.8, "min": 0.01, "max": 0.99, "step": 0.01}) + }} + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "detect_edge" + + CATEGORY = "image/preprocessors" + + def detect_edge(self, image, low_threshold, high_threshold): + output = canny(image.to(comfy.model_management.get_torch_device()).movedim(-1, 1), low_threshold, high_threshold) + img_out = output[1].to(comfy.model_management.intermediate_device()).repeat(1, 3, 1, 1).movedim(1, -1) + return (img_out,) + +``` diff --git a/docs/md/Comfy/Nodes/CheckpointLoader.md b/docs/md/Comfy/Nodes/CheckpointLoader.md new file mode 100644 index 0000000000..22e96bef6d --- /dev/null +++ b/docs/md/Comfy/Nodes/CheckpointLoader.md @@ -0,0 +1,53 @@ +# Load Checkpoint With Config (DEPRECATED) +## Documentation +- Class name: `CheckpointLoader` +- Category: `advanced/loaders` +- Output node: `False` + +The `CheckpointLoader` node is designed to load model checkpoints along with optional VAE and CLIP components based on specified configuration and checkpoint names. It facilitates the initialization of models for further operations or inference by loading the necessary components from disk. +## Input types +### Required +- **`config_name`** + - Specifies the name of the configuration file to use, which is essential for determining the model's architecture and parameters for initialization. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`ckpt_name`** + - Indicates the specific checkpoint file name to load, which contains the trained model weights necessary for the model's operation. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - Returns the main model loaded from the checkpoint. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip`** + - Optionally returns the CLIP component if available and requested. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`vae`** + - Optionally returns the VAE component if available and requested. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class CheckpointLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "config_name": (folder_paths.get_filename_list("configs"), ), + "ckpt_name": (folder_paths.get_filename_list("checkpoints"), )}} + RETURN_TYPES = ("MODEL", "CLIP", "VAE") + FUNCTION = "load_checkpoint" + + CATEGORY = "advanced/loaders" + + def load_checkpoint(self, config_name, ckpt_name, output_vae=True, output_clip=True): + config_path = folder_paths.get_full_path("configs", config_name) + ckpt_path = folder_paths.get_full_path("checkpoints", ckpt_name) + return comfy.sd.load_checkpoint(config_path, ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) + +``` diff --git a/docs/md/Comfy/Nodes/CheckpointLoaderSimple.md b/docs/md/Comfy/Nodes/CheckpointLoaderSimple.md new file mode 100644 index 0000000000..67d95b2de4 --- /dev/null +++ b/docs/md/Comfy/Nodes/CheckpointLoaderSimple.md @@ -0,0 +1,49 @@ +# Load Checkpoint +## Documentation +- Class name: `CheckpointLoaderSimple` +- Category: `loaders` +- Output node: `False` + +The `CheckpointLoaderSimple` node is designed to load model checkpoints without requiring a configuration name. It simplifies the checkpoint loading process by guessing the configuration based on the checkpoint name, making it easier to use for quick experiments or when the configuration is implicit. +## Input types +### Required +- **`ckpt_name`** + - Specifies the name of the checkpoint to load. This parameter is crucial as it determines which checkpoint file will be used for loading the model, CLIP, and VAE components. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - Returns the loaded model component from the specified checkpoint. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip`** + - Returns the loaded CLIP component from the specified checkpoint. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`vae`** + - Returns the loaded VAE component from the specified checkpoint. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `CLIPTextEncode,VAEDecode,LoraLoader,CLIPSetLastLayer,Reroute,CR Apply LoRA Stack,VAEEncode,KSampler,UltimateSDUpscale` + + +## Source code +```python +class CheckpointLoaderSimple: + @classmethod + def INPUT_TYPES(s): + return {"required": { "ckpt_name": (folder_paths.get_filename_list("checkpoints"), ), + }} + RETURN_TYPES = ("MODEL", "CLIP", "VAE") + FUNCTION = "load_checkpoint" + + CATEGORY = "loaders" + + def load_checkpoint(self, ckpt_name, output_vae=True, output_clip=True): + ckpt_path = folder_paths.get_full_path("checkpoints", ckpt_name) + out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) + return out[:3] + +``` diff --git a/docs/md/Comfy/Nodes/CheckpointSave.md b/docs/md/Comfy/Nodes/CheckpointSave.md new file mode 100644 index 0000000000..c88f50ef7a --- /dev/null +++ b/docs/md/Comfy/Nodes/CheckpointSave.md @@ -0,0 +1,56 @@ +# CheckpointSave +## Documentation +- Class name: `CheckpointSave` +- Category: `advanced/model_merging` +- Output node: `True` + +The `CheckpointSave` node is designed for saving models, CLIP, and VAE checkpoints to a specified directory. It allows for the inclusion of additional information such as prompts and extra PNG metadata. +## Input types +### Required +- **`model`** + - The model to be saved. It is crucial for preserving the trained state of the model for future use or further training. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip`** + - The CLIP model to be saved alongside the main model. Essential for tasks that require text-image alignment. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`vae`** + - The VAE model to be saved. Important for generative tasks, especially in the context of image synthesis. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`filename_prefix`** + - The prefix for the filename under which the checkpoint will be saved. Allows for organized storage and easy retrieval. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class CheckpointSave: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "clip": ("CLIP",), + "vae": ("VAE",), + "filename_prefix": ("STRING", {"default": "checkpoints/ComfyUI"}),}, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},} + RETURN_TYPES = () + FUNCTION = "save" + OUTPUT_NODE = True + + CATEGORY = "advanced/model_merging" + + def save(self, model, clip, vae, filename_prefix, prompt=None, extra_pnginfo=None): + save_checkpoint(model, clip=clip, vae=vae, filename_prefix=filename_prefix, output_dir=self.output_dir, prompt=prompt, extra_pnginfo=extra_pnginfo) + return {} + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningAverage.md b/docs/md/Comfy/Nodes/ConditioningAverage.md new file mode 100644 index 0000000000..4a964565ef --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningAverage.md @@ -0,0 +1,72 @@ +# ConditioningAverage +## Documentation +- Class name: `ConditioningAverage` +- Category: `conditioning` +- Output node: `False` + +The `ConditioningAverage` node blends two sets of conditioning data by averaging them together, weighted by a specified strength. It handles both the tensor representations and optional pooled outputs, ensuring compatibility in dimensions and applying the weighted average accordingly. +## Input types +### Required +- **`conditioning_to`** + - The target conditioning data to which the averaging operation will be applied. It plays a crucial role in determining the final output, as it is directly modified by the weighted average of itself and the `conditioning_from` data. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`conditioning_from`** + - The source conditioning data that will be averaged with the `conditioning_to` data. This input is essential for the blending process, contributing to the final conditioned output based on its weighted average. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`conditioning_to_strength`** + - A scalar value that determines the weight of the `conditioning_to` data in the averaging process. It directly influences the balance between the original and the blended conditioning data, affecting the final output. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The result of the weighted averaging process, combining `conditioning_to` and `conditioning_from` data. It reflects the blended conditioning data, adjusted according to the specified strength. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `FaceFixerOpenCV` + + +## Source code +```python +class ConditioningAverage : + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning_to": ("CONDITIONING", ), "conditioning_from": ("CONDITIONING", ), + "conditioning_to_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}) + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "addWeighted" + + CATEGORY = "conditioning" + + def addWeighted(self, conditioning_to, conditioning_from, conditioning_to_strength): + out = [] + + if len(conditioning_from) > 1: + print("Warning: ConditioningAverage conditioning_from contains more than 1 cond, only the first one will actually be applied to conditioning_to.") + + cond_from = conditioning_from[0][0] + pooled_output_from = conditioning_from[0][1].get("pooled_output", None) + + for i in range(len(conditioning_to)): + t1 = conditioning_to[i][0] + pooled_output_to = conditioning_to[i][1].get("pooled_output", pooled_output_from) + t0 = cond_from[:,:t1.shape[1]] + if t0.shape[1] < t1.shape[1]: + t0 = torch.cat([t0] + [torch.zeros((1, (t1.shape[1] - t0.shape[1]), t1.shape[2]))], dim=1) + + tw = torch.mul(t1, conditioning_to_strength) + torch.mul(t0, (1.0 - conditioning_to_strength)) + t_to = conditioning_to[i][1].copy() + if pooled_output_from is not None and pooled_output_to is not None: + t_to["pooled_output"] = torch.mul(pooled_output_to, conditioning_to_strength) + torch.mul(pooled_output_from, (1.0 - conditioning_to_strength)) + elif pooled_output_from is not None: + t_to["pooled_output"] = pooled_output_from + + n = [tw, t_to] + out.append(n) + return (out, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningCombine.md b/docs/md/Comfy/Nodes/ConditioningCombine.md new file mode 100644 index 0000000000..b0a233e8c4 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningCombine.md @@ -0,0 +1,42 @@ +# Conditioning (Combine) +## Documentation +- Class name: `ConditioningCombine` +- Category: `conditioning` +- Output node: `False` + +This node combines two conditioning inputs into a single output by simply adding them together. It's useful for merging conditioning information from different sources. +## Input types +### Required +- **`conditioning_1`** + - The first conditioning input to be combined. It plays an equal role in the combination process as `conditioning_2`. + - Python dtype: `Tuple[torch.Tensor, Dict[str, Any]]` + - Comfy dtype: `CONDITIONING` +- **`conditioning_2`** + - The second conditioning input to be combined. It is equally important in the combination process as `conditioning_1`. + - Python dtype: `Tuple[torch.Tensor, Dict[str, Any]]` + - Comfy dtype: `CONDITIONING` +## Output types +- **`conditioning`** + - The result of combining `conditioning_1` and `conditioning_2`, providing a merged conditioning output. + - Python dtype: `Tuple[torch.Tensor, Dict[str, Any]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: `KSampler,ConditioningCombine,KSampler Adv. (Efficient),CR Conditioning Input Switch,Attention couple,KSampler with Variations,CR Module Pipe Loader,ControlNetApplyAdvanced` + + +## Source code +```python +class ConditioningCombine: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning_1": ("CONDITIONING", ), "conditioning_2": ("CONDITIONING", )}} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "combine" + + CATEGORY = "conditioning" + + def combine(self, conditioning_1, conditioning_2): + return (conditioning_1 + conditioning_2, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningConcat.md b/docs/md/Comfy/Nodes/ConditioningConcat.md new file mode 100644 index 0000000000..059a183431 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningConcat.md @@ -0,0 +1,58 @@ +# Conditioning (Concat) +## Documentation +- Class name: `ConditioningConcat` +- Category: `conditioning` +- Output node: `False` + +The `ConditioningConcat` node concatenates the conditioning information from one source to another. It specifically takes the first conditioning element from the `conditioning_from` input and concatenates it to each element in the `conditioning_to` input, effectively applying the conditioning from one source across another. +## Input types +### Required +- **`conditioning_to`** + - The target conditioning data to which the conditioning from another source will be concatenated. It plays a crucial role in determining the final output by receiving additional conditioning information. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`conditioning_from`** + - The source of conditioning data to be concatenated to the target conditioning data. Only the first element of this input is used, which emphasizes its selective influence on the target conditioning. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Output types +- **`conditioning`** + - The output is the modified target conditioning data with the first element from the source conditioning data concatenated to each of its elements. + - Python dtype: `Tuple[List[Tuple[torch.Tensor, Dict[str, Any]]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ControlNetApplyAdvanced,SamplerCustom,ConditioningConcat` + + +## Source code +```python +class ConditioningConcat: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "conditioning_to": ("CONDITIONING",), + "conditioning_from": ("CONDITIONING",), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "concat" + + CATEGORY = "conditioning" + + def concat(self, conditioning_to, conditioning_from): + out = [] + + if len(conditioning_from) > 1: + print("Warning: ConditioningConcat conditioning_from contains more than 1 cond, only the first one will actually be applied to conditioning_to.") + + cond_from = conditioning_from[0][0] + + for i in range(len(conditioning_to)): + t1 = conditioning_to[i][0] + tw = torch.cat((t1, cond_from),1) + n = [tw, conditioning_to[i][1].copy()] + out.append(n) + + return (out, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningSetArea.md b/docs/md/Comfy/Nodes/ConditioningSetArea.md new file mode 100644 index 0000000000..09436f78cd --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningSetArea.md @@ -0,0 +1,71 @@ +# Conditioning (Set Area) +## Documentation +- Class name: `ConditioningSetArea` +- Category: `conditioning` +- Output node: `False` + +This node modifies the conditioning data by setting specific areas and strengths for each conditioning element. It's used to apply localized adjustments to the conditioning, allowing for more precise control over the generation process. +## Input types +### Required +- **`conditioning`** + - The base conditioning data to be modified. It's crucial for defining the context or content that the generation process should adhere to. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`width`** + - Specifies the width of the area to be set. It determines how wide the area of effect will be. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Specifies the height of the area to be set. It determines how tall the area of effect will be. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`x`** + - The x-coordinate of the top-left corner of the area to be set. It positions the area horizontally. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate of the top-left corner of the area to be set. It positions the area vertically. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`strength`** + - Determines the intensity of the effect applied to the specified area. Higher values result in stronger effects. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The modified conditioning data with specified areas and strengths applied to each element. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: `ConditioningCombine,SetNode` + + +## Source code +```python +class ConditioningSetArea: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "width": ("INT", {"default": 64, "min": 64, "max": MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 64, "min": 64, "max": MAX_RESOLUTION, "step": 8}), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "append" + + CATEGORY = "conditioning" + + def append(self, conditioning, width, height, x, y, strength): + c = [] + for t in conditioning: + n = [t[0], t[1].copy()] + n[1]['area'] = (height // 8, width // 8, y // 8, x // 8) + n[1]['strength'] = strength + n[1]['set_area_to_bounds'] = False + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningSetAreaPercentage.md b/docs/md/Comfy/Nodes/ConditioningSetAreaPercentage.md new file mode 100644 index 0000000000..4703842ae4 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningSetAreaPercentage.md @@ -0,0 +1,71 @@ +# Conditioning (Set Area with Percentage) +## Documentation +- Class name: `ConditioningSetAreaPercentage` +- Category: `conditioning` +- Output node: `False` + +This node modifies the conditioning by setting the area of interest as a percentage of the total image size. It allows for specifying the area's width, height, and its position (x, y) relative to the image, along with the strength of the conditioning effect. This is useful for focusing the model's attention on specific parts of an image during generation. +## Input types +### Required +- **`conditioning`** + - The conditioning data to be modified. It's crucial for directing the model's focus or altering its behavior based on specified conditions. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`width`** + - The width of the area of interest as a percentage of the total image width. It defines how wide the focused area should be. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`height`** + - The height of the area of interest as a percentage of the total image height. It defines how tall the focused area should be. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`x`** + - The x-coordinate of the top-left corner of the area of interest, as a percentage of the total image width. It specifies where the area begins horizontally. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`y`** + - The y-coordinate of the top-left corner of the area of interest, as a percentage of the total image height. It specifies where the area begins vertically. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`strength`** + - The strength of the conditioning effect within the specified area. It determines how strongly the model should focus on or alter this area. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The modified conditioning data with the specified area and strength settings applied. It's used to direct the model's focus or alter its behavior. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class ConditioningSetAreaPercentage: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "width": ("FLOAT", {"default": 1.0, "min": 0, "max": 1.0, "step": 0.01}), + "height": ("FLOAT", {"default": 1.0, "min": 0, "max": 1.0, "step": 0.01}), + "x": ("FLOAT", {"default": 0, "min": 0, "max": 1.0, "step": 0.01}), + "y": ("FLOAT", {"default": 0, "min": 0, "max": 1.0, "step": 0.01}), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "append" + + CATEGORY = "conditioning" + + def append(self, conditioning, width, height, x, y, strength): + c = [] + for t in conditioning: + n = [t[0], t[1].copy()] + n[1]['area'] = ("percentage", height, width, y, x) + n[1]['strength'] = strength + n[1]['set_area_to_bounds'] = False + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningSetAreaStrength.md b/docs/md/Comfy/Nodes/ConditioningSetAreaStrength.md new file mode 100644 index 0000000000..ffe3a351e1 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningSetAreaStrength.md @@ -0,0 +1,49 @@ +# ConditioningSetAreaStrength +## Documentation +- Class name: `ConditioningSetAreaStrength` +- Category: `conditioning` +- Output node: `False` + +The `ConditioningSetAreaStrength` node modifies the strength attribute of each conditioning element in a given list. This operation allows for the adjustment of the influence or intensity of the conditioning elements. +## Input types +### Required +- **`conditioning`** + - The list of conditioning elements to be modified. Each element's strength attribute will be adjusted. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`strength`** + - The new strength value to be applied to all conditioning elements. This value determines the intensity of the effect each conditioning element has. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The modified list of conditioning elements, with updated strength values for each element. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class ConditioningSetAreaStrength: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "append" + + CATEGORY = "conditioning" + + def append(self, conditioning, strength): + c = [] + for t in conditioning: + n = [t[0], t[1].copy()] + n[1]['strength'] = strength + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningSetMask.md b/docs/md/Comfy/Nodes/ConditioningSetMask.md new file mode 100644 index 0000000000..8018bcb074 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningSetMask.md @@ -0,0 +1,67 @@ +# Conditioning (Set Mask) +## Documentation +- Class name: `ConditioningSetMask` +- Category: `conditioning` +- Output node: `False` + +The `ConditioningSetMask` node is designed to apply a mask to a given conditioning set, potentially altering the area of effect based on the mask's bounds and a specified strength. This operation is useful for selectively applying conditioning to specific regions of an input. +## Input types +### Required +- **`conditioning`** + - The conditioning set to which the mask will be applied. This parameter is crucial for defining the context or conditions under which certain operations should be performed. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`mask`** + - The mask to be applied to the conditioning set. It defines the area over which the conditioning should be altered. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`strength`** + - Determines the intensity of the mask's effect on the conditioning set. A higher value means a stronger effect. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`set_cond_area`** + - Specifies whether to set the conditioning area to the mask bounds or use the default setting. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`conditioning`** + - The modified conditioning set with the mask applied. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: `Attention couple,ConditioningCombine` + + +## Source code +```python +class ConditioningSetMask: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "mask": ("MASK", ), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + "set_cond_area": (["default", "mask bounds"],), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "append" + + CATEGORY = "conditioning" + + def append(self, conditioning, mask, set_cond_area, strength): + c = [] + set_area_to_bounds = False + if set_cond_area != "default": + set_area_to_bounds = True + if len(mask.shape) < 3: + mask = mask.unsqueeze(0) + for t in conditioning: + n = [t[0], t[1].copy()] + _, h, w = mask.shape + n[1]['mask'] = mask + n[1]['set_area_to_bounds'] = set_area_to_bounds + n[1]['mask_strength'] = strength + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningSetTimestepRange.md b/docs/md/Comfy/Nodes/ConditioningSetTimestepRange.md new file mode 100644 index 0000000000..a718766980 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningSetTimestepRange.md @@ -0,0 +1,56 @@ +# ConditioningSetTimestepRange +## Documentation +- Class name: `ConditioningSetTimestepRange` +- Category: `advanced/conditioning` +- Output node: `False` + +This node modifies the conditioning data by setting a specific range of timesteps for the conditioning process. It allows for the precise control of the start and end points of the conditioning, enabling more targeted and efficient generation. +## Input types +### Required +- **`conditioning`** + - The conditioning data to be modified. It serves as the basis for the generation process, dictating the characteristics and constraints of the output. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`start`** + - The starting point of the timestep range, as a percentage of the total timesteps. It specifies the beginning of the conditioning effect. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`end`** + - The ending point of the timestep range, as a percentage of the total timesteps. It determines when the conditioning effect concludes. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The modified conditioning data with the specified timestep range applied. This data is used for further processing or generation. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: `CR Conditioning Mixer` + + +## Source code +```python +class ConditioningSetTimestepRange: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}), + "end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}) + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "set_range" + + CATEGORY = "advanced/conditioning" + + def set_range(self, conditioning, start, end): + c = [] + for t in conditioning: + d = t[1].copy() + d['start_percent'] = start + d['end_percent'] = end + n = [t[0], d] + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ConditioningZeroOut.md b/docs/md/Comfy/Nodes/ConditioningZeroOut.md new file mode 100644 index 0000000000..d393639bb3 --- /dev/null +++ b/docs/md/Comfy/Nodes/ConditioningZeroOut.md @@ -0,0 +1,45 @@ +# ConditioningZeroOut +## Documentation +- Class name: `ConditioningZeroOut` +- Category: `advanced/conditioning` +- Output node: `False` + +The `ConditioningZeroOut` node zeroes out the `pooled_output` tensor within each conditioning element, effectively neutralizing its influence in subsequent operations. This operation is useful for selectively disabling parts of the conditioning data. +## Input types +### Required +- **`conditioning`** + - The conditioning data to be modified. This operation zeroes out the `pooled_output` tensor within each element, which can be useful for experiments or when certain parts of the conditioning are not desired. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +## Output types +- **`conditioning`** + - The modified conditioning data with the `pooled_output` tensor zeroed out in each element. This allows for selective disabling of parts of the conditioning. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `UltimateSDUpscale` + + +## Source code +```python +class ConditioningZeroOut: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", )}} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "zero_out" + + CATEGORY = "advanced/conditioning" + + def zero_out(self, conditioning): + c = [] + for t in conditioning: + d = t[1].copy() + if "pooled_output" in d: + d["pooled_output"] = torch.zeros_like(d["pooled_output"]) + n = [torch.zeros_like(t[0]), d] + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ControlNetApply.md b/docs/md/Comfy/Nodes/ControlNetApply.md new file mode 100644 index 0000000000..8530fe8345 --- /dev/null +++ b/docs/md/Comfy/Nodes/ControlNetApply.md @@ -0,0 +1,67 @@ +# Apply ControlNet +## Documentation +- Class name: `ControlNetApply` +- Category: `conditioning` +- Output node: `False` + +This node applies a control network to an input image and conditioning data, adjusting the conditioning based on the control network's output. The strength parameter controls the intensity of the control network's effect, allowing for fine-tuning of the applied conditioning. +## Input types +### Required +- **`conditioning`** + - The conditioning data to be modified by the control network. It's crucial for determining the final output as it guides the control network's adjustments. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`control_net`** + - The control network model used to adjust the conditioning data based on the input image. It plays a key role in how the conditioning is modified. + - Python dtype: `ControlNet` + - Comfy dtype: `CONTROL_NET` +- **`image`** + - The input image that the control network uses to adjust the conditioning data. It serves as a reference for the control network's modifications. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`strength`** + - A parameter that controls the intensity of the control network's effect on the conditioning data. It allows for fine-tuning the impact of the control network. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The modified conditioning data after applying the control network. It reflects the adjustments made based on the input image and control network's output. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,Reroute,ConditioningSetArea,UltimateSDUpscale,KSampler Adv. (Efficient),ttN pipeKSampler,YDetailer,ToBasicPipe,ToDetailerPipe,ControlNetApply` + + +## Source code +```python +class ControlNetApply: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "control_net": ("CONTROL_NET", ), + "image": ("IMAGE", ), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}) + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "apply_controlnet" + + CATEGORY = "conditioning" + + def apply_controlnet(self, conditioning, control_net, image, strength): + if strength == 0: + return (conditioning, ) + + c = [] + control_hint = image.movedim(-1,1) + for t in conditioning: + n = [t[0], t[1].copy()] + c_net = control_net.copy().set_cond_hint(control_hint, strength) + if 'control' in t[1]: + c_net.set_previous_controlnet(t[1]['control']) + n[1]['control'] = c_net + n[1]['control_apply_to_uncond'] = True + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/ControlNetApplyAdvanced.md b/docs/md/Comfy/Nodes/ControlNetApplyAdvanced.md new file mode 100644 index 0000000000..2a94e3a39b --- /dev/null +++ b/docs/md/Comfy/Nodes/ControlNetApplyAdvanced.md @@ -0,0 +1,96 @@ +# Apply ControlNet (Advanced) +## Documentation +- Class name: `ControlNetApplyAdvanced` +- Category: `conditioning` +- Output node: `False` + +This node applies advanced control net transformations to a given image, based on the provided conditioning, control net, and strength parameters. It supports additional customization through start and end percentages, allowing for more precise control over the application of the control net. The method enhances or modifies the conditioning of the image by applying control hints derived from the image itself, adjusted by the strength parameter, and further refined by the specified percentage range. +## Input types +### Required +- **`positive`** + - The positive conditioning to be applied or enhanced on the image. It plays a crucial role in determining the final output by providing a base set of conditions that the control net aims to amplify or modify. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`negative`** + - The negative conditioning, serving as a counterbalance to the positive conditioning. It helps in fine-tuning the final output by providing conditions that the control net should suppress or negate. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`control_net`** + - The control net model used to apply transformations to the image. It is a crucial component that determines how the conditioning is modified or enhanced. + - Python dtype: `ControlNet` + - Comfy dtype: `CONTROL_NET` +- **`image`** + - The target image to which the control net transformations are applied. The image serves as a basis for generating control hints that guide the conditioning modification. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`strength`** + - Determines the intensity of the control net application. A higher strength value results in more pronounced modifications to the conditioning. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`start_percent`** + - Specifies the starting percentage of the control net application, allowing for targeted modifications within a specific range of the image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`end_percent`** + - Defines the ending percentage of the control net application, complementing the start_percent to confine the control net's effects within a precise segment of the image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The adjusted negative conditioning post control net application, showcasing the suppression or negation of certain conditions. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,KSamplerAdvanced,ControlNetApplyAdvanced,KSampler //Inspire,UltimateSDUpscale,ToBasicPipe,FaceDetailer,Reroute,SamplerCustom,Bus Node` + + +## Source code +```python +class ControlNetApplyAdvanced: + @classmethod + def INPUT_TYPES(s): + return {"required": {"positive": ("CONDITIONING", ), + "negative": ("CONDITIONING", ), + "control_net": ("CONTROL_NET", ), + "image": ("IMAGE", ), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + "start_percent": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}), + "end_percent": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}) + }} + + RETURN_TYPES = ("CONDITIONING","CONDITIONING") + RETURN_NAMES = ("positive", "negative") + FUNCTION = "apply_controlnet" + + CATEGORY = "conditioning" + + def apply_controlnet(self, positive, negative, control_net, image, strength, start_percent, end_percent): + if strength == 0: + return (positive, negative) + + control_hint = image.movedim(-1,1) + cnets = {} + + out = [] + for conditioning in [positive, negative]: + c = [] + for t in conditioning: + d = t[1].copy() + + prev_cnet = d.get('control', None) + if prev_cnet in cnets: + c_net = cnets[prev_cnet] + else: + c_net = control_net.copy().set_cond_hint(control_hint, strength, (start_percent, end_percent)) + c_net.set_previous_controlnet(prev_cnet) + cnets[prev_cnet] = c_net + + d['control'] = c_net + d['control_apply_to_uncond'] = False + n = [t[0], d] + c.append(n) + out.append(c) + return (out[0], out[1]) + +``` diff --git a/docs/md/Comfy/Nodes/ControlNetLoader.md b/docs/md/Comfy/Nodes/ControlNetLoader.md new file mode 100644 index 0000000000..518e6a881c --- /dev/null +++ b/docs/md/Comfy/Nodes/ControlNetLoader.md @@ -0,0 +1,41 @@ +# Load ControlNet Model +## Documentation +- Class name: `ControlNetLoader` +- Category: `loaders` +- Output node: `False` + +This node is responsible for loading a ControlNet model by name. It retrieves the full path of the specified ControlNet using a helper function and then loads the ControlNet model from that path. +## Input types +### Required +- **`control_net_name`** + - Specifies the name of the ControlNet model to load. This name is used to retrieve the model's full path for loading. The list of available ControlNet names is dynamically generated based on the contents of a specific directory. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`control_net`** + - The loaded ControlNet model, ready for use in further processing or conditioning steps. + - Python dtype: `ControlNet` + - Comfy dtype: `CONTROL_NET` +## Usage tips +- Infra type: `CPU` +- Common nodes: `ControlNetApplyAdvanced,ControlNetApply,ACN_AdvancedControlNetApply,Reroute,ImpactControlNetApplySEGS` + + +## Source code +```python +class ControlNetLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "control_net_name": (folder_paths.get_filename_list("controlnet"), )}} + + RETURN_TYPES = ("CONTROL_NET",) + FUNCTION = "load_controlnet" + + CATEGORY = "loaders" + + def load_controlnet(self, control_net_name): + controlnet_path = folder_paths.get_full_path("controlnet", control_net_name) + controlnet = comfy.controlnet.load_controlnet(controlnet_path) + return (controlnet,) + +``` diff --git a/docs/md/Comfy/Nodes/CropMask.md b/docs/md/Comfy/Nodes/CropMask.md new file mode 100644 index 0000000000..5c3bf8b736 --- /dev/null +++ b/docs/md/Comfy/Nodes/CropMask.md @@ -0,0 +1,66 @@ +# CropMask +## Documentation +- Class name: `CropMask` +- Category: `mask` +- Output node: `False` + +The `CropMask` node is designed to crop a given mask to a specified rectangular area. It reshapes the input mask to ensure compatibility, then extracts the specified region based on the provided coordinates and dimensions. +## Input types +### Required +- **`mask`** + - The mask to be cropped. It's reshaped to ensure compatibility before cropping. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`x`** + - The x-coordinate of the top-left corner of the cropping rectangle. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate of the top-left corner of the cropping rectangle. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`width`** + - The width of the cropping rectangle. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - The height of the cropping rectangle. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`mask`** + - The cropped mask, which is a subsection of the original mask based on the specified coordinates and dimensions. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `VAEEncodeForInpaint,MaskToImage` + + +## Source code +```python +class CropMask: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mask": ("MASK",), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "width": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + "height": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + + FUNCTION = "crop" + + def crop(self, mask, x, y, width, height): + mask = mask.reshape((-1, mask.shape[-2], mask.shape[-1])) + out = mask[:, y:y + height, x:x + width] + return (out,) + +``` diff --git a/docs/md/Comfy/Nodes/DiffControlNetLoader.md b/docs/md/Comfy/Nodes/DiffControlNetLoader.md new file mode 100644 index 0000000000..64a8d45dec --- /dev/null +++ b/docs/md/Comfy/Nodes/DiffControlNetLoader.md @@ -0,0 +1,46 @@ +# Load ControlNet Model (diff) +## Documentation +- Class name: `DiffControlNetLoader` +- Category: `loaders` +- Output node: `False` + +This node is responsible for loading a differential control network model, which is a specialized model designed to apply modifications or controls to another model based on a given control network name and the model itself. It enhances the flexibility and capability of the control network by allowing it to adapt based on the specific model it is applied to. +## Input types +### Required +- **`model`** + - The model parameter is crucial as it specifies the base model to which the differential control network will be applied. This allows the control network to adapt its behavior based on the characteristics of the base model, enhancing its effectiveness. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`control_net_name`** + - The control_net_name parameter specifies the name of the control network to be loaded. It is essential for identifying the correct control network file from a list of available control networks, enabling the dynamic loading of different control networks based on requirements. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`control_net`** + - The output is a control network model that has been loaded and potentially adapted based on the input model. This control network can then be applied to modify or control the behavior of the input model in specific ways. + - Python dtype: `ControlNet` + - Comfy dtype: `CONTROL_NET` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class DiffControlNetLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "control_net_name": (folder_paths.get_filename_list("controlnet"), )}} + + RETURN_TYPES = ("CONTROL_NET",) + FUNCTION = "load_controlnet" + + CATEGORY = "loaders" + + def load_controlnet(self, model, control_net_name): + controlnet_path = folder_paths.get_full_path("controlnet", control_net_name) + controlnet = comfy.controlnet.load_controlnet(controlnet_path, model) + return (controlnet,) + +``` diff --git a/docs/md/Comfy/Nodes/DiffusersLoader.md b/docs/md/Comfy/Nodes/DiffusersLoader.md new file mode 100644 index 0000000000..09b9b974fc --- /dev/null +++ b/docs/md/Comfy/Nodes/DiffusersLoader.md @@ -0,0 +1,60 @@ +# DiffusersLoader +## Documentation +- Class name: `DiffusersLoader` +- Category: `advanced/loaders/deprecated` +- Output node: `False` + +The `DiffusersLoader` node is responsible for loading models related to the diffusers library, specifically handling the loading of diffusion models, CLIP models, and VAEs based on the provided model path. It supports conditional loading of VAE and CLIP models based on the input flags. +## Input types +### Required +- **`model_path`** + - The path to the model directory. This path is crucial for locating the specific models to be loaded. The correct identification and loading of the diffusion model, CLIP model, and VAE depend on this path, directly influencing the node's execution and the types of models available for subsequent operations. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - Returns the loaded diffusion model. + - Python dtype: `comfy.sd.DiffusionModel` + - Comfy dtype: `MODEL` +- **`clip`** + - Returns the loaded CLIP model, if requested. + - Python dtype: `Optional[comfy.sd.CLIPModel]` + - Comfy dtype: `CLIP` +- **`vae`** + - Returns the loaded VAE model, if requested. + - Python dtype: `Optional[comfy.sd.VAEModel]` + - Comfy dtype: `VAE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class DiffusersLoader: + @classmethod + def INPUT_TYPES(cls): + paths = [] + for search_path in folder_paths.get_folder_paths("diffusers"): + if os.path.exists(search_path): + for root, subdir, files in os.walk(search_path, followlinks=True): + if "model_index.json" in files: + paths.append(os.path.relpath(root, start=search_path)) + + return {"required": {"model_path": (paths,), }} + RETURN_TYPES = ("MODEL", "CLIP", "VAE") + FUNCTION = "load_checkpoint" + + CATEGORY = "advanced/loaders/deprecated" + + def load_checkpoint(self, model_path, output_vae=True, output_clip=True): + for search_path in folder_paths.get_folder_paths("diffusers"): + if os.path.exists(search_path): + path = os.path.join(search_path, model_path) + if os.path.exists(path): + model_path = path + break + + return comfy.diffusers_load.load_diffusers(model_path, output_vae=output_vae, output_clip=output_clip, embedding_directory=folder_paths.get_folder_paths("embeddings")) + +``` diff --git a/docs/md/Comfy/Nodes/DualCLIPLoader.md b/docs/md/Comfy/Nodes/DualCLIPLoader.md new file mode 100644 index 0000000000..e6c6363f45 --- /dev/null +++ b/docs/md/Comfy/Nodes/DualCLIPLoader.md @@ -0,0 +1,46 @@ +# DualCLIPLoader +## Documentation +- Class name: `DualCLIPLoader` +- Category: `advanced/loaders` +- Output node: `False` + +The DualCLIPLoader node is designed to load two CLIP models simultaneously. It takes the names of two CLIP models as input, retrieves their paths, and loads them into a single CLIP model instance with embeddings from a specified directory. This node is useful for operations that require the simultaneous use of two distinct CLIP models. +## Input types +### Required +- **`clip_name1`** + - Specifies the name of the first CLIP model to be loaded. The name is used to retrieve the model's path for loading. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`clip_name2`** + - Specifies the name of the second CLIP model to be loaded. Similar to clip_name1, it is used to retrieve the model's path for loading. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`clip`** + - The loaded CLIP model instance, which combines the embeddings and configurations of the two specified CLIP models. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class DualCLIPLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_name1": (folder_paths.get_filename_list("clip"), ), "clip_name2": (folder_paths.get_filename_list("clip"), ), + }} + RETURN_TYPES = ("CLIP",) + FUNCTION = "load_clip" + + CATEGORY = "advanced/loaders" + + def load_clip(self, clip_name1, clip_name2): + clip_path1 = folder_paths.get_full_path("clip", clip_name1) + clip_path2 = folder_paths.get_full_path("clip", clip_name2) + clip = comfy.sd.load_clip(ckpt_paths=[clip_path1, clip_path2], embedding_directory=folder_paths.get_folder_paths("embeddings")) + return (clip,) + +``` diff --git a/docs/md/Comfy/Nodes/EmptyImage.md b/docs/md/Comfy/Nodes/EmptyImage.md new file mode 100644 index 0000000000..72b807f78f --- /dev/null +++ b/docs/md/Comfy/Nodes/EmptyImage.md @@ -0,0 +1,60 @@ +# EmptyImage +## Documentation +- Class name: `EmptyImage` +- Category: `image` +- Output node: `False` + +The `EmptyImage` node generates a blank image with specified dimensions, batch size, and color. It utilizes PyTorch tensors to create an image filled with the given color. +## Input types +### Required +- **`width`** + - Specifies the width of the generated image. It directly influences the dimensions of the output image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Determines the height of the generated image, affecting its dimensions. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`batch_size`** + - Indicates the number of images to generate in a single batch. Affects the first dimension of the output tensor. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`color`** + - Defines the color of the generated image using a hexadecimal RGB value. This parameter sets the overall color tone of the image. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - The output is a tensor representing a batch of images filled with the specified color. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ImageCompositeMasked` + + +## Source code +```python +class EmptyImage: + def __init__(self, device="cpu"): + self.device = device + + @classmethod + def INPUT_TYPES(s): + return {"required": { "width": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + "height": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "color": ("INT", {"default": 0, "min": 0, "max": 0xFFFFFF, "step": 1, "display": "color"}), + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "generate" + + CATEGORY = "image" + + def generate(self, width, height, batch_size=1, color=0): + r = torch.full([batch_size, height, width, 1], ((color >> 16) & 0xFF) / 0xFF) + g = torch.full([batch_size, height, width, 1], ((color >> 8) & 0xFF) / 0xFF) + b = torch.full([batch_size, height, width, 1], ((color) & 0xFF) / 0xFF) + return (torch.cat((r, g, b), dim=-1), ) + +``` diff --git a/docs/md/Comfy/Nodes/EmptyLatentImage.md b/docs/md/Comfy/Nodes/EmptyLatentImage.md new file mode 100644 index 0000000000..692e92dbcb --- /dev/null +++ b/docs/md/Comfy/Nodes/EmptyLatentImage.md @@ -0,0 +1,52 @@ +# Empty Latent Image +## Documentation +- Class name: `EmptyLatentImage` +- Category: `latent` +- Output node: `False` + +This node generates an empty latent image tensor with specified dimensions and batch size. The tensor is filled with zeros and is intended for use as a placeholder or starting point in generative models. +## Input types +### Required +- **`width`** + - Specifies the width of the latent image to be generated. It determines the horizontal dimension of the tensor. The width is crucial for defining the resolution and scale of the generated latent space, impacting the detail level of subsequent image generation. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Specifies the height of the latent image to be generated. It affects the vertical dimension of the tensor. Similar to width, the height is essential for setting the resolution and scale of the latent space, influencing the detail and proportions of generated images. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`batch_size`** + - Determines the number of latent images to generate in a single batch. This parameter allows for batch processing of images, enabling efficient generation and manipulation of multiple latent images simultaneously, which is vital for scaling up image generation processes. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The output is a tensor representing a batch of empty latent images. These images are initialized with zeros and are ready for further processing or manipulation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,SamplerCustom,KSamplerAdvanced,CR Module Pipe Loader,KSampler Adv. (Efficient),ImpactKSamplerBasicPipe,Anything Everywhere,Reroute,KRestartSamplerAdv,LatentComposite` + + +## Source code +```python +class EmptyLatentImage: + def __init__(self): + self.device = comfy.model_management.intermediate_device() + + @classmethod + def INPUT_TYPES(s): + return {"required": { "width": ("INT", {"default": 512, "min": 16, "max": MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 512, "min": 16, "max": MAX_RESOLUTION, "step": 8}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096})}} + RETURN_TYPES = ("LATENT",) + FUNCTION = "generate" + + CATEGORY = "latent" + + def generate(self, width, height, batch_size=1): + latent = torch.zeros([batch_size, 4, height // 8, width // 8], device=self.device) + return ({"samples":latent}, ) + +``` diff --git a/docs/md/Comfy/Nodes/ExponentialScheduler.md b/docs/md/Comfy/Nodes/ExponentialScheduler.md new file mode 100644 index 0000000000..7f1a6eb14d --- /dev/null +++ b/docs/md/Comfy/Nodes/ExponentialScheduler.md @@ -0,0 +1,52 @@ +# ExponentialScheduler +## Documentation +- Class name: `ExponentialScheduler` +- Category: `sampling/custom_sampling/schedulers` +- Output node: `False` + +The ExponentialScheduler node generates a sequence of sigma values following an exponential schedule for diffusion sampling. It calculates these values based on the number of steps and the minimum and maximum sigma values provided. +## Input types +### Required +- **`steps`** + - Defines the total number of steps for the diffusion process. The number of steps influences the granularity of the generated sigma sequence, with more steps resulting in a finer sequence. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`sigma_max`** + - The maximum sigma value in the generated sequence. It sets the upper limit of noise levels for the diffusion process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sigma_min`** + - The minimum sigma value in the generated sequence. It sets the lower limit of noise levels, influencing the starting point of the diffusion process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`sigmas`** + - The generated sequence of sigma values following an exponential schedule. These values are used to control the noise levels at each step of the diffusion process. + - Python dtype: `Tuple[torch.Tensor]` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ExponentialScheduler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "sigma_max": ("FLOAT", {"default": 14.614642, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + "sigma_min": ("FLOAT", {"default": 0.0291675, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/schedulers" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, steps, sigma_max, sigma_min): + sigmas = k_diffusion_sampling.get_sigmas_exponential(n=steps, sigma_min=sigma_min, sigma_max=sigma_max) + return (sigmas, ) + +``` diff --git a/docs/md/Comfy/Nodes/FeatherMask.md b/docs/md/Comfy/Nodes/FeatherMask.md new file mode 100644 index 0000000000..e9ff5a613f --- /dev/null +++ b/docs/md/Comfy/Nodes/FeatherMask.md @@ -0,0 +1,87 @@ +# FeatherMask +## Documentation +- Class name: `FeatherMask` +- Category: `mask` +- Output node: `False` + +The `FeatherMask` node applies a feathering effect to the edges of a given mask. This effect gradually fades the mask's edges by adjusting their opacity, creating a smoother transition at the specified boundaries. +## Input types +### Required +- **`mask`** + - The mask to which the feathering effect will be applied. It's crucial for creating smooth transitions at the mask's edges. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`left`** + - Specifies the width of the feathering effect on the left edge of the mask. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`top`** + - Specifies the height of the feathering effect on the top edge of the mask. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`right`** + - Specifies the width of the feathering effect on the right edge of the mask. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`bottom`** + - Specifies the height of the feathering effect on the bottom edge of the mask. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`mask`** + - The modified mask with a feathered edge effect applied to its specified boundaries. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `MaskComposite,Reroute` + + +## Source code +```python +class FeatherMask: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mask": ("MASK",), + "left": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "top": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "right": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "bottom": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + + FUNCTION = "feather" + + def feather(self, mask, left, top, right, bottom): + output = mask.reshape((-1, mask.shape[-2], mask.shape[-1])).clone() + + left = min(left, output.shape[-1]) + right = min(right, output.shape[-1]) + top = min(top, output.shape[-2]) + bottom = min(bottom, output.shape[-2]) + + for x in range(left): + feather_rate = (x + 1.0) / left + output[:, :, x] *= feather_rate + + for x in range(right): + feather_rate = (x + 1) / right + output[:, :, -x] *= feather_rate + + for y in range(top): + feather_rate = (y + 1) / top + output[:, y, :] *= feather_rate + + for y in range(bottom): + feather_rate = (y + 1) / bottom + output[:, -y, :] *= feather_rate + + return (output,) + +``` diff --git a/docs/md/Comfy/Nodes/FlipSigmas.md b/docs/md/Comfy/Nodes/FlipSigmas.md new file mode 100644 index 0000000000..5034f7e5f8 --- /dev/null +++ b/docs/md/Comfy/Nodes/FlipSigmas.md @@ -0,0 +1,44 @@ +# FlipSigmas +## Documentation +- Class name: `FlipSigmas` +- Category: `sampling/custom_sampling/sigmas` +- Output node: `False` + +The `FlipSigmas` node is designed to manipulate a sequence of sigma values, which are typically used in diffusion models for controlling the noise level at each step of the generation process. It reverses the order of the sigma values and ensures that the first value is not zero by setting a minimum threshold if needed. This operation can be crucial for certain sampling strategies where the direction of processing or the starting point within the noise schedule needs to be adjusted. +## Input types +### Required +- **`sigmas`** + - The `sigmas` parameter represents a sequence of sigma values that control the noise level at different steps of a diffusion process. Reversing the order of these values can be essential for altering the direction of the diffusion process, ensuring that the generation starts from a different noise level. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Output types +- **`sigmas`** + - Returns the modified sequence of sigma values, with the order reversed and the first value adjusted if it was originally zero, to ensure it's non-zero. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class FlipSigmas: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"sigmas": ("SIGMAS", ), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/sigmas" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, sigmas): + sigmas = sigmas.flip(0) + if sigmas[0] == 0: + sigmas[0] = 0.0001 + return (sigmas,) + +``` diff --git a/docs/md/Comfy/Nodes/FreeU.md b/docs/md/Comfy/Nodes/FreeU.md new file mode 100644 index 0000000000..694b2fd27e --- /dev/null +++ b/docs/md/Comfy/Nodes/FreeU.md @@ -0,0 +1,81 @@ +# FreeU +## Documentation +- Class name: `FreeU` +- Category: `model_patches` +- Output node: `False` + +The `FreeU` node is designed to modify the output of a model by applying a custom patching function. This function adjusts the model's output based on predefined scaling factors and optionally applies a Fourier filter to handle spatial frequencies. The patching process is intended to enhance or alter the model's output in a specific manner, potentially improving the quality or relevance of the generated content. +## Input types +### Required +- **`model`** + - The model to be patched. This is the primary input and the target of the patching operation, which will have its output block modified to apply the custom adjustments. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`b1`** + - A scaling factor for adjusting the model's output. It is part of the scale dictionary that determines how the output is modified. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`b2`** + - Another scaling factor for adjusting the model's output, used in conjunction with `b1` to define the scale dictionary. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`s1`** + - A scaling parameter for the Fourier filter, affecting how spatial frequencies are handled in the patching process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`s2`** + - Another scaling parameter for the Fourier filter, working alongside `s1` to fine-tune the spatial frequency handling. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The patched model, which has been modified by the `FreeU` node to include custom adjustments and potentially enhanced output. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,KSamplerAdvanced,UltimateSDUpscale,Bus Node,DynamicThresholdingFull,Anything Everywhere` + + +## Source code +```python +class FreeU: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "b1": ("FLOAT", {"default": 1.1, "min": 0.0, "max": 10.0, "step": 0.01}), + "b2": ("FLOAT", {"default": 1.2, "min": 0.0, "max": 10.0, "step": 0.01}), + "s1": ("FLOAT", {"default": 0.9, "min": 0.0, "max": 10.0, "step": 0.01}), + "s2": ("FLOAT", {"default": 0.2, "min": 0.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "model_patches" + + def patch(self, model, b1, b2, s1, s2): + model_channels = model.model.model_config.unet_config["model_channels"] + scale_dict = {model_channels * 4: (b1, s1), model_channels * 2: (b2, s2)} + on_cpu_devices = {} + + def output_block_patch(h, hsp, transformer_options): + scale = scale_dict.get(h.shape[1], None) + if scale is not None: + h[:,:h.shape[1] // 2] = h[:,:h.shape[1] // 2] * scale[0] + if hsp.device not in on_cpu_devices: + try: + hsp = Fourier_filter(hsp, threshold=1, scale=scale[1]) + except: + print("Device", hsp.device, "does not support the torch.fft functions used in the FreeU node, switching to CPU.") + on_cpu_devices[hsp.device] = True + hsp = Fourier_filter(hsp.cpu(), threshold=1, scale=scale[1]).to(hsp.device) + else: + hsp = Fourier_filter(hsp.cpu(), threshold=1, scale=scale[1]).to(hsp.device) + + return h, hsp + + m = model.clone() + m.set_model_output_block_patch(output_block_patch) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/FreeU_V2.md b/docs/md/Comfy/Nodes/FreeU_V2.md new file mode 100644 index 0000000000..1b8077c88c --- /dev/null +++ b/docs/md/Comfy/Nodes/FreeU_V2.md @@ -0,0 +1,88 @@ +# FreeU_V2 +## Documentation +- Class name: `FreeU_V2` +- Category: `model_patches` +- Output node: `False` + +The `FreeU_V2` node applies a patch to a model, modifying its output block to adjust the scale of certain layers based on predefined scale factors and applying a Fourier filter to the hidden state patch. This process is designed to enhance the model's output by manipulating the scale and frequency components of the hidden states. +## Input types +### Required +- **`model`** + - The model to which the patch will be applied. This patch modifies the model's behavior by adjusting the scale of its layers and applying Fourier filtering to enhance the output. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`b1`** + - A scale factor for adjusting the model's layers. It's part of the mechanism to enhance the model's output by scaling certain layers differently. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`b2`** + - Another scale factor for layer adjustment, working alongside `b1` to fine-tune the model's output enhancement. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`s1`** + - A scale parameter for the Fourier filter applied to the hidden state patch, influencing the frequency components manipulation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`s2`** + - Another scale parameter for the Fourier filter, working with `s1` to adjust the frequency components of the model's output. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The modified model with the applied patch, featuring adjusted layer scales and Fourier-filtered hidden states for enhanced output. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,UltimateSDUpscale,FaceDetailer,Anything Everywhere,Reroute,PatchModelAddDownscale,KSamplerAdvanced,VideoLinearCFGGuidance,IPAdapter,UltimateSDUpscaleNoUpscale` + + +## Source code +```python +class FreeU_V2: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "b1": ("FLOAT", {"default": 1.3, "min": 0.0, "max": 10.0, "step": 0.01}), + "b2": ("FLOAT", {"default": 1.4, "min": 0.0, "max": 10.0, "step": 0.01}), + "s1": ("FLOAT", {"default": 0.9, "min": 0.0, "max": 10.0, "step": 0.01}), + "s2": ("FLOAT", {"default": 0.2, "min": 0.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "model_patches" + + def patch(self, model, b1, b2, s1, s2): + model_channels = model.model.model_config.unet_config["model_channels"] + scale_dict = {model_channels * 4: (b1, s1), model_channels * 2: (b2, s2)} + on_cpu_devices = {} + + def output_block_patch(h, hsp, transformer_options): + scale = scale_dict.get(h.shape[1], None) + if scale is not None: + hidden_mean = h.mean(1).unsqueeze(1) + B = hidden_mean.shape[0] + hidden_max, _ = torch.max(hidden_mean.view(B, -1), dim=-1, keepdim=True) + hidden_min, _ = torch.min(hidden_mean.view(B, -1), dim=-1, keepdim=True) + hidden_mean = (hidden_mean - hidden_min.unsqueeze(2).unsqueeze(3)) / (hidden_max - hidden_min).unsqueeze(2).unsqueeze(3) + + h[:,:h.shape[1] // 2] = h[:,:h.shape[1] // 2] * ((scale[0] - 1 ) * hidden_mean + 1) + + if hsp.device not in on_cpu_devices: + try: + hsp = Fourier_filter(hsp, threshold=1, scale=scale[1]) + except: + print("Device", hsp.device, "does not support the torch.fft functions used in the FreeU node, switching to CPU.") + on_cpu_devices[hsp.device] = True + hsp = Fourier_filter(hsp.cpu(), threshold=1, scale=scale[1]).to(hsp.device) + else: + hsp = Fourier_filter(hsp.cpu(), threshold=1, scale=scale[1]).to(hsp.device) + + return h, hsp + + m = model.clone() + m.set_model_output_block_patch(output_block_patch) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/GLIGENLoader.md b/docs/md/Comfy/Nodes/GLIGENLoader.md new file mode 100644 index 0000000000..cba420c6ae --- /dev/null +++ b/docs/md/Comfy/Nodes/GLIGENLoader.md @@ -0,0 +1,41 @@ +# GLIGENLoader +## Documentation +- Class name: `GLIGENLoader` +- Category: `loaders` +- Output node: `False` + +The GLIGENLoader node is responsible for loading a GLIGEN model from a specified path. It utilizes the `load_gligen` function to retrieve the model based on the given name, making it available for further processing or application within the system. +## Input types +### Required +- **`gligen_name`** + - Specifies the name of the GLIGEN model to be loaded. This name is used to locate the model file within a predefined directory structure. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`gligen`** + - The loaded GLIGEN model, ready for use in generating or manipulating images. + - Python dtype: `Gligen` + - Comfy dtype: `GLIGEN` +## Usage tips +- Infra type: `CPU` +- Common nodes: `GLIGENTextBoxApply` + + +## Source code +```python +class GLIGENLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "gligen_name": (folder_paths.get_filename_list("gligen"), )}} + + RETURN_TYPES = ("GLIGEN",) + FUNCTION = "load_gligen" + + CATEGORY = "loaders" + + def load_gligen(self, gligen_name): + gligen_path = folder_paths.get_full_path("gligen", gligen_name) + gligen = comfy.sd.load_gligen(gligen_path) + return (gligen,) + +``` diff --git a/docs/md/Comfy/Nodes/GLIGENTextBoxApply.md b/docs/md/Comfy/Nodes/GLIGENTextBoxApply.md new file mode 100644 index 0000000000..280a6b5d14 --- /dev/null +++ b/docs/md/Comfy/Nodes/GLIGENTextBoxApply.md @@ -0,0 +1,85 @@ +# GLIGENTextBoxApply +## Documentation +- Class name: `GLIGENTextBoxApply` +- Category: `conditioning/gligen` +- Output node: `False` + +This node applies text-based conditioning to a given set of conditioning parameters using a GLIGEN text box model. It encodes the provided text using a CLIP model, then modifies the conditioning parameters to include position-based information derived from the encoded text, dimensions, and specified position. +## Input types +### Required +- **`conditioning_to`** + - The initial set of conditioning parameters to which the text-based conditioning will be applied. It serves as the base for further modifications. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`clip`** + - The CLIP model used for encoding the provided text into a format that can be utilized for conditioning. + - Python dtype: `CLIP` + - Comfy dtype: `CLIP` +- **`gligen_textbox_model`** + - The GLIGEN model responsible for generating text boxes based on the encoded text and specified dimensions. + - Python dtype: `GLIGEN` + - Comfy dtype: `GLIGEN` +- **`text`** + - The text to be encoded and applied as part of the conditioning. It supports multiline input. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`width`** + - The width of the text box in pixels. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - The height of the text box in pixels. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`x`** + - The x-coordinate of the top-left corner of the text box. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate of the top-left corner of the text box. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`conditioning`** + - The modified set of conditioning parameters, now including the text-based conditioning. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,ToDetailerPipe` + + +## Source code +```python +class GLIGENTextBoxApply: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning_to": ("CONDITIONING", ), + "clip": ("CLIP", ), + "gligen_textbox_model": ("GLIGEN", ), + "text": ("STRING", {"multiline": True}), + "width": ("INT", {"default": 64, "min": 8, "max": MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 64, "min": 8, "max": MAX_RESOLUTION, "step": 8}), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "append" + + CATEGORY = "conditioning/gligen" + + def append(self, conditioning_to, clip, gligen_textbox_model, text, width, height, x, y): + c = [] + cond, cond_pooled = clip.encode_from_tokens(clip.tokenize(text), return_pooled="unprojected") + for t in conditioning_to: + n = [t[0], t[1].copy()] + position_params = [(cond_pooled, height // 8, width // 8, y // 8, x // 8)] + prev = [] + if "gligen" in n[1]: + prev = n[1]['gligen'][2] + + n[1]['gligen'] = ("position", gligen_textbox_model, prev + position_params) + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/GrowMask.md b/docs/md/Comfy/Nodes/GrowMask.md new file mode 100644 index 0000000000..2885cbf387 --- /dev/null +++ b/docs/md/Comfy/Nodes/GrowMask.md @@ -0,0 +1,69 @@ +# GrowMask +## Documentation +- Class name: `GrowMask` +- Category: `mask` +- Output node: `False` + +The `GrowMask` node is designed to modify the size of a given mask by either expanding or contracting it, with an option to taper the corners for a smoother transition. This operation is useful for adjusting mask boundaries in image processing tasks. +## Input types +### Required +- **`mask`** + - The input mask to be modified. It serves as the base for the expansion or contraction operation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`expand`** + - Determines the number of pixels by which the mask will be expanded (positive value) or contracted (negative value). + - Python dtype: `int` + - Comfy dtype: `INT` +- **`tapered_corners`** + - A boolean flag that, when set to True, applies a tapering effect to the corners of the mask during the expansion or contraction process, creating a smoother transition. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +## Output types +- **`mask`** + - The modified mask after the expansion or contraction operation, potentially with tapered corners if specified. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `MaskBlur+,ImageCompositeMasked,MaskToImage,VAEEncodeForInpaint` + + +## Source code +```python +class GrowMask: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mask": ("MASK",), + "expand": ("INT", {"default": 0, "min": -MAX_RESOLUTION, "max": MAX_RESOLUTION, "step": 1}), + "tapered_corners": ("BOOLEAN", {"default": True}), + }, + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + + FUNCTION = "expand_mask" + + def expand_mask(self, mask, expand, tapered_corners): + c = 0 if tapered_corners else 1 + kernel = np.array([[c, 1, c], + [1, 1, 1], + [c, 1, c]]) + mask = mask.reshape((-1, mask.shape[-2], mask.shape[-1])) + out = [] + for m in mask: + output = m.numpy() + for _ in range(abs(expand)): + if expand < 0: + output = scipy.ndimage.grey_erosion(output, footprint=kernel) + else: + output = scipy.ndimage.grey_dilation(output, footprint=kernel) + output = torch.from_numpy(output) + out.append(output) + return (torch.stack(out, dim=0),) + +``` diff --git a/docs/md/Comfy/Nodes/HyperTile.md b/docs/md/Comfy/Nodes/HyperTile.md new file mode 100644 index 0000000000..044e8890ea --- /dev/null +++ b/docs/md/Comfy/Nodes/HyperTile.md @@ -0,0 +1,100 @@ +# HyperTile +## Documentation +- Class name: `HyperTile` +- Category: `model_patches` +- Output node: `False` + +The `HyperTile` node is designed to modify the attention mechanism of a given model to enhance its ability to handle images with varying resolutions and aspect ratios. It dynamically adjusts the tiling of the input based on the model's channel configuration and the specified tile size, aiming to optimize the processing of images for tasks that may benefit from attention to finer details or specific regions. This adjustment is made by partitioning the input into smaller, more manageable tiles, potentially improving the model's performance on detailed or high-resolution images. +## Input types +### Required +- **`model`** + - The model to be patched with the HyperTile functionality. This model is expected to have an attention mechanism that can be modified to incorporate the HyperTile strategy, enhancing its ability to process images with varying resolutions and aspect ratios. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`tile_size`** + - Specifies the base size for the tiles into which the input image is divided. This size influences how the input is partitioned and can affect the granularity of the attention mechanism's focus. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`swap_size`** + - Determines the threshold for swapping the dimensions of the tiles, which can be useful for optimizing the processing of images with certain aspect ratios. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`max_depth`** + - Defines the maximum depth for applying the tiling strategy. This parameter controls how deeply nested the tiling can be, potentially allowing for more detailed attention to specific regions of the input. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`scale_depth`** + - A boolean flag indicating whether to scale the depth of the tiling based on the model's channel configuration. This can help in adapting the tiling strategy to the specific characteristics of the model and the input. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +## Output types +- **`model`** + - The patched model with the HyperTile functionality applied. This model is now capable of handling images with varying resolutions and aspect ratios more effectively, thanks to the dynamic tiling strategy. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PatchModelAddDownscale,FreeU_V2` + + +## Source code +```python +class HyperTile: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "tile_size": ("INT", {"default": 256, "min": 1, "max": 2048}), + "swap_size": ("INT", {"default": 2, "min": 1, "max": 128}), + "max_depth": ("INT", {"default": 0, "min": 0, "max": 10}), + "scale_depth": ("BOOLEAN", {"default": False}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "model_patches" + + def patch(self, model, tile_size, swap_size, max_depth, scale_depth): + model_channels = model.model.model_config.unet_config["model_channels"] + + latent_tile_size = max(32, tile_size) // 8 + self.temp = None + + def hypertile_in(q, k, v, extra_options): + model_chans = q.shape[-2] + orig_shape = extra_options['original_shape'] + apply_to = [] + for i in range(max_depth + 1): + apply_to.append((orig_shape[-2] / (2 ** i)) * (orig_shape[-1] / (2 ** i))) + + if model_chans in apply_to: + shape = extra_options["original_shape"] + aspect_ratio = shape[-1] / shape[-2] + + hw = q.size(1) + h, w = round(math.sqrt(hw * aspect_ratio)), round(math.sqrt(hw / aspect_ratio)) + + factor = (2 ** apply_to.index(model_chans)) if scale_depth else 1 + nh = random_divisor(h, latent_tile_size * factor, swap_size) + nw = random_divisor(w, latent_tile_size * factor, swap_size) + + if nh * nw > 1: + q = rearrange(q, "b (nh h nw w) c -> (b nh nw) (h w) c", h=h // nh, w=w // nw, nh=nh, nw=nw) + self.temp = (nh, nw, h, w) + return q, k, v + + return q, k, v + def hypertile_out(out, extra_options): + if self.temp is not None: + nh, nw, h, w = self.temp + self.temp = None + out = rearrange(out, "(b nh nw) hw c -> b nh nw hw c", nh=nh, nw=nw) + out = rearrange(out, "b nh nw (h w) c -> b (nh h nw w) c", h=h // nh, w=w // nw) + return out + + + m = model.clone() + m.set_model_attn1_patch(hypertile_in) + m.set_model_attn1_output_patch(hypertile_out) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/HypernetworkLoader.md b/docs/md/Comfy/Nodes/HypernetworkLoader.md new file mode 100644 index 0000000000..9898b60bd1 --- /dev/null +++ b/docs/md/Comfy/Nodes/HypernetworkLoader.md @@ -0,0 +1,55 @@ +# HypernetworkLoader +## Documentation +- Class name: `HypernetworkLoader` +- Category: `loaders` +- Output node: `False` + +The `HypernetworkLoader` node is designed to load and apply a hypernetwork to a given model. It clones the original model, loads a hypernetwork patch based on the specified hypernetwork name and strength, and applies this patch to the attention layers of the cloned model. This process allows for dynamic modification of the model's behavior based on the hypernetwork, potentially enhancing its performance or adapting it to specific tasks. +## Input types +### Required +- **`model`** + - The original model to which the hypernetwork will be applied. This model is cloned to ensure that the original remains unchanged. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`hypernetwork_name`** + - The name of the hypernetwork to load. This name is used to retrieve the specific hypernetwork patch from a predefined directory. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`strength`** + - A multiplier for the hypernetwork's effect on the model's attention layers, allowing for fine-tuning of the hypernetwork's impact. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The cloned model with the hypernetwork patch applied to its attention layers. This modified model is expected to exhibit altered behavior based on the hypernetwork. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class HypernetworkLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "hypernetwork_name": (folder_paths.get_filename_list("hypernetworks"), ), + "strength": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "load_hypernetwork" + + CATEGORY = "loaders" + + def load_hypernetwork(self, model, hypernetwork_name, strength): + hypernetwork_path = folder_paths.get_full_path("hypernetworks", hypernetwork_name) + model_hypernetwork = model.clone() + patch = load_hypernetwork_patch(hypernetwork_path, strength) + if patch is not None: + model_hypernetwork.set_model_attn1_patch(patch) + model_hypernetwork.set_model_attn2_patch(patch) + return (model_hypernetwork,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageBatch.md b/docs/md/Comfy/Nodes/ImageBatch.md new file mode 100644 index 0000000000..6fde94b678 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageBatch.md @@ -0,0 +1,47 @@ +# Batch Images +## Documentation +- Class name: `ImageBatch` +- Category: `image` +- Output node: `False` + +The `ImageBatch` node combines two images into a single batch. If the images have different sizes, it resizes the second image to match the first one's dimensions using bilinear interpolation before combining them. +## Input types +### Required +- **`image1`** + - The first image to be included in the batch. It serves as the reference for the size to which the second image will be resized if necessary. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`image2`** + - The second image to be included in the batch. It is resized to match the first image's dimensions if they differ, using bilinear interpolation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +- **`image`** + - The combined batch of the two input images. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ImageBatch,IPAdapterApply,CR Batch Process Switch,PreviewImage,Preview Chooser` + + +## Source code +```python +class ImageBatch: + + @classmethod + def INPUT_TYPES(s): + return {"required": { "image1": ("IMAGE",), "image2": ("IMAGE",)}} + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "batch" + + CATEGORY = "image" + + def batch(self, image1, image2): + if image1.shape[1:] != image2.shape[1:]: + image2 = comfy.utils.common_upscale(image2.movedim(-1,1), image1.shape[2], image1.shape[1], "bilinear", "center").movedim(1,-1) + s = torch.cat((image1, image2), dim=0) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageBlend.md b/docs/md/Comfy/Nodes/ImageBlend.md new file mode 100644 index 0000000000..395c682f41 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageBlend.md @@ -0,0 +1,94 @@ +# ImageBlend +## Documentation +- Class name: `ImageBlend` +- Category: `image/postprocessing` +- Output node: `False` + +The `ImageBlend` node blends two images together based on a specified blend mode and blend factor. It supports various blend modes such as normal, multiply, screen, overlay, soft light, and difference. The blend factor determines the weight of each image in the final blend, allowing for a seamless mix of the two images. This node is useful for image post-processing tasks where combining images with different effects is required. +## Input types +### Required +- **`image1`** + - The first image to be blended. It serves as the base layer onto which the second image is blended. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`image2`** + - The second image to be blended. This image is combined with the first image according to the specified blend mode and factor. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`blend_factor`** + - Determines the weight of each image in the final blend. A higher blend factor gives more weight to the second image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`blend_mode`** + - Specifies the method of blending the two images. Supports various modes like normal, multiply, screen, overlay, soft light, and difference. + - Python dtype: `str` + - Comfy dtype: `['normal', 'multiply', 'screen', 'overlay', 'soft_light', 'difference']` +## Output types +- **`image`** + - The result of blending the two images together based on the specified blend mode and factor. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ImageCombine,Image Aspect Ratio,ImageSender,Mute / Bypass Repeater (rgthree)` + + +## Source code +```python +class Blend: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image1": ("IMAGE",), + "image2": ("IMAGE",), + "blend_factor": ("FLOAT", { + "default": 0.5, + "min": 0.0, + "max": 1.0, + "step": 0.01 + }), + "blend_mode": (["normal", "multiply", "screen", "overlay", "soft_light", "difference"],), + }, + } + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "blend_images" + + CATEGORY = "image/postprocessing" + + def blend_images(self, image1: torch.Tensor, image2: torch.Tensor, blend_factor: float, blend_mode: str): + image2 = image2.to(image1.device) + if image1.shape != image2.shape: + image2 = image2.permute(0, 3, 1, 2) + image2 = comfy.utils.common_upscale(image2, image1.shape[2], image1.shape[1], upscale_method='bicubic', crop='center') + image2 = image2.permute(0, 2, 3, 1) + + blended_image = self.blend_mode(image1, image2, blend_mode) + blended_image = image1 * (1 - blend_factor) + blended_image * blend_factor + blended_image = torch.clamp(blended_image, 0, 1) + return (blended_image,) + + def blend_mode(self, img1, img2, mode): + if mode == "normal": + return img2 + elif mode == "multiply": + return img1 * img2 + elif mode == "screen": + return 1 - (1 - img1) * (1 - img2) + elif mode == "overlay": + return torch.where(img1 <= 0.5, 2 * img1 * img2, 1 - 2 * (1 - img1) * (1 - img2)) + elif mode == "soft_light": + return torch.where(img2 <= 0.5, img1 - (1 - 2 * img2) * img1 * (1 - img1), img1 + (2 * img2 - 1) * (self.g(img1) - img1)) + elif mode == "difference": + return img1 - img2 + else: + raise ValueError(f"Unsupported blend mode: {mode}") + + def g(self, x): + return torch.where(x <= 0.25, ((16 * x - 12) * x + 4) * x, torch.sqrt(x)) + +``` diff --git a/docs/md/Comfy/Nodes/ImageBlur.md b/docs/md/Comfy/Nodes/ImageBlur.md new file mode 100644 index 0000000000..19cb52d356 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageBlur.md @@ -0,0 +1,79 @@ +# ImageBlur +## Documentation +- Class name: `ImageBlur` +- Category: `image/postprocessing` +- Output node: `False` + +The ImageBlur node applies a Gaussian blur to an input image. It uses a Gaussian kernel to smooth the image, which can be adjusted by specifying the blur radius and sigma value. This process can reduce image noise and detail, making it useful for post-processing effects or preparing images for further processing steps. +## Input types +### Required +- **`image`** + - The input image to be blurred. This is the primary target for the blur effect. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`blur_radius`** + - Determines the size of the Gaussian kernel used for blurring. A larger radius increases the blur effect. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`sigma`** + - Controls the spread of the blur. A higher sigma value results in a smoother, more widespread blur. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The blurred version of the input image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ImageUpscaleWithModel,Cut By Mask` + + +## Source code +```python +class Blur: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "blur_radius": ("INT", { + "default": 1, + "min": 1, + "max": 31, + "step": 1 + }), + "sigma": ("FLOAT", { + "default": 1.0, + "min": 0.1, + "max": 10.0, + "step": 0.1 + }), + }, + } + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "blur" + + CATEGORY = "image/postprocessing" + + def blur(self, image: torch.Tensor, blur_radius: int, sigma: float): + if blur_radius == 0: + return (image,) + + batch_size, height, width, channels = image.shape + + kernel_size = blur_radius * 2 + 1 + kernel = gaussian_kernel(kernel_size, sigma, device=image.device).repeat(channels, 1, 1).unsqueeze(1) + + image = image.permute(0, 3, 1, 2) # Torch wants (B, C, H, W) we use (B, H, W, C) + padded_image = F.pad(image, (blur_radius,blur_radius,blur_radius,blur_radius), 'reflect') + blurred = F.conv2d(padded_image, kernel, padding=kernel_size // 2, groups=channels)[:,:,blur_radius:-blur_radius, blur_radius:-blur_radius] + blurred = blurred.permute(0, 2, 3, 1) + + return (blurred,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageColorToMask.md b/docs/md/Comfy/Nodes/ImageColorToMask.md new file mode 100644 index 0000000000..b3f830d63e --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageColorToMask.md @@ -0,0 +1,51 @@ +# ImageColorToMask +## Documentation +- Class name: `ImageColorToMask` +- Category: `mask` +- Output node: `False` + +This node converts an image to a mask based on a specified color. It first normalizes the image's color values, then creates a mask by comparing each pixel's color to the specified color, resulting in a binary mask where matching pixels are marked. +## Input types +### Required +- **`image`** + - The input image to be converted into a mask. The image is normalized and compared against the specified color to generate the mask. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`color`** + - The target color used to generate the mask. Pixels in the image that match this color will be included in the mask. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`mask`** + - The output binary mask generated from the input image, where pixels matching the specified color are marked. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ImageColorToMask: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "color": ("INT", {"default": 0, "min": 0, "max": 0xFFFFFF, "step": 1, "display": "color"}), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + FUNCTION = "image_to_mask" + + def image_to_mask(self, image, color): + temp = (torch.clamp(image, 0, 1.0) * 255.0).round().to(torch.int) + temp = torch.bitwise_left_shift(temp[:,:,:,0], 16) + torch.bitwise_left_shift(temp[:,:,:,1], 8) + temp[:,:,:,2] + mask = torch.where(temp == color, 255, 0).float() + return (mask,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageCompositeMasked.md b/docs/md/Comfy/Nodes/ImageCompositeMasked.md new file mode 100644 index 0000000000..e91f4efddc --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageCompositeMasked.md @@ -0,0 +1,72 @@ +# ImageCompositeMasked +## Documentation +- Class name: `ImageCompositeMasked` +- Category: `image` +- Output node: `False` + +The `ImageCompositeMasked` node is designed for compositing two images together at specified coordinates, optionally resizing the source image and applying a mask to control the blending regions. This operation is fundamental in image editing and processing tasks where merging different image layers or elements is required. +## Input types +### Required +- **`destination`** + - The base image onto which the source image will be composited. It serves as the background layer in the compositing process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`source`** + - The image to be composited onto the destination image. It acts as the foreground layer and can be optionally resized to fit the destination image's context. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`x`** + - The x-coordinate on the destination image where the top-left corner of the source image will be placed. It determines the horizontal positioning of the source image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate on the destination image where the top-left corner of the source image will be placed. It determines the vertical positioning of the source image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`resize_source`** + - A boolean flag indicating whether the source image should be resized to match the destination image's dimensions. This allows for flexible compositing of images of different sizes. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +### Optional +- **`mask`** + - An optional mask that defines which parts of the source image should be visible when composited onto the destination image. This allows for selective blending and more complex compositing effects. + - Python dtype: `Optional[torch.Tensor]` + - Comfy dtype: `MASK` +## Output types +- **`image`** + - The resulting image after compositing the source image onto the destination image, with optional resizing and masking applied. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ImageInvert,PreviewImage,ImageCompositeMasked,ControlNetApplyAdvanced,VAEEncode,ColorCorrect` + + +## Source code +```python +class ImageCompositeMasked: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "destination": ("IMAGE",), + "source": ("IMAGE",), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "resize_source": ("BOOLEAN", {"default": False}), + }, + "optional": { + "mask": ("MASK",), + } + } + RETURN_TYPES = ("IMAGE",) + FUNCTION = "composite" + + CATEGORY = "image" + + def composite(self, destination, source, x, y, resize_source, mask = None): + destination = destination.clone().movedim(-1, 1) + output = composite(destination, source.movedim(-1, 1), x, y, mask, 1, resize_source).movedim(1, -1) + return (output,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageCrop.md b/docs/md/Comfy/Nodes/ImageCrop.md new file mode 100644 index 0000000000..9d15310ae9 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageCrop.md @@ -0,0 +1,64 @@ +# ImageCrop +## Documentation +- Class name: `ImageCrop` +- Category: `image/transform` +- Output node: `False` + +The `ImageCrop` node is designed to crop a given image to a specified width and height starting from a point (x, y). This operation is fundamental in image processing tasks where focusing on a specific region of an image or reducing the image size is required. +## Input types +### Required +- **`image`** + - The input image to be cropped. This parameter is crucial as it defines the source image from which a region will be extracted. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`width`** + - Specifies the width of the cropped image. It determines how wide the resulting image will be. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Specifies the height of the cropped image. It determines the vertical size of the resulting image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`x`** + - The x-coordinate of the top-left corner of the crop box. It defines where the cropping should start horizontally. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate of the top-left corner of the crop box. It defines where the cropping should start vertically. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - The cropped image. This output is the result of the cropping operation, containing only the specified region of the original image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEEncodeForInpaint,PreviewImage` + + +## Source code +```python +class ImageCrop: + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",), + "width": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + "height": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "crop" + + CATEGORY = "image/transform" + + def crop(self, image, width, height, x, y): + x = min(x, image.shape[2] - 1) + y = min(y, image.shape[1] - 1) + to_x = width + x + to_y = height + y + img = image[:,y:to_y, x:to_x, :] + return (img,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageFromBatch.md b/docs/md/Comfy/Nodes/ImageFromBatch.md new file mode 100644 index 0000000000..24d6980c18 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageFromBatch.md @@ -0,0 +1,53 @@ +# ImageFromBatch +## Documentation +- Class name: `ImageFromBatch` +- Category: `image/batch` +- Output node: `False` + +The `ImageFromBatch` node extracts a specific range of images from a batch based on the provided batch index and length. It allows for selective retrieval of images from a larger collection, facilitating operations like image processing on subsets of a batch. +## Input types +### Required +- **`image`** + - The batch of images from which a subset will be extracted. This parameter is crucial for specifying the source of images. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`batch_index`** + - The starting index within the batch from which the extraction begins. This parameter determines the starting point of the subset. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`length`** + - The number of images to extract from the batch, starting from the batch index. This parameter defines the size of the subset to be extracted. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - The extracted subset of images from the specified batch. This output is useful for further processing or analysis of specific images within a larger batch. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ImageFromBatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",), + "batch_index": ("INT", {"default": 0, "min": 0, "max": 63}), + "length": ("INT", {"default": 1, "min": 1, "max": 64}), + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "frombatch" + + CATEGORY = "image/batch" + + def frombatch(self, image, batch_index, length): + s_in = image + batch_index = min(s_in.shape[0] - 1, batch_index) + length = min(s_in.shape[0] - batch_index, length) + s = s_in[batch_index:batch_index + length].clone() + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageInvert.md b/docs/md/Comfy/Nodes/ImageInvert.md new file mode 100644 index 0000000000..16fdff650e --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageInvert.md @@ -0,0 +1,41 @@ +# Invert Image +## Documentation +- Class name: `ImageInvert` +- Category: `image` +- Output node: `False` + +The `ImageInvert` node inverts the colors of an input image by subtracting each color value from 1. This operation effectively reverses the color scheme of the image, turning light areas dark and vice versa. +## Input types +### Required +- **`image`** + - The input image to be inverted. The inversion process changes the color values of the image to their complementary values, affecting the overall appearance. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +- **`image`** + - The inverted image with its colors reversed. This output is useful for creating visual contrasts or for preprocessing steps in image processing pipelines. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PreviewImage,ImageScale,ImageCompositeMasked,Reroute,ArithmeticBlend` + + +## Source code +```python +class ImageInvert: + + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",)}} + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "invert" + + CATEGORY = "image" + + def invert(self, image): + s = 1.0 - image + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageOnlyCheckpointLoader.md b/docs/md/Comfy/Nodes/ImageOnlyCheckpointLoader.md new file mode 100644 index 0000000000..effffac446 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageOnlyCheckpointLoader.md @@ -0,0 +1,49 @@ +# Image Only Checkpoint Loader (img2vid model) +## Documentation +- Class name: `ImageOnlyCheckpointLoader` +- Category: `loaders/video_models` +- Output node: `False` + +This node is designed to load checkpoints specifically for image-based models in video generation tasks. It selectively loads the model, CLIP vision, and VAE components from a given checkpoint, allowing for a streamlined setup in scenarios where only image processing capabilities are required. +## Input types +### Required +- **`ckpt_name`** + - Specifies the name of the checkpoint to load. This parameter is crucial for identifying the correct file from which to load the model, CLIP vision, and VAE components. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - The main model loaded from the checkpoint, typically used for generating or manipulating images. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip_vision`** + - The CLIP vision component loaded from the checkpoint, useful for tasks involving image understanding and manipulation. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP_VISION` +- **`vae`** + - The VAE component loaded from the checkpoint, often used for encoding and decoding images in generative tasks. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `SVD_img2vid_Conditioning,VAEDecode,VideoLinearCFGGuidance,Reroute,StableZero123_Conditioning,LoraLoader,Anything Everywhere,KSampler,workflow/front` + + +## Source code +```python +class ImageOnlyCheckpointLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "ckpt_name": (folder_paths.get_filename_list("checkpoints"), ), + }} + RETURN_TYPES = ("MODEL", "CLIP_VISION", "VAE") + FUNCTION = "load_checkpoint" + + CATEGORY = "loaders/video_models" + + def load_checkpoint(self, ckpt_name, output_vae=True, output_clip=True): + ckpt_path = folder_paths.get_full_path("checkpoints", ckpt_name) + out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=False, output_clipvision=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) + return (out[0], out[3], out[2]) + +``` diff --git a/docs/md/Comfy/Nodes/ImageOnlyCheckpointSave.md b/docs/md/Comfy/Nodes/ImageOnlyCheckpointSave.md new file mode 100644 index 0000000000..9a6150a1d9 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageOnlyCheckpointSave.md @@ -0,0 +1,50 @@ +# ImageOnlyCheckpointSave +## Documentation +- Class name: `ImageOnlyCheckpointSave` +- Category: `_for_testing` +- Output node: `True` + +This node specializes in saving model checkpoints specifically for image-based models. It extends the functionality of a general checkpoint saving mechanism to include additional components like CLIP vision and VAE models, tailored for image processing tasks. The node allows for customization of the saved checkpoint's filename and supports optional inclusion of a prompt and extra PNG information. +## Input types +### Required +- **`model`** + - The primary model whose state is to be saved. This is central to the checkpoint saving process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip_vision`** + - A CLIP vision model component to be included in the checkpoint. This is important for tasks that involve understanding or generating images in the context of natural language descriptions. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP_VISION` +- **`vae`** + - A VAE (Variational Autoencoder) model component to be included in the checkpoint. VAEs are often used in image processing for generating new images or modifying existing ones. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`filename_prefix`** + - The prefix for the filename under which the checkpoint will be saved. This allows for organizing and identifying checkpoints easily. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class ImageOnlyCheckpointSave(comfy_extras.nodes_model_merging.CheckpointSave): + CATEGORY = "_for_testing" + + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "clip_vision": ("CLIP_VISION",), + "vae": ("VAE",), + "filename_prefix": ("STRING", {"default": "checkpoints/ComfyUI"}),}, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},} + + def save(self, model, clip_vision, vae, filename_prefix, prompt=None, extra_pnginfo=None): + comfy_extras.nodes_model_merging.save_checkpoint(model, clip_vision=clip_vision, vae=vae, filename_prefix=filename_prefix, output_dir=self.output_dir, prompt=prompt, extra_pnginfo=extra_pnginfo) + return {} + +``` diff --git a/docs/md/Comfy/Nodes/ImagePadForOutpaint.md b/docs/md/Comfy/Nodes/ImagePadForOutpaint.md new file mode 100644 index 0000000000..31cf2f7eb5 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImagePadForOutpaint.md @@ -0,0 +1,113 @@ +# Pad Image for Outpainting +## Documentation +- Class name: `ImagePadForOutpaint` +- Category: `image` +- Output node: `False` + +The ImagePadForOutpaint node is designed to prepare images for the outpainting process by padding them. This operation is crucial for expanding the canvas of an image, allowing for the generation or extension of image content beyond its original borders. +## Input types +### Required +- **`image`** + - The original image to be padded for outpainting. This image serves as the base for the outpainting process, where additional content will be generated around its borders. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`left`** + - Specifies the amount of padding to be applied to the left side of the image, influencing the extent of the outpainting area. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`top`** + - Specifies the amount of padding to be applied to the top side of the image, influencing the extent of the outpainting area. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`right`** + - Specifies the amount of padding to be applied to the right side of the image, influencing the extent of the outpainting area. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`bottom`** + - Specifies the amount of padding to be applied to the bottom side of the image, influencing the extent of the outpainting area. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`feathering`** + - Determines the smoothness of the transition between the original image and the padded area, enhancing the visual integration of the outpainting. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - The resulting image after padding has been applied. This image has an expanded canvas ready for outpainting. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`mask`** + - The mask indicating the original and padded areas, useful for distinguishing between them in subsequent processing steps. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ImagePadForOutpaint: + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "left": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "top": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "right": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "bottom": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "feathering": ("INT", {"default": 40, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + } + } + + RETURN_TYPES = ("IMAGE", "MASK") + FUNCTION = "expand_image" + + CATEGORY = "image" + + def expand_image(self, image, left, top, right, bottom, feathering): + d1, d2, d3, d4 = image.size() + + new_image = torch.ones( + (d1, d2 + top + bottom, d3 + left + right, d4), + dtype=torch.float32, + ) * 0.5 + + new_image[:, top:top + d2, left:left + d3, :] = image + + mask = torch.ones( + (d2 + top + bottom, d3 + left + right), + dtype=torch.float32, + ) + + t = torch.zeros( + (d2, d3), + dtype=torch.float32 + ) + + if feathering > 0 and feathering * 2 < d2 and feathering * 2 < d3: + + for i in range(d2): + for j in range(d3): + dt = i if top != 0 else d2 + db = d2 - i if bottom != 0 else d2 + + dl = j if left != 0 else d3 + dr = d3 - j if right != 0 else d3 + + d = min(dt, db, dl, dr) + + if d >= feathering: + continue + + v = (feathering - d) / feathering + + t[i, j] = v * v + + mask[top:top + d2, left:left + d3] = t + + return (new_image, mask) + +``` diff --git a/docs/md/Comfy/Nodes/ImageQuantize.md b/docs/md/Comfy/Nodes/ImageQuantize.md new file mode 100644 index 0000000000..cf777ca651 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageQuantize.md @@ -0,0 +1,105 @@ +# ImageQuantize +## Documentation +- Class name: `ImageQuantize` +- Category: `image/postprocessing` +- Output node: `False` + +The `ImageQuantize` node performs color quantization on an input image tensor, reducing the number of colors to a specified amount. This process can include dithering to reduce color banding, with support for different dithering techniques including none, Floyd-Steinberg, and Bayer dithering. +## Input types +### Required +- **`image`** + - The input image tensor to be quantized. The quantization process reduces the color palette of this image to the specified number of colors, potentially applying dithering to improve visual quality. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`colors`** + - Specifies the number of colors to reduce the input image to during the quantization process. This directly impacts the visual fidelity and the size of the resulting image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`dither`** + - Determines the dithering technique to be applied during quantization, which can help in reducing color banding effects. Supported options include 'none', 'floyd-steinberg', and various orders of 'bayer' dithering. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`image`** + - The quantized image tensor with reduced color palette. This output maintains the original dimensions of the input image but with a limited number of colors. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class Quantize: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "colors": ("INT", { + "default": 256, + "min": 1, + "max": 256, + "step": 1 + }), + "dither": (["none", "floyd-steinberg", "bayer-2", "bayer-4", "bayer-8", "bayer-16"],), + }, + } + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "quantize" + + CATEGORY = "image/postprocessing" + + def bayer(im, pal_im, order): + def normalized_bayer_matrix(n): + if n == 0: + return np.zeros((1,1), "float32") + else: + q = 4 ** n + m = q * normalized_bayer_matrix(n - 1) + return np.bmat(((m-1.5, m+0.5), (m+1.5, m-0.5))) / q + + num_colors = len(pal_im.getpalette()) // 3 + spread = 2 * 256 / num_colors + bayer_n = int(math.log2(order)) + bayer_matrix = torch.from_numpy(spread * normalized_bayer_matrix(bayer_n) + 0.5) + + result = torch.from_numpy(np.array(im).astype(np.float32)) + tw = math.ceil(result.shape[0] / bayer_matrix.shape[0]) + th = math.ceil(result.shape[1] / bayer_matrix.shape[1]) + tiled_matrix = bayer_matrix.tile(tw, th).unsqueeze(-1) + result.add_(tiled_matrix[:result.shape[0],:result.shape[1]]).clamp_(0, 255) + result = result.to(dtype=torch.uint8) + + im = Image.fromarray(result.cpu().numpy()) + im = im.quantize(palette=pal_im, dither=Image.Dither.NONE) + return im + + def quantize(self, image: torch.Tensor, colors: int, dither: str): + batch_size, height, width, _ = image.shape + result = torch.zeros_like(image) + + for b in range(batch_size): + im = Image.fromarray((image[b] * 255).to(torch.uint8).numpy(), mode='RGB') + + pal_im = im.quantize(colors=colors) # Required as described in https://github.com/python-pillow/Pillow/issues/5836 + + if dither == "none": + quantized_image = im.quantize(palette=pal_im, dither=Image.Dither.NONE) + elif dither == "floyd-steinberg": + quantized_image = im.quantize(palette=pal_im, dither=Image.Dither.FLOYDSTEINBERG) + elif dither.startswith("bayer"): + order = int(dither.split('-')[-1]) + quantized_image = Quantize.bayer(im, pal_im, order) + + quantized_array = torch.tensor(np.array(quantized_image.convert("RGB"))).float() / 255 + result[b] = quantized_array + + return (result,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageScale.md b/docs/md/Comfy/Nodes/ImageScale.md new file mode 100644 index 0000000000..d637942a6f --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageScale.md @@ -0,0 +1,72 @@ +# Upscale Image +## Documentation +- Class name: `ImageScale` +- Category: `image/upscaling` +- Output node: `False` + +The `ImageScale` node is designed for upscaling images to a specified width and height, with the option to crop the upscaled image. It supports various upscaling methods to adjust the image quality and size according to the user's needs. +## Input types +### Required +- **`image`** + - The input image to be upscaled. This is the primary data that the node operates on. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`upscale_method`** + - Specifies the method used for upscaling the image. Different methods can affect the quality and characteristics of the upscaled image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`width`** + - The target width of the upscaled image. If set to 0, the width is automatically calculated to maintain the aspect ratio based on the specified height. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - The target height of the upscaled image. If set to 0, the height is automatically calculated to maintain the aspect ratio based on the specified width. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`crop`** + - Determines whether and how the upscaled image should be cropped. This can be used to focus on a specific area of the image or to fit a particular aspect ratio. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`image`** + - The upscaled (and optionally cropped) version of the input image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEEncode,FILM VFI,LineArtPreprocessor,ControlNetApplyAdvanced,PreviewImage,VAEEncodeTiled,StableZero123_Conditioning,OpenposePreprocessor,IPAdapterApply,Image Rembg (Remove Background)` + + +## Source code +```python +class ImageScale: + upscale_methods = ["nearest-exact", "bilinear", "area", "bicubic", "lanczos"] + crop_methods = ["disabled", "center"] + + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",), "upscale_method": (s.upscale_methods,), + "width": ("INT", {"default": 512, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "height": ("INT", {"default": 512, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "crop": (s.crop_methods,)}} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "upscale" + + CATEGORY = "image/upscaling" + + def upscale(self, image, upscale_method, width, height, crop): + if width == 0 and height == 0: + s = image + else: + samples = image.movedim(-1,1) + + if width == 0: + width = max(1, round(samples.shape[3] * height / samples.shape[2])) + elif height == 0: + height = max(1, round(samples.shape[2] * width / samples.shape[3])) + + s = comfy.utils.common_upscale(samples, width, height, upscale_method, crop) + s = s.movedim(1,-1) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageScaleBy.md b/docs/md/Comfy/Nodes/ImageScaleBy.md new file mode 100644 index 0000000000..3fec19d882 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageScaleBy.md @@ -0,0 +1,54 @@ +# Upscale Image By +## Documentation +- Class name: `ImageScaleBy` +- Category: `image/upscaling` +- Output node: `False` + +The `ImageScaleBy` node is designed for upscaling images by a specified scale factor. It adjusts the dimensions of the input image based on the provided scale factor, using one of the specified upscale methods to maintain image quality. +## Input types +### Required +- **`image`** + - The input image to be upscaled. This parameter is crucial as it directly influences the output image's resolution and appearance. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`upscale_method`** + - Determines the algorithm used for upscaling the image. Different methods can affect the quality and characteristics of the upscaled image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`scale_by`** + - The factor by which the image's dimensions will be increased. Affects the final size of the upscaled image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The upscaled image, with dimensions adjusted according to the specified scale factor and upscale method. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEEncode,RIFE VFI,VHS_VideoCombine,VAEEncodeTiled,Reroute,PreviewImage,SaveImage,SVD_img2vid_Conditioning,SEGSPreview,ImageFilterSharpen` + + +## Source code +```python +class ImageScaleBy: + upscale_methods = ["nearest-exact", "bilinear", "area", "bicubic", "lanczos"] + + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",), "upscale_method": (s.upscale_methods,), + "scale_by": ("FLOAT", {"default": 1.0, "min": 0.01, "max": 8.0, "step": 0.01}),}} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "upscale" + + CATEGORY = "image/upscaling" + + def upscale(self, image, upscale_method, scale_by): + samples = image.movedim(-1,1) + width = round(samples.shape[3] * scale_by) + height = round(samples.shape[2] * scale_by) + s = comfy.utils.common_upscale(samples, width, height, upscale_method, "disabled") + s = s.movedim(1,-1) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageScaleToTotalPixels.md b/docs/md/Comfy/Nodes/ImageScaleToTotalPixels.md new file mode 100644 index 0000000000..0759d809be --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageScaleToTotalPixels.md @@ -0,0 +1,60 @@ +# ImageScaleToTotalPixels +## Documentation +- Class name: `ImageScaleToTotalPixels` +- Category: `image/upscaling` +- Output node: `False` + +This node scales an image to a specified total number of pixels (megapixels) while maintaining the aspect ratio. It uses various upscale methods to achieve the desired size. +## Input types +### Required +- **`image`** + - The input image to be scaled. The choice of upscale method and the target megapixels directly influence the quality and dimensions of the output image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`upscale_method`** + - The method used for upscaling the image. Different methods can affect the quality and characteristics of the upscaled image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`megapixels`** + - The target size of the output image in megapixels. This determines the total number of pixels in the scaled image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The scaled image with the specified total number of pixels. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `GetImageSize,ImageBatch,DWPreprocessor,VAEEncodeForInpaint,PreviewImage,ImpactImageInfo,Reroute,ControlNetApply,VAEEncode` + + +## Source code +```python +class ImageScaleToTotalPixels: + upscale_methods = ["nearest-exact", "bilinear", "area", "bicubic", "lanczos"] + crop_methods = ["disabled", "center"] + + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",), "upscale_method": (s.upscale_methods,), + "megapixels": ("FLOAT", {"default": 1.0, "min": 0.01, "max": 16.0, "step": 0.01}), + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "upscale" + + CATEGORY = "image/upscaling" + + def upscale(self, image, upscale_method, megapixels): + samples = image.movedim(-1,1) + total = int(megapixels * 1024 * 1024) + + scale_by = math.sqrt(total / (samples.shape[3] * samples.shape[2])) + width = round(samples.shape[3] * scale_by) + height = round(samples.shape[2] * scale_by) + + s = comfy.utils.common_upscale(samples, width, height, upscale_method, "disabled") + s = s.movedim(1,-1) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageSharpen.md b/docs/md/Comfy/Nodes/ImageSharpen.md new file mode 100644 index 0000000000..f76beee7ed --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageSharpen.md @@ -0,0 +1,94 @@ +# ImageSharpen +## Documentation +- Class name: `ImageSharpen` +- Category: `image/postprocessing` +- Output node: `False` + +The `ImageSharpen` node applies a sharpening filter to an image using a Gaussian kernel. It enhances the edges and details within the image by convolving it with a generated kernel based on the specified sharpen radius, sigma, and alpha parameters. The process involves adjusting the kernel to ensure the sum of its elements equals one, applying padding to the image, and then performing a 2D convolution. The result is clamped to the range [0, 1] to maintain valid image values. +## Input types +### Required +- **`image`** + - The input image to be sharpened. It is a tensor representing the image data. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`sharpen_radius`** + - Specifies the radius of the sharpening effect. A larger radius increases the area of influence for the sharpening, affecting more pixels around the edges. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`sigma`** + - Controls the spread of the Gaussian kernel used for sharpening. A higher sigma value results in a smoother, less localized sharpening effect. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`alpha`** + - Determines the strength of the sharpening effect. Higher alpha values produce a more pronounced sharpening. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The sharpened version of the input image, with enhanced edges and details. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PreviewImage,IPAdapterApplyFaceID,IPAdapterApply,ImageUpscaleWithModel,UltimateSDUpscale,VAEEncode` + + +## Source code +```python +class Sharpen: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "sharpen_radius": ("INT", { + "default": 1, + "min": 1, + "max": 31, + "step": 1 + }), + "sigma": ("FLOAT", { + "default": 1.0, + "min": 0.1, + "max": 10.0, + "step": 0.1 + }), + "alpha": ("FLOAT", { + "default": 1.0, + "min": 0.0, + "max": 5.0, + "step": 0.1 + }), + }, + } + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "sharpen" + + CATEGORY = "image/postprocessing" + + def sharpen(self, image: torch.Tensor, sharpen_radius: int, sigma:float, alpha: float): + if sharpen_radius == 0: + return (image,) + + batch_size, height, width, channels = image.shape + + kernel_size = sharpen_radius * 2 + 1 + kernel = gaussian_kernel(kernel_size, sigma, device=image.device) * -(alpha*10) + center = kernel_size // 2 + kernel[center, center] = kernel[center, center] - kernel.sum() + 1.0 + kernel = kernel.repeat(channels, 1, 1).unsqueeze(1) + + tensor_image = image.permute(0, 3, 1, 2) # Torch wants (B, C, H, W) we use (B, H, W, C) + tensor_image = F.pad(tensor_image, (sharpen_radius,sharpen_radius,sharpen_radius,sharpen_radius), 'reflect') + sharpened = F.conv2d(tensor_image, kernel, padding=center, groups=channels)[:,:,sharpen_radius:-sharpen_radius, sharpen_radius:-sharpen_radius] + sharpened = sharpened.permute(0, 2, 3, 1) + + result = torch.clamp(sharpened, 0, 1) + + return (result,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageToMask.md b/docs/md/Comfy/Nodes/ImageToMask.md new file mode 100644 index 0000000000..d2762db0e0 --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageToMask.md @@ -0,0 +1,50 @@ +# Convert Image to Mask +## Documentation +- Class name: `ImageToMask` +- Category: `mask` +- Output node: `False` + +This node converts an image into a mask based on a specified color channel (red, green, blue, or alpha). It isolates the selected channel from the input image to create a mask, which can be used for various image processing tasks that require a binary or grayscale representation of the image. +## Input types +### Required +- **`image`** + - The input image from which the mask will be created. The choice of channel affects the resulting mask, highlighting different features based on the channel's color spectrum. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`channel`** + - Specifies the color channel (red, green, blue, or alpha) to be used for mask creation. This choice determines which aspects of the image are emphasized in the resulting mask. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`mask`** + - The output mask generated from the specified color channel of the input image. This mask can be used for further image manipulation or analysis. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `GPU` +- Common nodes: `MaskComposite,GrowMask,IPAdapterApplyFaceID` + + +## Source code +```python +class ImageToMask: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "channel": (["red", "green", "blue", "alpha"],), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + FUNCTION = "image_to_mask" + + def image_to_mask(self, image, channel): + channels = ["red", "green", "blue", "alpha"] + mask = image[:, :, :, channels.index(channel)] + return (mask,) + +``` diff --git a/docs/md/Comfy/Nodes/ImageUpscaleWithModel.md b/docs/md/Comfy/Nodes/ImageUpscaleWithModel.md new file mode 100644 index 0000000000..eb111c579d --- /dev/null +++ b/docs/md/Comfy/Nodes/ImageUpscaleWithModel.md @@ -0,0 +1,66 @@ +# Upscale Image (using Model) +## Documentation +- Class name: `ImageUpscaleWithModel` +- Category: `image/upscaling` +- Output node: `False` + +This node is designed for upscaling images using a specified upscale model. It dynamically adjusts the tiling to manage memory efficiently and avoid out-of-memory errors, ensuring the upscale process can handle images of various sizes. +## Input types +### Required +- **`upscale_model`** + - The upscale model to be used for image upscaling. It's crucial for defining the upscaling algorithm and its parameters. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `UPSCALE_MODEL` +- **`image`** + - The input image to be upscaled. This image is processed and upscaled according to the specified model. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +- **`image`** + - The upscaled image, processed according to the upscale model and the dynamic tiling strategy to manage memory usage. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ImageScaleBy,ImageScale,SaveImage,VHS_VideoCombine,PreviewImage,Reroute,ImageScaleToTotalPixels,FaceDetailer,CR Image Output,ImageCASharpening+` + + +## Source code +```python +class ImageUpscaleWithModel: + @classmethod + def INPUT_TYPES(s): + return {"required": { "upscale_model": ("UPSCALE_MODEL",), + "image": ("IMAGE",), + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "upscale" + + CATEGORY = "image/upscaling" + + def upscale(self, upscale_model, image): + device = model_management.get_torch_device() + upscale_model.to(device) + in_img = image.movedim(-1,-3).to(device) + free_memory = model_management.get_free_memory(device) + + tile = 512 + overlap = 32 + + oom = True + while oom: + try: + steps = in_img.shape[0] * comfy.utils.get_tiled_scale_steps(in_img.shape[3], in_img.shape[2], tile_x=tile, tile_y=tile, overlap=overlap) + pbar = comfy.utils.ProgressBar(steps) + s = comfy.utils.tiled_scale(in_img, lambda a: upscale_model(a), tile_x=tile, tile_y=tile, overlap=overlap, upscale_amount=upscale_model.scale, pbar=pbar) + oom = False + except model_management.OOM_EXCEPTION as e: + tile //= 2 + if tile < 128: + raise e + + upscale_model.cpu() + s = torch.clamp(s.movedim(-3,-1), min=0, max=1.0) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/InpaintModelConditioning.md b/docs/md/Comfy/Nodes/InpaintModelConditioning.md new file mode 100644 index 0000000000..b0ec8ed286 --- /dev/null +++ b/docs/md/Comfy/Nodes/InpaintModelConditioning.md @@ -0,0 +1,100 @@ +# InpaintModelConditioning +## Documentation +- Class name: `InpaintModelConditioning` +- Category: `conditioning/inpaint` +- Output node: `False` + +The InpaintModelConditioning node is designed for conditioning the inpainting process by encoding the input parameters into a format suitable for the model. It takes in positive and negative conditioning, a VAE model, an image, and a mask to produce conditioning outputs and a latent representation. +## Input types +### Required +- **`positive`** + - Positive conditioning influences the inpainting process by specifying desired features or attributes in the generated output. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`negative`** + - Negative conditioning specifies undesired features or attributes, helping to guide the model away from certain outcomes. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`vae`** + - The VAE model parameter is crucial for encoding the image into a latent space, facilitating the inpainting process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`pixels`** + - The image to be inpainted, provided as pixel data. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`mask`** + - A mask indicating the areas to inpaint, with the same dimensions as the input image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`conditioning`** + - The modified negative conditioning after processing. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`latent`** + - A latent representation of the inpainted image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class InpaintModelConditioning: + @classmethod + def INPUT_TYPES(s): + return {"required": {"positive": ("CONDITIONING", ), + "negative": ("CONDITIONING", ), + "vae": ("VAE", ), + "pixels": ("IMAGE", ), + "mask": ("MASK", ), + }} + + RETURN_TYPES = ("CONDITIONING","CONDITIONING","LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "encode" + + CATEGORY = "conditioning/inpaint" + + def encode(self, positive, negative, pixels, vae, mask): + x = (pixels.shape[1] // 8) * 8 + y = (pixels.shape[2] // 8) * 8 + mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(pixels.shape[1], pixels.shape[2]), mode="bilinear") + + orig_pixels = pixels + pixels = orig_pixels.clone() + if pixels.shape[1] != x or pixels.shape[2] != y: + x_offset = (pixels.shape[1] % 8) // 2 + y_offset = (pixels.shape[2] % 8) // 2 + pixels = pixels[:,x_offset:x + x_offset, y_offset:y + y_offset,:] + mask = mask[:,:,x_offset:x + x_offset, y_offset:y + y_offset] + + m = (1.0 - mask.round()).squeeze(1) + for i in range(3): + pixels[:,:,:,i] -= 0.5 + pixels[:,:,:,i] *= m + pixels[:,:,:,i] += 0.5 + concat_latent = vae.encode(pixels) + orig_latent = vae.encode(orig_pixels) + + out_latent = {} + + out_latent["samples"] = orig_latent + out_latent["noise_mask"] = mask + + out = [] + for conditioning in [positive, negative]: + c = [] + for t in conditioning: + d = t[1].copy() + d["concat_latent_image"] = concat_latent + d["concat_mask"] = mask + n = [t[0], d] + c.append(n) + out.append(c) + return (out[0], out[1], out_latent) + +``` diff --git a/docs/md/Comfy/Nodes/InvertMask.md b/docs/md/Comfy/Nodes/InvertMask.md new file mode 100644 index 0000000000..0b13b4368b --- /dev/null +++ b/docs/md/Comfy/Nodes/InvertMask.md @@ -0,0 +1,45 @@ +# InvertMask +## Documentation +- Class name: `InvertMask` +- Category: `mask` +- Output node: `False` + +The `InvertMask` node inverts the values of a given mask, effectively transforming the masked areas into unmasked areas and vice versa. This operation is useful in scenarios where the focus of interest in an image needs to be reversed. +## Input types +### Required +- **`mask`** + - The input mask to be inverted. This mask defines areas of interest that will be transformed to their opposite values. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`mask`** + - The inverted mask, where the originally masked areas are now unmasked, and the originally unmasked areas are now masked. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `MaskToImage,PorterDuffImageComposite,ImageCompositeMasked,VAEEncodeForInpaint,FeatherMask` + + +## Source code +```python +class InvertMask: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mask": ("MASK",), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + + FUNCTION = "invert" + + def invert(self, mask): + out = 1.0 - mask + return (out,) + +``` diff --git a/docs/md/Comfy/Nodes/JoinImageWithAlpha.md b/docs/md/Comfy/Nodes/JoinImageWithAlpha.md new file mode 100644 index 0000000000..904dae84c2 --- /dev/null +++ b/docs/md/Comfy/Nodes/JoinImageWithAlpha.md @@ -0,0 +1,55 @@ +# Join Image with Alpha +## Documentation +- Class name: `JoinImageWithAlpha` +- Category: `mask/compositing` +- Output node: `False` + +This node combines an image and an alpha mask to produce a single image with transparency information. It adjusts the alpha mask to match the image dimensions and merges them, ensuring the output image retains the original colors with the specified transparency. +## Input types +### Required +- **`image`** + - The primary image to which the alpha mask will be applied. It defines the visual content of the output. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`alpha`** + - The alpha mask that specifies the transparency level for each pixel in the image. It is adjusted to match the image dimensions before application. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`image`** + - The output is a single image that combines the input image with the alpha mask, incorporating transparency information. + - Python dtype: `Tuple[torch.Tensor]` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class JoinImageWithAlpha: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + "alpha": ("MASK",), + } + } + + CATEGORY = "mask/compositing" + RETURN_TYPES = ("IMAGE",) + FUNCTION = "join_image_with_alpha" + + def join_image_with_alpha(self, image: torch.Tensor, alpha: torch.Tensor): + batch_size = min(len(image), len(alpha)) + out_images = [] + + alpha = 1.0 - resize_mask(alpha, image.shape[1:]) + for i in range(batch_size): + out_images.append(torch.cat((image[i][:,:,:3], alpha[i].unsqueeze(2)), dim=2)) + + result = (torch.stack(out_images),) + return result + +``` diff --git a/docs/md/Comfy/Nodes/KSampler.md b/docs/md/Comfy/Nodes/KSampler.md new file mode 100644 index 0000000000..ca12539351 --- /dev/null +++ b/docs/md/Comfy/Nodes/KSampler.md @@ -0,0 +1,87 @@ +# KSampler +## Documentation +- Class name: `KSampler` +- Category: `sampling` +- Output node: `False` + +The `KSampler` node is designed for sampling operations using a variety of samplers and schedulers. It allows for the generation of new samples based on the provided model, seed, steps, and other parameters. This node is capable of handling different types of conditioning, adjusting the noise level, and utilizing specific samplers and schedulers to influence the generation process. +## Input types +### Required +- **`model`** + - Specifies the model to be used for sampling. The choice of model affects the quality and characteristics of the generated samples. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`seed`** + - Determines the random seed for generating samples, ensuring reproducibility of results. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`steps`** + - Defines the number of steps to be taken in the sampling process, impacting the detail and quality of the generated samples. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`cfg`** + - Controls the conditioning factor, influencing the diversity and fidelity of the generated samples. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sampler_name`** + - Selects the specific sampler to be used, affecting the sampling strategy and outcome. + - Python dtype: `str` + - Comfy dtype: `['euler', 'euler_ancestral', 'heun'...]` +- **`scheduler`** + - Chooses the scheduler for controlling the sampling process, impacting the progression and quality of samples. + - Python dtype: `str` + - Comfy dtype: `['normal', 'karras', 'exponential'...]` +- **`positive`** + - Provides positive conditioning to guide the sampling towards desired attributes. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `CONDITIONING` +- **`negative`** + - Provides negative conditioning to steer the sampling away from undesired attributes. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `CONDITIONING` +- **`latent_image`** + - The initial latent image to be modified through the sampling process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +- **`denoise`** + - Adjusts the level of denoising applied to the samples, affecting their clarity and detail. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`latent`** + - The generated samples after the sampling process, represented in latent space. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEDecode,LatentUpscaleBy,VAEDecodeTiled,Reroute,LatentComposite,NNLatentUpscale,LatentUpscale,KSampler,workflow/front,SetNode` + + +## Source code +```python +class KSampler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}), + "steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}), + "sampler_name": (comfy.samplers.KSampler.SAMPLERS, ), + "scheduler": (comfy.samplers.KSampler.SCHEDULERS, ), + "positive": ("CONDITIONING", ), + "negative": ("CONDITIONING", ), + "latent_image": ("LATENT", ), + "denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + } + } + + RETURN_TYPES = ("LATENT",) + FUNCTION = "sample" + + CATEGORY = "sampling" + + def sample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=1.0): + return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) + +``` diff --git a/docs/md/Comfy/Nodes/KSamplerAdvanced.md b/docs/md/Comfy/Nodes/KSamplerAdvanced.md new file mode 100644 index 0000000000..511d60a951 --- /dev/null +++ b/docs/md/Comfy/Nodes/KSamplerAdvanced.md @@ -0,0 +1,108 @@ +# KSampler (Advanced) +## Documentation +- Class name: `KSamplerAdvanced` +- Category: `sampling` +- Output node: `False` + +The `KSamplerAdvanced` node appears to be a conceptual or placeholder entity, as there is no direct implementation or method details provided in the context. Typically, such a node might represent an advanced version of a sampling mechanism, potentially offering more sophisticated or customizable sampling strategies compared to its base version. However, without specific implementation details, it's not possible to provide a precise analysis of its functionality. +## Input types +### Required +- **`model`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `MODEL` +- **`add_noise`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `['enable', 'disable'...]` +- **`noise_seed`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`steps`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`cfg`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`sampler_name`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `['euler', 'euler_ancestral', 'heun'...]` +- **`scheduler`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `['normal', 'karras', 'exponential'...]` +- **`positive`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `CONDITIONING` +- **`negative`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `CONDITIONING` +- **`latent_image`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `LATENT` +- **`start_at_step`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`end_at_step`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`return_with_leftover_noise`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `['disable', 'enable'...]` +## Output types +- **`latent`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `CPU` +- Common nodes: `VAEDecode,Reroute,workflow/KSampler + Vae,gcLatentTunnel,Anything Everywhere,LatentCompositeMasked,KSampler,LatentBlend,Mute / Bypass Repeater (rgthree)` + + +## Source code +```python +class KSamplerAdvanced: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "add_noise": (["enable", "disable"], ), + "noise_seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}), + "steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}), + "sampler_name": (comfy.samplers.KSampler.SAMPLERS, ), + "scheduler": (comfy.samplers.KSampler.SCHEDULERS, ), + "positive": ("CONDITIONING", ), + "negative": ("CONDITIONING", ), + "latent_image": ("LATENT", ), + "start_at_step": ("INT", {"default": 0, "min": 0, "max": 10000}), + "end_at_step": ("INT", {"default": 10000, "min": 0, "max": 10000}), + "return_with_leftover_noise": (["disable", "enable"], ), + } + } + + RETURN_TYPES = ("LATENT",) + FUNCTION = "sample" + + CATEGORY = "sampling" + + def sample(self, model, add_noise, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, start_at_step, end_at_step, return_with_leftover_noise, denoise=1.0): + force_full_denoise = True + if return_with_leftover_noise == "enable": + force_full_denoise = False + disable_noise = False + if add_noise == "disable": + disable_noise = True + return common_ksampler(model, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise) + +``` diff --git a/docs/md/Comfy/Nodes/KSamplerSelect.md b/docs/md/Comfy/Nodes/KSamplerSelect.md new file mode 100644 index 0000000000..850e35c975 --- /dev/null +++ b/docs/md/Comfy/Nodes/KSamplerSelect.md @@ -0,0 +1,42 @@ +# KSamplerSelect +## Documentation +- Class name: `KSamplerSelect` +- Category: `sampling/custom_sampling/samplers` +- Output node: `False` + +The `KSamplerSelect` node is designed to select a specific sampler object based on the provided sampler name. It facilitates the dynamic selection of sampling strategies within the broader sampling framework, allowing for flexible experimentation and optimization of sampling processes. +## Input types +### Required +- **`sampler_name`** + - Specifies the name of the sampler to be selected. This parameter enables the dynamic selection of different sampling strategies, allowing users to tailor the sampling process to specific requirements or preferences. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`sampler`** + - Returns the selected sampler object, which can be used for further sampling operations. This enables the dynamic and flexible use of different sampling strategies within the application. + - Python dtype: `comfy.samplers.Sampler` + - Comfy dtype: `SAMPLER` +## Usage tips +- Infra type: `CPU` +- Common nodes: `SamplerCustom,Reroute` + + +## Source code +```python +class KSamplerSelect: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"sampler_name": (comfy.samplers.SAMPLER_NAMES, ), + } + } + RETURN_TYPES = ("SAMPLER",) + CATEGORY = "sampling/custom_sampling/samplers" + + FUNCTION = "get_sampler" + + def get_sampler(self, sampler_name): + sampler = comfy.samplers.sampler_object(sampler_name) + return (sampler, ) + +``` diff --git a/docs/md/Comfy/Nodes/KarrasScheduler.md b/docs/md/Comfy/Nodes/KarrasScheduler.md new file mode 100644 index 0000000000..22237f8301 --- /dev/null +++ b/docs/md/Comfy/Nodes/KarrasScheduler.md @@ -0,0 +1,57 @@ +# KarrasScheduler +## Documentation +- Class name: `KarrasScheduler` +- Category: `sampling/custom_sampling/schedulers` +- Output node: `False` + +The KarrasScheduler node is designed to generate a sequence of noise levels (sigmas) based on the Karras et al. (2022) noise schedule. This sequence is used in diffusion models to gradually add or remove noise from images during the sampling process. +## Input types +### Required +- **`steps`** + - Specifies the number of steps in the noise schedule. A higher number of steps results in a more gradual transition of noise levels. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`sigma_max`** + - The maximum noise level in the schedule. It defines the starting point of noise addition or the end point of noise removal. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sigma_min`** + - The minimum noise level in the schedule. It defines the end point of noise addition or the starting point of noise removal. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`rho`** + - A parameter that controls the shape of the noise schedule curve, affecting how quickly or slowly the noise levels change. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`sigmas`** + - A tensor of noise levels (sigmas) generated according to the Karras et al. (2022) noise schedule. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class KarrasScheduler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "sigma_max": ("FLOAT", {"default": 14.614642, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + "sigma_min": ("FLOAT", {"default": 0.0291675, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + "rho": ("FLOAT", {"default": 7.0, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/schedulers" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, steps, sigma_max, sigma_min, rho): + sigmas = k_diffusion_sampling.get_sigmas_karras(n=steps, sigma_min=sigma_min, sigma_max=sigma_max, rho=rho) + return (sigmas, ) + +``` diff --git a/docs/md/Comfy/Nodes/LatentAdd.md b/docs/md/Comfy/Nodes/LatentAdd.md new file mode 100644 index 0000000000..fc57808b77 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentAdd.md @@ -0,0 +1,50 @@ +# LatentAdd +## Documentation +- Class name: `LatentAdd` +- Category: `latent/advanced` +- Output node: `False` + +The `LatentAdd` node performs element-wise addition of two latent representations. This operation can be used to combine features or effects from two different latent spaces. +## Input types +### Required +- **`samples1`** + - The first set of latent representations to be added. It serves as the base to which the second set is added. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`samples2`** + - The second set of latent representations to be added to the first. It is reshaped to match the first set's dimensions if necessary. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Output types +- **`latent`** + - The result of element-wise addition of the two input latent representations. It combines the features or effects from both inputs. + - Python dtype: `Tuple[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentAdd: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples1": ("LATENT",), "samples2": ("LATENT",)}} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "op" + + CATEGORY = "latent/advanced" + + def op(self, samples1, samples2): + samples_out = samples1.copy() + + s1 = samples1["samples"] + s2 = samples2["samples"] + + s2 = reshape_latent_to(s1.shape, s2) + samples_out["samples"] = s1 + s2 + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentBatch.md b/docs/md/Comfy/Nodes/LatentBatch.md new file mode 100644 index 0000000000..af73084752 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentBatch.md @@ -0,0 +1,52 @@ +# LatentBatch +## Documentation +- Class name: `LatentBatch` +- Category: `latent/batch` +- Output node: `False` + +The `LatentBatch` node combines two sets of latent representations (samples) into a single batch. It ensures that the dimensions of the two input samples match, potentially resizing one of them, before concatenating them along the batch dimension. Additionally, it manages batch indices for the combined samples. +## Input types +### Required +- **`samples1`** + - The first set of latent representations to be combined. It is crucial for constructing the output batch by providing the initial part of the combined samples. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`samples2`** + - The second set of latent representations to be combined with the first. It may be resized to match the dimensions of `samples1` before concatenation, contributing to the latter part of the combined samples. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Output types +- **`latent`** + - The combined set of latent representations from `samples1` and `samples2`, concatenated along the batch dimension. It includes updated batch indices reflecting the new combined batch. + - Python dtype: `Tuple[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentBatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples1": ("LATENT",), "samples2": ("LATENT",)}} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "batch" + + CATEGORY = "latent/batch" + + def batch(self, samples1, samples2): + samples_out = samples1.copy() + s1 = samples1["samples"] + s2 = samples2["samples"] + + if s1.shape[1:] != s2.shape[1:]: + s2 = comfy.utils.common_upscale(s2, s1.shape[3], s1.shape[2], "bilinear", "center") + s = torch.cat((s1, s2), dim=0) + samples_out["samples"] = s + samples_out["batch_index"] = samples1.get("batch_index", [x for x in range(0, s1.shape[0])]) + samples2.get("batch_index", [x for x in range(0, s2.shape[0])]) + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentBatchSeedBehavior.md b/docs/md/Comfy/Nodes/LatentBatchSeedBehavior.md new file mode 100644 index 0000000000..66962fa219 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentBatchSeedBehavior.md @@ -0,0 +1,53 @@ +# LatentBatchSeedBehavior +## Documentation +- Class name: `LatentBatchSeedBehavior` +- Category: `latent/advanced` +- Output node: `False` + +This node modifies the batch index of a given set of latent samples based on the specified seed behavior. It supports either a fixed or random seed behavior, which determines how the batch index is adjusted or removed to ensure consistency or variability in batch processing. +## Input types +### Required +- **`samples`** + - The latent samples to be processed. This input is crucial for determining the structure and content of the output based on the seed behavior. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`seed_behavior`** + - Determines whether the batch index should be adjusted in a fixed manner or removed for randomness. This choice affects how the samples are grouped or individualized during processing. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`latent`** + - The modified set of latent samples, with adjustments to the batch index based on the seed behavior. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class LatentBatchSeedBehavior: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "seed_behavior": (["random", "fixed"],{"default": "fixed"}),}} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "op" + + CATEGORY = "latent/advanced" + + def op(self, samples, seed_behavior): + samples_out = samples.copy() + latent = samples["samples"] + if seed_behavior == "random": + if 'batch_index' in samples_out: + samples_out.pop('batch_index') + elif seed_behavior == "fixed": + batch_number = samples_out.get("batch_index", [0])[0] + samples_out["batch_index"] = [batch_number] * latent.shape[0] + + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentBlend.md b/docs/md/Comfy/Nodes/LatentBlend.md new file mode 100644 index 0000000000..ae08886ddd --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentBlend.md @@ -0,0 +1,75 @@ +# Latent Blend +## Documentation +- Class name: `LatentBlend` +- Category: `_for_testing` +- Output node: `False` + +The `LatentBlend` node blends two sets of latent representations (samples) based on a specified blend factor. It supports different blending modes to combine the features of the input samples in various ways, allowing for flexible manipulation of latent spaces. +## Input types +### Required +- **`samples1`** + - The first set of latent representations to be blended. It plays a crucial role in determining the base features of the output. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`samples2`** + - The second set of latent representations to be blended with the first. It contributes to the final blend by adding or modifying features based on the blend mode and factor. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`blend_factor`** + - A value between 0 and 1 that determines the weight of each input sample in the final blend. A higher blend factor gives more weight to the features of the first set of samples. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`latent`** + - The blended latent representations, combining features from both input sets according to the blend factor and mode. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `NNLatentUpscale,SamplerCustom,KSamplerAdvanced,Mute / Bypass Repeater (rgthree)` + + +## Source code +```python +class LatentBlend: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "samples1": ("LATENT",), + "samples2": ("LATENT",), + "blend_factor": ("FLOAT", { + "default": 0.5, + "min": 0, + "max": 1, + "step": 0.01 + }), + }} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "blend" + + CATEGORY = "_for_testing" + + def blend(self, samples1, samples2, blend_factor:float, blend_mode: str="normal"): + + samples_out = samples1.copy() + samples1 = samples1["samples"] + samples2 = samples2["samples"] + + if samples1.shape != samples2.shape: + samples2.permute(0, 3, 1, 2) + samples2 = comfy.utils.common_upscale(samples2, samples1.shape[3], samples1.shape[2], 'bicubic', crop='center') + samples2.permute(0, 2, 3, 1) + + samples_blended = self.blend_mode(samples1, samples2, blend_mode) + samples_blended = samples1 * blend_factor + samples_blended * (1 - blend_factor) + samples_out["samples"] = samples_blended + return (samples_out,) + + def blend_mode(self, img1, img2, mode): + if mode == "normal": + return img2 + else: + raise ValueError(f"Unsupported blend mode: {mode}") + +``` diff --git a/docs/md/Comfy/Nodes/LatentComposite.md b/docs/md/Comfy/Nodes/LatentComposite.md new file mode 100644 index 0000000000..44dddffae2 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentComposite.md @@ -0,0 +1,84 @@ +# Latent Composite +## Documentation +- Class name: `LatentComposite` +- Category: `latent` +- Output node: `False` + +This node is designed to perform a composite operation on latent representations. It combines the 'destination' latent image with the 'source' latent image at specified coordinates, optionally resizing the source to match the destination's resolution. This operation is useful for blending or overlaying different latent images to create complex or composite images. +## Input types +### Required +- **`samples_to`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `LATENT` +- **`samples_from`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `LATENT` +- **`x`** + - The x-coordinate (horizontal position) where the source latent image will be placed on the destination. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate (vertical position) where the source latent image will be placed on the destination. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`feather`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The resulting latent representation after the composite operation. It combines elements of both the source and destination latent images. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `LatentComposite,KSampler` + + +## Source code +```python +class LatentComposite: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples_to": ("LATENT",), + "samples_from": ("LATENT",), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "feather": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "composite" + + CATEGORY = "latent" + + def composite(self, samples_to, samples_from, x, y, composite_method="normal", feather=0): + x = x // 8 + y = y // 8 + feather = feather // 8 + samples_out = samples_to.copy() + s = samples_to["samples"].clone() + samples_to = samples_to["samples"] + samples_from = samples_from["samples"] + if feather == 0: + s[:,:,y:y+samples_from.shape[2],x:x+samples_from.shape[3]] = samples_from[:,:,:samples_to.shape[2] - y, :samples_to.shape[3] - x] + else: + samples_from = samples_from[:,:,:samples_to.shape[2] - y, :samples_to.shape[3] - x] + mask = torch.ones_like(samples_from) + for t in range(feather): + if y != 0: + mask[:,:,t:1+t,:] *= ((1.0/feather) * (t + 1)) + + if y + samples_from.shape[2] < samples_to.shape[2]: + mask[:,:,mask.shape[2] -1 -t: mask.shape[2]-t,:] *= ((1.0/feather) * (t + 1)) + if x != 0: + mask[:,:,:,t:1+t] *= ((1.0/feather) * (t + 1)) + if x + samples_from.shape[3] < samples_to.shape[3]: + mask[:,:,:,mask.shape[3]- 1 - t: mask.shape[3]- t] *= ((1.0/feather) * (t + 1)) + rev_mask = torch.ones_like(mask) - mask + s[:,:,y:y+samples_from.shape[2],x:x+samples_from.shape[3]] = samples_from[:,:,:samples_to.shape[2] - y, :samples_to.shape[3] - x] * mask + s[:,:,y:y+samples_from.shape[2],x:x+samples_from.shape[3]] * rev_mask + samples_out["samples"] = s + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentCompositeMasked.md b/docs/md/Comfy/Nodes/LatentCompositeMasked.md new file mode 100644 index 0000000000..9af7a749e5 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentCompositeMasked.md @@ -0,0 +1,74 @@ +# LatentCompositeMasked +## Documentation +- Class name: `LatentCompositeMasked` +- Category: `latent` +- Output node: `False` + +The `LatentCompositeMasked` node combines two latent representations (destination and source) at specified coordinates, optionally resizing the source and applying a mask to the composite operation. This node is useful for blending or overlaying features from one latent space onto another, potentially with selective masking. +## Input types +### Required +- **`destination`** + - The latent representation onto which the source will be composited. Acts as the background for the composite operation. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`source`** + - The latent representation that will be composited onto the destination. Acts as the overlay in the composite operation. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`x`** + - The x-coordinate in the destination latent space where the source will be placed. Allows for precise positioning of the source. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate in the destination latent space where the source will be placed. Allows for precise positioning of the source. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`resize_source`** + - Determines whether the source latent representation should be resized to match the destination's dimensions before compositing. Enables flexibility in combining latents of different sizes. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +### Optional +- **`mask`** + - An optional mask that can be applied to the composite operation, allowing for selective blending of the source onto the destination. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`latent`** + - The resulting latent representation after compositing the source onto the destination, potentially with resizing and masking. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSamplerAdvanced` + + +## Source code +```python +class LatentCompositeMasked: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "destination": ("LATENT",), + "source": ("LATENT",), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "resize_source": ("BOOLEAN", {"default": False}), + }, + "optional": { + "mask": ("MASK",), + } + } + RETURN_TYPES = ("LATENT",) + FUNCTION = "composite" + + CATEGORY = "latent" + + def composite(self, destination, source, x, y, resize_source, mask = None): + output = destination.copy() + destination = destination["samples"].clone() + source = source["samples"] + output["samples"] = composite(destination, source, x, y, mask, 8, resize_source) + return (output,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentCrop.md b/docs/md/Comfy/Nodes/LatentCrop.md new file mode 100644 index 0000000000..6ca05fcb4b --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentCrop.md @@ -0,0 +1,75 @@ +# Crop Latent +## Documentation +- Class name: `LatentCrop` +- Category: `latent/transform` +- Output node: `False` + +The `LatentCrop` node performs cropping operations on latent representations of images. It adjusts the dimensions of the given latent samples to the specified width and height, starting from the (x, y) coordinates. +## Input types +### Required +- **`samples`** + - The latent representation of images to be cropped. This parameter is crucial for defining the input data on which the cropping operation will be performed. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`width`** + - The desired width of the cropped latent representation. It determines how wide the output latent image will be. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - The desired height of the cropped latent representation. It affects the vertical dimension of the output latent image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`x`** + - The x-coordinate from where the cropping should start horizontally. It defines the left boundary of the crop area. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The y-coordinate from where the cropping should start vertically. It defines the top boundary of the crop area. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The cropped latent representation of images. This output retains the essential features of the input latent samples within the specified crop area. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentCrop: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "width": ("INT", {"default": 512, "min": 64, "max": MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 512, "min": 64, "max": MAX_RESOLUTION, "step": 8}), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "crop" + + CATEGORY = "latent/transform" + + def crop(self, samples, width, height, x, y): + s = samples.copy() + samples = samples['samples'] + x = x // 8 + y = y // 8 + + #enfonce minimum size of 64 + if x > (samples.shape[3] - 8): + x = samples.shape[3] - 8 + if y > (samples.shape[2] - 8): + y = samples.shape[2] - 8 + + new_height = height // 8 + new_width = width // 8 + to_x = new_width + x + to_y = new_height + y + s['samples'] = samples[:,:,y:to_y, x:to_x] + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentFlip.md b/docs/md/Comfy/Nodes/LatentFlip.md new file mode 100644 index 0000000000..fcf1e78d3a --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentFlip.md @@ -0,0 +1,50 @@ +# Flip Latent +## Documentation +- Class name: `LatentFlip` +- Category: `latent/transform` +- Output node: `False` + +The `LatentFlip` node performs a flip transformation on latent representations of images. Depending on the specified method, it can flip the images either vertically or horizontally. This operation is useful for data augmentation or to correct the orientation of images in the latent space. +## Input types +### Required +- **`samples`** + - The latent representation of images to be flipped. This input is crucial as it directly determines the data that will undergo the transformation. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`flip_method`** + - Specifies the axis along which the images in the latent space will be flipped. This parameter controls the direction of the flip, allowing for vertical or horizontal flipping. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`latent`** + - The flipped latent representation of images. This output is the result of applying the specified flip transformation to the input latent images. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentFlip: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "flip_method": (["x-axis: vertically", "y-axis: horizontally"],), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "flip" + + CATEGORY = "latent/transform" + + def flip(self, samples, flip_method): + s = samples.copy() + if flip_method.startswith("x"): + s["samples"] = torch.flip(samples["samples"], dims=[2]) + elif flip_method.startswith("y"): + s["samples"] = torch.flip(samples["samples"], dims=[3]) + + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentFromBatch.md b/docs/md/Comfy/Nodes/LatentFromBatch.md new file mode 100644 index 0000000000..a9a22610a8 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentFromBatch.md @@ -0,0 +1,66 @@ +# Latent From Batch +## Documentation +- Class name: `LatentFromBatch` +- Category: `latent/batch` +- Output node: `False` + +This node extracts a specific batch of samples from a larger batch of latent representations based on the provided batch index and length. It supports handling of noise masks associated with the samples, ensuring that the extracted batch retains the corresponding noise masks. +## Input types +### Required +- **`samples`** + - The latent representations from which a specific batch is to be extracted. This parameter is crucial for selecting the subset of data to work with. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`batch_index`** + - The index of the first sample in the batch to be extracted. This parameter determines the starting point of the subset. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`length`** + - The number of samples to extract from the batch, starting from the batch index. This parameter defines the size of the subset. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The extracted batch of latent representations, including any associated noise masks and batch indices. + - Python dtype: `Tuple[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `CPU` +- Common nodes: `LatentBlend` + + +## Source code +```python +class LatentFromBatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "batch_index": ("INT", {"default": 0, "min": 0, "max": 63}), + "length": ("INT", {"default": 1, "min": 1, "max": 64}), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "frombatch" + + CATEGORY = "latent/batch" + + def frombatch(self, samples, batch_index, length): + s = samples.copy() + s_in = samples["samples"] + batch_index = min(s_in.shape[0] - 1, batch_index) + length = min(s_in.shape[0] - batch_index, length) + s["samples"] = s_in[batch_index:batch_index + length].clone() + if "noise_mask" in samples: + masks = samples["noise_mask"] + if masks.shape[0] == 1: + s["noise_mask"] = masks.clone() + else: + if masks.shape[0] < s_in.shape[0]: + masks = masks.repeat(math.ceil(s_in.shape[0] / masks.shape[0]), 1, 1, 1)[:s_in.shape[0]] + s["noise_mask"] = masks[batch_index:batch_index + length].clone() + if "batch_index" not in s: + s["batch_index"] = [x for x in range(batch_index, batch_index+length)] + else: + s["batch_index"] = samples["batch_index"][batch_index:batch_index + length] + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentInterpolate.md b/docs/md/Comfy/Nodes/LatentInterpolate.md new file mode 100644 index 0000000000..706ef7f31a --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentInterpolate.md @@ -0,0 +1,68 @@ +# LatentInterpolate +## Documentation +- Class name: `LatentInterpolate` +- Category: `latent/advanced` +- Output node: `False` + +The `LatentInterpolate` node performs interpolation between two sets of latent samples based on a specified ratio. It normalizes the samples, computes a weighted average based on the ratio, and scales the result according to the original magnitudes. +## Input types +### Required +- **`samples1`** + - The first set of latent samples to interpolate from. It serves as the base for the interpolation process. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`samples2`** + - The second set of latent samples to interpolate towards. It is combined with the first set based on the interpolation ratio. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`ratio`** + - A floating-point value that determines the weight of each set of samples in the interpolation. A ratio of 0 yields the first set, while a ratio of 1 yields the second set. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`latent`** + - The interpolated latent samples, combining features from both input sets according to the specified ratio. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler (Efficient),Latent Noise Injection` + + +## Source code +```python +class LatentInterpolate: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples1": ("LATENT",), + "samples2": ("LATENT",), + "ratio": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "op" + + CATEGORY = "latent/advanced" + + def op(self, samples1, samples2, ratio): + samples_out = samples1.copy() + + s1 = samples1["samples"] + s2 = samples2["samples"] + + s2 = reshape_latent_to(s1.shape, s2) + + m1 = torch.linalg.vector_norm(s1, dim=(1)) + m2 = torch.linalg.vector_norm(s2, dim=(1)) + + s1 = torch.nan_to_num(s1 / m1) + s2 = torch.nan_to_num(s2 / m2) + + t = (s1 * ratio + s2 * (1.0 - ratio)) + mt = torch.linalg.vector_norm(t, dim=(1)) + st = torch.nan_to_num(t / mt) + + samples_out["samples"] = st * (m1 * ratio + m2 * (1.0 - ratio)) + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentMultiply.md b/docs/md/Comfy/Nodes/LatentMultiply.md new file mode 100644 index 0000000000..4069f90eef --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentMultiply.md @@ -0,0 +1,49 @@ +# LatentMultiply +## Documentation +- Class name: `LatentMultiply` +- Category: `latent/advanced` +- Output node: `False` + +This node multiplies each element in a given latent representation by a specified multiplier. It's useful for adjusting the intensity or scale of features within the latent space. +## Input types +### Required +- **`samples`** + - The latent representation to be modified. This input is crucial for defining the features or characteristics of the data that will be scaled. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`multiplier`** + - A scalar value by which each element of the latent representation is multiplied. This allows for scaling the intensity of the latent features. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`latent`** + - The modified latent representation after each of its elements has been scaled by the multiplier. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentMultiply: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "multiplier": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + }} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "op" + + CATEGORY = "latent/advanced" + + def op(self, samples, multiplier): + samples_out = samples.copy() + + s1 = samples["samples"] + samples_out["samples"] = s1 * multiplier + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentRotate.md b/docs/md/Comfy/Nodes/LatentRotate.md new file mode 100644 index 0000000000..447c52f54d --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentRotate.md @@ -0,0 +1,54 @@ +# Rotate Latent +## Documentation +- Class name: `LatentRotate` +- Category: `latent/transform` +- Output node: `False` + +The `LatentRotate` node rotates latent representations of images by specified angles. It supports rotation by 0, 90, 180, and 270 degrees, allowing for flexible manipulation of the latent space. +## Input types +### Required +- **`samples`** + - The latent representation of an image to be rotated. This input is crucial for defining the starting point of the rotation operation. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`rotation`** + - Specifies the angle by which the latent image should be rotated. This parameter directly influences the orientation of the resulting latent image. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`latent`** + - The rotated latent representation of the image. This output reflects the transformation applied by the specified rotation angle. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentRotate: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "rotation": (["none", "90 degrees", "180 degrees", "270 degrees"],), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "rotate" + + CATEGORY = "latent/transform" + + def rotate(self, samples, rotation): + s = samples.copy() + rotate_by = 0 + if rotation.startswith("90"): + rotate_by = 1 + elif rotation.startswith("180"): + rotate_by = 2 + elif rotation.startswith("270"): + rotate_by = 3 + + s["samples"] = torch.rot90(samples["samples"], k=rotate_by, dims=[3, 2]) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentSubtract.md b/docs/md/Comfy/Nodes/LatentSubtract.md new file mode 100644 index 0000000000..7a8b0d777d --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentSubtract.md @@ -0,0 +1,50 @@ +# LatentSubtract +## Documentation +- Class name: `LatentSubtract` +- Category: `latent/advanced` +- Output node: `False` + +The LatentSubtract node performs element-wise subtraction between two latent representations. It ensures that the shapes of the latents are compatible before performing the subtraction, adjusting the second latent's shape if necessary. +## Input types +### Required +- **`samples1`** + - The first latent representation to be subtracted from. It serves as the base for the subtraction operation. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`samples2`** + - The second latent representation to be subtracted. This latent is adjusted to match the shape of the first latent before the subtraction. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Output types +- **`latent`** + - The result of the element-wise subtraction between the two latent representations. + - Python dtype: `Tuple[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class LatentSubtract: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples1": ("LATENT",), "samples2": ("LATENT",)}} + + RETURN_TYPES = ("LATENT",) + FUNCTION = "op" + + CATEGORY = "latent/advanced" + + def op(self, samples1, samples2): + samples_out = samples1.copy() + + s1 = samples1["samples"] + s2 = samples2["samples"] + + s2 = reshape_latent_to(s1.shape, s2) + samples_out["samples"] = s1 - s2 + return (samples_out,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentUpscale.md b/docs/md/Comfy/Nodes/LatentUpscale.md new file mode 100644 index 0000000000..05ecc23ec6 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentUpscale.md @@ -0,0 +1,76 @@ +# Upscale Latent +## Documentation +- Class name: `LatentUpscale` +- Category: `latent` +- Output node: `False` + +The LatentUpscale node is designed to upscale latent representations of images. It allows for the specification of the desired output dimensions (width and height) and the method of upscaling. Additionally, it supports cropping of the upscaled image to a specified method. This node is particularly useful for adjusting the resolution of latent images in preparation for further processing or visualization. +## Input types +### Required +- **`samples`** + - The latent representation of an image to be upscaled. This input is crucial as it provides the base data that the upscaling process will modify. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`upscale_method`** + - Specifies the method used for upscaling the latent image. Different methods can affect the quality and characteristics of the upscaled image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`width`** + - The desired width of the upscaled image. This parameter allows for precise control over the dimensions of the output image. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - The desired height of the upscaled image. Similar to the width parameter, this allows for setting the specific height of the output. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`crop`** + - Determines how the upscaled image should be cropped. This can be used to focus on or exclude certain parts of the image after upscaling. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`latent`** + - The upscaled latent representation of the image. This output is the result of applying the specified upscaling method and dimensions to the input latent image. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler` + + +## Source code +```python +class LatentUpscale: + upscale_methods = ["nearest-exact", "bilinear", "area", "bicubic", "bislerp"] + crop_methods = ["disabled", "center"] + + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), "upscale_method": (s.upscale_methods,), + "width": ("INT", {"default": 512, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 512, "min": 0, "max": MAX_RESOLUTION, "step": 8}), + "crop": (s.crop_methods,)}} + RETURN_TYPES = ("LATENT",) + FUNCTION = "upscale" + + CATEGORY = "latent" + + def upscale(self, samples, upscale_method, width, height, crop): + if width == 0 and height == 0: + s = samples + else: + s = samples.copy() + + if width == 0: + height = max(64, height) + width = max(64, round(samples["samples"].shape[3] * height / samples["samples"].shape[2])) + elif height == 0: + width = max(64, width) + height = max(64, round(samples["samples"].shape[2] * width / samples["samples"].shape[3])) + else: + width = max(64, width) + height = max(64, height) + + s["samples"] = comfy.utils.common_upscale(samples["samples"], width // 8, height // 8, upscale_method, crop) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/LatentUpscaleBy.md b/docs/md/Comfy/Nodes/LatentUpscaleBy.md new file mode 100644 index 0000000000..4cc5481f80 --- /dev/null +++ b/docs/md/Comfy/Nodes/LatentUpscaleBy.md @@ -0,0 +1,53 @@ +# Upscale Latent By +## Documentation +- Class name: `LatentUpscaleBy` +- Category: `latent` +- Output node: `False` + +The `LatentUpscaleBy` node is designed to upscale latent representations of images by a specified scale factor using various interpolation methods. It allows for the adjustment of the size of latent images to enhance detail or match specific dimensions. +## Input types +### Required +- **`samples`** + - The latent representation of images to be upscaled. This input is crucial for the operation as it determines the base data that will be modified through the upscaling process. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`upscale_method`** + - Specifies the interpolation method used for upscaling. Different methods can affect the quality and characteristics of the upscaled image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`scale_by`** + - The factor by which the latent images will be upscaled. This directly influences the output size of the images, allowing for precise control over the upscaling process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`latent`** + - The upscaled latent representation of images. This output provides the enhanced images after applying the specified upscaling method and scale factor. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,Reroute,VAEDecode,KSampler (Efficient),LatentInterpolate` + + +## Source code +```python +class LatentUpscaleBy: + upscale_methods = ["nearest-exact", "bilinear", "area", "bicubic", "bislerp"] + + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), "upscale_method": (s.upscale_methods,), + "scale_by": ("FLOAT", {"default": 1.5, "min": 0.01, "max": 8.0, "step": 0.01}),}} + RETURN_TYPES = ("LATENT",) + FUNCTION = "upscale" + + CATEGORY = "latent" + + def upscale(self, samples, upscale_method, scale_by): + s = samples.copy() + width = round(samples["samples"].shape[3] * scale_by) + height = round(samples["samples"].shape[2] * scale_by) + s["samples"] = comfy.utils.common_upscale(samples["samples"], width, height, upscale_method, "disabled") + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/LoadImage.md b/docs/md/Comfy/Nodes/LoadImage.md new file mode 100644 index 0000000000..693b55dc20 --- /dev/null +++ b/docs/md/Comfy/Nodes/LoadImage.md @@ -0,0 +1,87 @@ +# Load Image +## Documentation +- Class name: `LoadImage` +- Category: `image` +- Output node: `False` + +The `LoadImage` node is designed to load images from a specified path, process them for use in machine learning models, and handle images with transparency by generating masks. It supports loading images in sequence, applying necessary transformations such as EXIF orientation correction, normalization, and converting them to tensors suitable for model input. Additionally, it generates a mask for images with an alpha channel, indicating areas of transparency. +## Input types +### Required +- **`image`** + - The `image` parameter specifies the image or sequence of images to be loaded and processed. It plays a crucial role in determining the source of the image data and influences the output by dictating which image(s) will undergo processing and potential masking. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`image`** + - The processed image or sequence of images, transformed and normalized for model consumption, returned as a tensor. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`mask`** + - A mask indicating areas of transparency in the input image(s), useful for certain image processing or machine learning tasks, returned as a tensor. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `SVD_img2vid_Conditioning,ReActorFaceSwap,IPAdapterApply,Reroute,VAEEncodeForInpaint,PrepImageForClipVision,MiDaS-DepthMapPreprocessor,OpenposePreprocessor,InpaintPreprocessor,VAEEncode` + + +## Source code +```python +class LoadImage: + @classmethod + def INPUT_TYPES(s): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] + return {"required": + {"image": (sorted(files), {"image_upload": True})}, + } + + CATEGORY = "image" + + RETURN_TYPES = ("IMAGE", "MASK") + FUNCTION = "load_image" + def load_image(self, image): + image_path = folder_paths.get_annotated_filepath(image) + img = Image.open(image_path) + output_images = [] + output_masks = [] + for i in ImageSequence.Iterator(img): + i = ImageOps.exif_transpose(i) + if i.mode == 'I': + i = i.point(lambda i: i * (1 / 255)) + image = i.convert("RGB") + image = np.array(image).astype(np.float32) / 255.0 + image = torch.from_numpy(image)[None,] + if 'A' in i.getbands(): + mask = np.array(i.getchannel('A')).astype(np.float32) / 255.0 + mask = 1. - torch.from_numpy(mask) + else: + mask = torch.zeros((64,64), dtype=torch.float32, device="cpu") + output_images.append(image) + output_masks.append(mask.unsqueeze(0)) + + if len(output_images) > 1: + output_image = torch.cat(output_images, dim=0) + output_mask = torch.cat(output_masks, dim=0) + else: + output_image = output_images[0] + output_mask = output_masks[0] + + return (output_image, output_mask) + + @classmethod + def IS_CHANGED(s, image): + image_path = folder_paths.get_annotated_filepath(image) + m = hashlib.sha256() + with open(image_path, 'rb') as f: + m.update(f.read()) + return m.digest().hex() + + @classmethod + def VALIDATE_INPUTS(s, image): + if not folder_paths.exists_annotated_filepath(image): + return "Invalid image file: {}".format(image) + + return True + +``` diff --git a/docs/md/Comfy/Nodes/LoadImageMask.md b/docs/md/Comfy/Nodes/LoadImageMask.md new file mode 100644 index 0000000000..e870de54c4 --- /dev/null +++ b/docs/md/Comfy/Nodes/LoadImageMask.md @@ -0,0 +1,79 @@ +# Load Image (as Mask) +## Documentation +- Class name: `LoadImageMask` +- Category: `mask` +- Output node: `False` + +The `LoadImageMask` node is designed to load an image from a specified path, process it to handle different image modes and orientations, and extract a mask based on the specified color channel. It supports handling images with alpha channels for mask extraction and ensures images are in the correct orientation and format for further processing. +## Input types +### Required +- **`image`** + - The `image` parameter specifies the file name of the image to be loaded and processed. It is crucial for identifying the correct file within a predefined directory structure and extracting the relevant mask data for further operations. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`channel`** + - The `channel` parameter specifies the color channel ('red', 'green', 'blue', 'alpha') from which the mask will be extracted. This allows for flexible mask creation based on different color channels of the image. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`mask`** + - The extracted mask from the specified color channel of the image. Useful for segmentation tasks and further image processing. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class LoadImageMask: + _color_channels = ["alpha", "red", "green", "blue"] + @classmethod + def INPUT_TYPES(s): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] + return {"required": + {"image": (sorted(files), {"image_upload": True}), + "channel": (s._color_channels, ), } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + FUNCTION = "load_image" + def load_image(self, image, channel): + image_path = folder_paths.get_annotated_filepath(image) + i = Image.open(image_path) + i = ImageOps.exif_transpose(i) + if i.getbands() != ("R", "G", "B", "A"): + if i.mode == 'I': + i = i.point(lambda i: i * (1 / 255)) + i = i.convert("RGBA") + mask = None + c = channel[0].upper() + if c in i.getbands(): + mask = np.array(i.getchannel(c)).astype(np.float32) / 255.0 + mask = torch.from_numpy(mask) + if c == 'A': + mask = 1. - mask + else: + mask = torch.zeros((64,64), dtype=torch.float32, device="cpu") + return (mask.unsqueeze(0),) + + @classmethod + def IS_CHANGED(s, image, channel): + image_path = folder_paths.get_annotated_filepath(image) + m = hashlib.sha256() + with open(image_path, 'rb') as f: + m.update(f.read()) + return m.digest().hex() + + @classmethod + def VALIDATE_INPUTS(s, image): + if not folder_paths.exists_annotated_filepath(image): + return "Invalid image file: {}".format(image) + + return True + +``` diff --git a/docs/md/Comfy/Nodes/LoadLatent.md b/docs/md/Comfy/Nodes/LoadLatent.md new file mode 100644 index 0000000000..46f8251344 --- /dev/null +++ b/docs/md/Comfy/Nodes/LoadLatent.md @@ -0,0 +1,61 @@ +# LoadLatent +## Documentation +- Class name: `LoadLatent` +- Category: `_for_testing` +- Output node: `False` + +This node is responsible for loading latent representations from files with a '.latent' extension located in a specified directory. It adjusts the scale of the latent tensor based on its version. +## Input types +### Required +- **`latent`** + - Specifies the name of the latent file to be loaded. The file must exist in the pre-defined input directory and have a '.latent' extension. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`latent`** + - Returns the loaded and potentially rescaled latent tensor as a dictionary with the key 'samples'. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class LoadLatent: + @classmethod + def INPUT_TYPES(s): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f)) and f.endswith(".latent")] + return {"required": {"latent": [sorted(files), ]}, } + + CATEGORY = "_for_testing" + + RETURN_TYPES = ("LATENT", ) + FUNCTION = "load" + + def load(self, latent): + latent_path = folder_paths.get_annotated_filepath(latent) + latent = safetensors.torch.load_file(latent_path, device="cpu") + multiplier = 1.0 + if "latent_format_version_0" not in latent: + multiplier = 1.0 / 0.18215 + samples = {"samples": latent["latent_tensor"].float() * multiplier} + return (samples, ) + + @classmethod + def IS_CHANGED(s, latent): + image_path = folder_paths.get_annotated_filepath(latent) + m = hashlib.sha256() + with open(image_path, 'rb') as f: + m.update(f.read()) + return m.digest().hex() + + @classmethod + def VALIDATE_INPUTS(s, latent): + if not folder_paths.exists_annotated_filepath(latent): + return "Invalid latent file: {}".format(latent) + return True + +``` diff --git a/docs/md/Comfy/Nodes/LoraLoader.md b/docs/md/Comfy/Nodes/LoraLoader.md new file mode 100644 index 0000000000..61feafdb23 --- /dev/null +++ b/docs/md/Comfy/Nodes/LoraLoader.md @@ -0,0 +1,84 @@ +# Load LoRA +## Documentation +- Class name: `LoraLoader` +- Category: `loaders` +- Output node: `False` + +The LoraLoader node is designed to load and apply LoRA (Low-Rank Adaptation) adjustments to given models and CLIP instances. It dynamically adjusts the strength of the LoRA modifications based on provided parameters, allowing for fine-tuned control over the adaptation process. This node supports conditional loading to avoid redundant operations if the same LoRA configuration has been loaded previously. +## Input types +### Required +- **`model`** + - The model parameter represents the neural network model to which LoRA adjustments will be applied. It's crucial for adapting the model's behavior without extensive retraining. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip`** + - The clip parameter signifies the CLIP model instance that will undergo LoRA adjustments. This is essential for modifying CLIP's behavior in a controlled manner. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`lora_name`** + - Specifies the name of the LoRA configuration to load. This parameter is key to identifying the specific LoRA adjustments to apply. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`strength_model`** + - Determines the intensity of LoRA adjustments applied to the model. This allows for precise control over the adaptation's impact. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`strength_clip`** + - Sets the intensity of LoRA adjustments for the CLIP model, enabling fine-tuned control over its adaptation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The modified model with LoRA adjustments applied. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip`** + - The CLIP model instance after LoRA adjustments. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +## Usage tips +- Infra type: `GPU` +- Common nodes: `LoraLoader,CLIPTextEncode,Reroute,VideoLinearCFGGuidance,KSampler,FaceDetailer,ModelSamplingDiscrete,ADE_AnimateDiffLoaderWithContext,KSampler //Inspire,ToBasicPipe` + + +## Source code +```python +class LoraLoader: + def __init__(self): + self.loaded_lora = None + + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "clip": ("CLIP", ), + "lora_name": (folder_paths.get_filename_list("loras"), ), + "strength_model": ("FLOAT", {"default": 1.0, "min": -20.0, "max": 20.0, "step": 0.01}), + "strength_clip": ("FLOAT", {"default": 1.0, "min": -20.0, "max": 20.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL", "CLIP") + FUNCTION = "load_lora" + + CATEGORY = "loaders" + + def load_lora(self, model, clip, lora_name, strength_model, strength_clip): + if strength_model == 0 and strength_clip == 0: + return (model, clip) + + lora_path = folder_paths.get_full_path("loras", lora_name) + lora = None + if self.loaded_lora is not None: + if self.loaded_lora[0] == lora_path: + lora = self.loaded_lora[1] + else: + temp = self.loaded_lora + self.loaded_lora = None + del temp + + if lora is None: + lora = comfy.utils.load_torch_file(lora_path, safe_load=True) + self.loaded_lora = (lora_path, lora) + + model_lora, clip_lora = comfy.sd.load_lora_for_models(model, clip, lora, strength_model, strength_clip) + return (model_lora, clip_lora) + +``` diff --git a/docs/md/Comfy/Nodes/LoraLoaderModelOnly.md b/docs/md/Comfy/Nodes/LoraLoaderModelOnly.md new file mode 100644 index 0000000000..29ac8f191f --- /dev/null +++ b/docs/md/Comfy/Nodes/LoraLoaderModelOnly.md @@ -0,0 +1,47 @@ +# LoraLoaderModelOnly +## Documentation +- Class name: `LoraLoaderModelOnly` +- Category: `loaders` +- Output node: `False` + +This node is specialized in loading a LoRA model with a specified strength for the model only, excluding any CLIP modifications. It leverages the underlying LoRA loading mechanism to apply the LoRA adjustments to the provided model based on the specified LoRA name and strength, enhancing or altering the model's behavior without affecting any associated CLIP model. +## Input types +### Required +- **`model`** + - The model to which the LoRA adjustments will be applied. It's crucial for defining the base model that will be enhanced or modified by the LoRA parameters. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`lora_name`** + - The name of the LoRA file to be loaded. This determines which specific LoRA adjustments are applied to the model, impacting the model's behavior and performance. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`strength_model`** + - Defines the intensity of the LoRA adjustments applied to the model. This parameter allows for fine-tuning the effect of LoRA on the model, enabling customization of the model's behavior. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The model with LoRA adjustments applied. This output represents the enhanced or modified version of the input model, reflecting the specified LoRA adjustments. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IPAdapterApplyFaceID,LoraLoaderModelOnly,KSampler,FreeU_V2,UltimateSDUpscale` + + +## Source code +```python +class LoraLoaderModelOnly(LoraLoader): + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "lora_name": (folder_paths.get_filename_list("loras"), ), + "strength_model": ("FLOAT", {"default": 1.0, "min": -20.0, "max": 20.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "load_lora_model_only" + + def load_lora_model_only(self, model, lora_name, strength_model): + return (self.load_lora(model, None, lora_name, strength_model, 0)[0],) + +``` diff --git a/docs/md/Comfy/Nodes/MaskComposite.md b/docs/md/Comfy/Nodes/MaskComposite.md new file mode 100644 index 0000000000..729142c533 --- /dev/null +++ b/docs/md/Comfy/Nodes/MaskComposite.md @@ -0,0 +1,89 @@ +# MaskComposite +## Documentation +- Class name: `MaskComposite` +- Category: `mask` +- Output node: `False` + +This node is designed to perform operations on masks, such as combining them through mathematical or logical operations. It allows for the manipulation of mask data to achieve desired effects or transformations. +## Input types +### Required +- **`destination`** + - The primary mask onto which the operation will be applied. It serves as the base for the composite operation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`source`** + - The secondary mask that will be used in the operation with the destination mask. It can be combined in various ways depending on the specified operation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`x`** + - The horizontal offset at which the source mask will be applied to the destination mask. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`y`** + - The vertical offset at which the source mask will be applied to the destination mask. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`operation`** + - Specifies the type of operation to be performed when combining the destination and source masks. Options include mathematical and logical operations like multiply, add, subtract, and, or, xor. + - Python dtype: `str` + - Comfy dtype: `['multiply', 'add', 'subtract', 'and', 'or', 'xor']` +## Output types +- **`mask`** + - The result of the composite operation applied to the masks, producing a new mask. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `ToBinaryMask,LatentCompositeMasked,MaskToImage` + + +## Source code +```python +class MaskComposite: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "destination": ("MASK",), + "source": ("MASK",), + "x": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "y": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 1}), + "operation": (["multiply", "add", "subtract", "and", "or", "xor"],), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + + FUNCTION = "combine" + + def combine(self, destination, source, x, y, operation): + output = destination.reshape((-1, destination.shape[-2], destination.shape[-1])).clone() + source = source.reshape((-1, source.shape[-2], source.shape[-1])) + + left, top = (x, y,) + right, bottom = (min(left + source.shape[-1], destination.shape[-1]), min(top + source.shape[-2], destination.shape[-2])) + visible_width, visible_height = (right - left, bottom - top,) + + source_portion = source[:, :visible_height, :visible_width] + destination_portion = destination[:, top:bottom, left:right] + + if operation == "multiply": + output[:, top:bottom, left:right] = destination_portion * source_portion + elif operation == "add": + output[:, top:bottom, left:right] = destination_portion + source_portion + elif operation == "subtract": + output[:, top:bottom, left:right] = destination_portion - source_portion + elif operation == "and": + output[:, top:bottom, left:right] = torch.bitwise_and(destination_portion.round().bool(), source_portion.round().bool()).float() + elif operation == "or": + output[:, top:bottom, left:right] = torch.bitwise_or(destination_portion.round().bool(), source_portion.round().bool()).float() + elif operation == "xor": + output[:, top:bottom, left:right] = torch.bitwise_xor(destination_portion.round().bool(), source_portion.round().bool()).float() + + output = torch.clamp(output, 0.0, 1.0) + + return (output,) + +``` diff --git a/docs/md/Comfy/Nodes/MaskToImage.md b/docs/md/Comfy/Nodes/MaskToImage.md new file mode 100644 index 0000000000..e2016c9e28 --- /dev/null +++ b/docs/md/Comfy/Nodes/MaskToImage.md @@ -0,0 +1,44 @@ +# Convert Mask to Image +## Documentation +- Class name: `MaskToImage` +- Category: `mask` +- Output node: `False` + +The `MaskToImage` node converts a mask into an image by reshaping and expanding the mask dimensions to match those of an image. This process involves adjusting the mask to have three color channels, effectively transforming it into a grayscale image where the mask's values are replicated across all three channels. +## Input types +### Required +- **`mask`** + - The mask to be converted into an image. This mask serves as the basis for generating a grayscale image where the original mask values are duplicated across the RGB channels. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`image`** + - The resulting image obtained from the mask. This image is a grayscale representation where the mask's values are replicated across the RGB channels, allowing it to be used in contexts expecting an image format. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PreviewImage,Paste By Mask,RegionalIPAdapterColorMask //Inspire,ImageCompositeMasked,ImageInvert,Image To Mask,Cut By Mask,Blur` + + +## Source code +```python +class MaskToImage: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "mask": ("MASK",), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "mask_to_image" + + def mask_to_image(self, mask): + result = mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])).movedim(1, -1).expand(-1, -1, -1, 3) + return (result,) + +``` diff --git a/docs/md/Comfy/Nodes/ModelMergeAdd.md b/docs/md/Comfy/Nodes/ModelMergeAdd.md new file mode 100644 index 0000000000..6b4228db0d --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelMergeAdd.md @@ -0,0 +1,48 @@ +# ModelMergeAdd +## Documentation +- Class name: `ModelMergeAdd` +- Category: `advanced/model_merging` +- Output node: `False` + +Merges two models by cloning the first model and adding key patches from the second model. This process involves copying the first model and then applying modifications from the second model to the cloned version, effectively merging their characteristics. +## Input types +### Required +- **`model1`** + - The base model to be cloned and merged with the second model. It serves as the starting point for the merging process. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`model2`** + - The model from which key patches are extracted and added to the cloned first model. It contributes its unique characteristics to the merged model. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +## Output types +- **`model`** + - The resulting model after merging the key patches from the second model into the cloned first model. It combines the characteristics of both input models. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ModelAdd: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model1": ("MODEL",), + "model2": ("MODEL",), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "merge" + + CATEGORY = "advanced/model_merging" + + def merge(self, model1, model2): + m = model1.clone() + kp = model2.get_key_patches("diffusion_model.") + for k in kp: + m.add_patches({k: kp[k]}, 1.0, 1.0) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/ModelMergeBlocks.md b/docs/md/Comfy/Nodes/ModelMergeBlocks.md new file mode 100644 index 0000000000..52663cf01a --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelMergeBlocks.md @@ -0,0 +1,74 @@ +# ModelMergeBlocks +## Documentation +- Class name: `ModelMergeBlocks` +- Category: `advanced/model_merging` +- Output node: `False` + +This node merges two models by blending their components based on specified ratios. It allows for fine-tuned control over how different parts of the models are combined, making it useful for creating hybrid models with characteristics of both input models. +## Input types +### Required +- **`model1`** + - The first model to be merged. It serves as the base model which will be cloned and modified by adding patches from the second model. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`model2`** + - The second model from which key patches are extracted and merged into the first model based on specified ratios. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`input`** + - A default ratio for merging model components not explicitly specified by other parameters. It affects the overall blending of the two models. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`middle`** + - Specifies the blending ratio for components in the middle of the model structure. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`out`** + - Determines the blending ratio for the output components of the model. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The resulting merged model, incorporating elements from both input models according to the specified ratios. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `CPU` +- Common nodes: `KSampler,CR Load LoRA` + + +## Source code +```python +class ModelMergeBlocks: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model1": ("MODEL",), + "model2": ("MODEL",), + "input": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "middle": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "out": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}) + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "merge" + + CATEGORY = "advanced/model_merging" + + def merge(self, model1, model2, **kwargs): + m = model1.clone() + kp = model2.get_key_patches("diffusion_model.") + default_ratio = next(iter(kwargs.values())) + + for k in kp: + ratio = default_ratio + k_unet = k[len("diffusion_model."):] + + last_arg_size = 0 + for arg in kwargs: + if k_unet.startswith(arg) and last_arg_size < len(arg): + ratio = kwargs[arg] + last_arg_size = len(arg) + + m.add_patches({k: kp[k]}, 1.0 - ratio, ratio) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/ModelMergeSimple.md b/docs/md/Comfy/Nodes/ModelMergeSimple.md new file mode 100644 index 0000000000..dd78eb5a19 --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelMergeSimple.md @@ -0,0 +1,53 @@ +# ModelMergeSimple +## Documentation +- Class name: `ModelMergeSimple` +- Category: `advanced/model_merging` +- Output node: `False` + +Merges two models by blending their parameters based on a specified ratio. This operation allows for the creation of a new model that inherits characteristics from both input models, with the ratio determining the influence of each model on the resulting merged model. +## Input types +### Required +- **`model1`** + - The first model to merge. It serves as the base model onto which patches from the second model are applied. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`model2`** + - The second model to merge. Its parameters are extracted as patches and applied to the first model based on the specified ratio. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`ratio`** + - Determines the blend ratio between the two models' parameters. A ratio closer to 0 gives more weight to the first model, while a ratio closer to 1 gives more weight to the second model. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The resulting model after merging the parameters of the two input models based on the specified ratio. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,Anything Everywhere,CR Apply LoRA Stack,ModelMergeSimple,Reroute` + + +## Source code +```python +class ModelMergeSimple: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model1": ("MODEL",), + "model2": ("MODEL",), + "ratio": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "merge" + + CATEGORY = "advanced/model_merging" + + def merge(self, model1, model2, ratio): + m = model1.clone() + kp = model2.get_key_patches("diffusion_model.") + for k in kp: + m.add_patches({k: kp[k]}, 1.0 - ratio, ratio) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/ModelMergeSubtract.md b/docs/md/Comfy/Nodes/ModelMergeSubtract.md new file mode 100644 index 0000000000..a7031e2c85 --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelMergeSubtract.md @@ -0,0 +1,53 @@ +# ModelMergeSubtract +## Documentation +- Class name: `ModelMergeSubtract` +- Category: `advanced/model_merging` +- Output node: `False` + +This node performs a subtraction operation between two models by applying a multiplier to the patches extracted from the second model before subtracting them from the first model. This operation allows for the dynamic adjustment of the influence one model has over another, enabling the creation of a new model that incorporates characteristics of both input models with a specified intensity. +## Input types +### Required +- **`model1`** + - The first model to be used in the subtraction operation. It serves as the base model from which patches of the second model are subtracted. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`model2`** + - The second model from which patches are extracted and subtracted from the first model, after being scaled by the multiplier. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +- **`multiplier`** + - A scaling factor applied to the patches of the second model before subtraction. It controls the intensity of the subtraction effect on the resulting model. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The resulting model after subtracting scaled patches of the second model from the first model. + - Python dtype: `comfy.model_base.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class ModelSubtract: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model1": ("MODEL",), + "model2": ("MODEL",), + "multiplier": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "merge" + + CATEGORY = "advanced/model_merging" + + def merge(self, model1, model2, multiplier): + m = model1.clone() + kp = model2.get_key_patches("diffusion_model.") + for k in kp: + m.add_patches({k: kp[k]}, - multiplier, multiplier) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/ModelSamplingContinuousEDM.md b/docs/md/Comfy/Nodes/ModelSamplingContinuousEDM.md new file mode 100644 index 0000000000..de3e61b801 --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelSamplingContinuousEDM.md @@ -0,0 +1,68 @@ +# ModelSamplingContinuousEDM +## Documentation +- Class name: `ModelSamplingContinuousEDM` +- Category: `advanced/model` +- Output node: `False` + +This node patches a given model with an advanced continuous EDM (Energy-based Diffusion Model) sampling strategy, allowing for the specification of a range for the noise levels (sigma) used during the sampling process. It supports different types of sampling methods, such as `eps` and `v_prediction`, to adjust the model's behavior for generating outputs. +## Input types +### Required +- **`model`** + - The model to be patched with the advanced continuous EDM sampling strategy. It is the core component that will be modified to incorporate the new sampling behavior. + - Python dtype: `comfy.model_management.Model` + - Comfy dtype: `MODEL` +- **`sampling`** + - Specifies the type of sampling method to be used, such as `eps` for epsilon sampling or `v_prediction` for velocity prediction. This choice influences how the model generates outputs by altering the noise application strategy. + - Python dtype: `Union[str, List[str]]` + - Comfy dtype: `['STRING']` +- **`sigma_max`** + - The maximum value of noise level (sigma) allowed in the sampling process. It defines the upper bound of the noise range, influencing the variability and quality of the generated outputs. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sigma_min`** + - The minimum value of noise level (sigma) allowed in the sampling process. It sets the lower bound of the noise range, affecting the smoothness and detail of the generated outputs. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The patched model with the advanced continuous EDM sampling strategy applied. This modified model is capable of generating outputs with the specified noise level range and sampling method. + - Python dtype: `comfy.model_management.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `FreeU_V2` + + +## Source code +```python +class ModelSamplingContinuousEDM: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "sampling": (["v_prediction", "eps"],), + "sigma_max": ("FLOAT", {"default": 120.0, "min": 0.0, "max": 1000.0, "step":0.001, "round": False}), + "sigma_min": ("FLOAT", {"default": 0.002, "min": 0.0, "max": 1000.0, "step":0.001, "round": False}), + }} + + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "advanced/model" + + def patch(self, model, sampling, sigma_max, sigma_min): + m = model.clone() + + if sampling == "eps": + sampling_type = comfy.model_sampling.EPS + elif sampling == "v_prediction": + sampling_type = comfy.model_sampling.V_PREDICTION + + class ModelSamplingAdvanced(comfy.model_sampling.ModelSamplingContinuousEDM, sampling_type): + pass + + model_sampling = ModelSamplingAdvanced(model.model.model_config) + model_sampling.set_sigma_range(sigma_min, sigma_max) + m.add_object_patch("model_sampling", model_sampling) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/ModelSamplingDiscrete.md b/docs/md/Comfy/Nodes/ModelSamplingDiscrete.md new file mode 100644 index 0000000000..ce17111cd3 --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelSamplingDiscrete.md @@ -0,0 +1,71 @@ +# ModelSamplingDiscrete +## Documentation +- Class name: `ModelSamplingDiscrete` +- Category: `advanced/model` +- Output node: `False` + +The `ModelSamplingDiscrete` node is designed to modify a given model by applying a discrete sampling strategy. It supports various sampling methods such as `eps`, `v_prediction`, `lcm`, and `x0`, each of which alters the model's behavior in specific ways. This node also allows for the optional use of zero-shot noise rescaling (zsnr) to adjust the model's sigma values for noise control. +## Input types +### Required +- **`model`** + - The `model` parameter represents the model to be modified. It is the core input around which the sampling strategy is applied, determining the base behavior before any modifications. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`sampling`** + - The `sampling` parameter specifies the discrete sampling strategy to be applied to the model. It affects how the model generates outputs by altering its internal sampling mechanism. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`zsnr`** + - The `zsnr` flag, when enabled, applies zero-shot noise rescaling to the model's sigma values. This can help in controlling the noise level in the model's output, potentially improving the quality. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +## Output types +- **`model`** + - The modified model with the applied discrete sampling strategy and optional zero-shot noise rescaling. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `Reroute,KSampler,SamplerCustom,CR Module Pipe Loader,FreeU (Advanced),Anything Everywhere,ReroutePrimitive|pysssss,BasicScheduler,KSampler (Efficient)` + + +## Source code +```python +class ModelSamplingDiscrete: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "sampling": (["eps", "v_prediction", "lcm", "x0"],), + "zsnr": ("BOOLEAN", {"default": False}), + }} + + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "advanced/model" + + def patch(self, model, sampling, zsnr): + m = model.clone() + + sampling_base = comfy.model_sampling.ModelSamplingDiscrete + if sampling == "eps": + sampling_type = comfy.model_sampling.EPS + elif sampling == "v_prediction": + sampling_type = comfy.model_sampling.V_PREDICTION + elif sampling == "lcm": + sampling_type = LCM + sampling_base = ModelSamplingDiscreteDistilled + elif sampling == "x0": + sampling_type = X0 + + class ModelSamplingAdvanced(sampling_base, sampling_type): + pass + + model_sampling = ModelSamplingAdvanced(model.model.model_config) + if zsnr: + model_sampling.set_sigmas(rescale_zero_terminal_snr_sigmas(model_sampling.sigmas)) + + m.add_object_patch("model_sampling", model_sampling) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/ModelSamplingStableCascade.md b/docs/md/Comfy/Nodes/ModelSamplingStableCascade.md new file mode 100644 index 0000000000..4d13b71849 --- /dev/null +++ b/docs/md/Comfy/Nodes/ModelSamplingStableCascade.md @@ -0,0 +1,56 @@ +# ModelSamplingStableCascade +## Documentation +- Class name: `ModelSamplingStableCascade` +- Category: `advanced/model` +- Output node: `False` + +This node applies a patch to a given model to enhance its sampling capabilities by integrating a custom sampling strategy. It clones the input model and adds a new sampling method, which is a combination of StableCascadeSampling and EPS, configured with a specified shift parameter. This allows for advanced control over the sampling process, potentially improving the model's performance or generating results with specific characteristics. +## Input types +### Required +- **`model`** + - The model to which the sampling strategy will be applied. This parameter is crucial as it determines the base model that will be enhanced with the new sampling method. + - Python dtype: `comfy.model_sampling.Model` + - Comfy dtype: `MODEL` +- **`shift`** + - A floating-point value that specifies the shift parameter for the new sampling method. This parameter fine-tunes the sampling process, affecting the characteristics of the generated results. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The enhanced model with the new sampling method applied. This model is a clone of the input model, augmented with the custom sampling strategy. + - Python dtype: `comfy.model_sampling.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ModelSamplingStableCascade: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "shift": ("FLOAT", {"default": 2.0, "min": 0.0, "max": 100.0, "step":0.01}), + }} + + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "advanced/model" + + def patch(self, model, shift): + m = model.clone() + + sampling_base = comfy.model_sampling.StableCascadeSampling + sampling_type = comfy.model_sampling.EPS + + class ModelSamplingAdvanced(sampling_base, sampling_type): + pass + + model_sampling = ModelSamplingAdvanced(model.model.model_config) + model_sampling.set_parameters(shift) + m.add_object_patch("model_sampling", model_sampling) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/PatchModelAddDownscale.md b/docs/md/Comfy/Nodes/PatchModelAddDownscale.md new file mode 100644 index 0000000000..6aa1514bc7 --- /dev/null +++ b/docs/md/Comfy/Nodes/PatchModelAddDownscale.md @@ -0,0 +1,96 @@ +# PatchModelAddDownscale (Kohya Deep Shrink) +## Documentation +- Class name: `PatchModelAddDownscale` +- Category: `_for_testing` +- Output node: `False` + +This node applies a downscaling and optional upscaling transformation to specific blocks within a model's architecture, based on sigma values derived from start and end percentages. It allows for the adjustment of the model's internal representations at different stages of processing, potentially enhancing detail or altering the model's behavior in generating outputs. +## Input types +### Required +- **`model`** + - The model to be patched with downscaling and optional upscaling transformations. It serves as the base for modifications aimed at altering the model's internal processing for potentially enhanced output generation. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`block_number`** + - Specifies the block within the model's architecture where the downscaling transformation is to be applied. This allows for targeted modification of the model's processing at a specific stage. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`downscale_factor`** + - Determines the factor by which the input is downscaled. A higher downscale factor results in a more significant reduction in dimensionality, potentially affecting the model's detail capturing capability. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`start_percent`** + - Defines the starting point of the sigma range for applying the downscaling transformation. This parameter helps in targeting the transformation to specific stages of the model's processing based on noise levels. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`end_percent`** + - Sets the end point of the sigma range for the downscaling transformation, allowing for precise control over the stages of the model's processing that are affected. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`downscale_after_skip`** + - A boolean flag indicating whether the downscaling should be applied after a skip connection within the model. This choice can influence how the model integrates information across different processing stages. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`downscale_method`** + - Specifies the method used for downscaling the input. Different methods can affect the quality and characteristics of the downscaled output. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`upscale_method`** + - Determines the method used for optional upscaling after downscaling. Choosing an appropriate method can influence the restoration of detail in the upscaled output. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - The patched model with downscaling and optional upscaling transformations applied to specified blocks. This modified model can exhibit altered behavior in generating outputs, potentially with enhanced detail or different characteristics. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,KRestartSamplerAdv,ModelSamplingDiscrete,LoraLoader,IPAdapterApply` + + +## Source code +```python +class PatchModelAddDownscale: + upscale_methods = ["bicubic", "nearest-exact", "bilinear", "area", "bislerp"] + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "block_number": ("INT", {"default": 3, "min": 1, "max": 32, "step": 1}), + "downscale_factor": ("FLOAT", {"default": 2.0, "min": 0.1, "max": 9.0, "step": 0.001}), + "start_percent": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}), + "end_percent": ("FLOAT", {"default": 0.35, "min": 0.0, "max": 1.0, "step": 0.001}), + "downscale_after_skip": ("BOOLEAN", {"default": True}), + "downscale_method": (s.upscale_methods,), + "upscale_method": (s.upscale_methods,), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "_for_testing" + + def patch(self, model, block_number, downscale_factor, start_percent, end_percent, downscale_after_skip, downscale_method, upscale_method): + sigma_start = model.model.model_sampling.percent_to_sigma(start_percent) + sigma_end = model.model.model_sampling.percent_to_sigma(end_percent) + + def input_block_patch(h, transformer_options): + if transformer_options["block"][1] == block_number: + sigma = transformer_options["sigmas"][0].item() + if sigma <= sigma_start and sigma >= sigma_end: + h = comfy.utils.common_upscale(h, round(h.shape[-1] * (1.0 / downscale_factor)), round(h.shape[-2] * (1.0 / downscale_factor)), downscale_method, "disabled") + return h + + def output_block_patch(h, hsp, transformer_options): + if h.shape[2] != hsp.shape[2]: + h = comfy.utils.common_upscale(h, hsp.shape[-1], hsp.shape[-2], upscale_method, "disabled") + return h, hsp + + m = model.clone() + if downscale_after_skip: + m.set_model_input_block_patch_after_skip(input_block_patch) + else: + m.set_model_input_block_patch(input_block_patch) + m.set_model_output_block_patch(output_block_patch) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/PerpNeg.md b/docs/md/Comfy/Nodes/PerpNeg.md new file mode 100644 index 0000000000..eaf326eced --- /dev/null +++ b/docs/md/Comfy/Nodes/PerpNeg.md @@ -0,0 +1,74 @@ +# Perp-Neg +## Documentation +- Class name: `PerpNeg` +- Category: `_for_testing` +- Output node: `False` + +The PerpNeg node applies a specialized transformation to a given model, focusing on adjusting the model's behavior based on negative conditioning. It clones the input model and modifies its sampling configuration function to compute a perpendicular negative (Perp-Neg) vector, which is scaled and used to adjust the model's output. This process involves complex tensor operations and is designed to enhance the model's generation capabilities by incorporating negative conditioning in a novel way. +## Input types +### Required +- **`model`** + - The input model to be transformed. This model is cloned and its sampling configuration is modified to incorporate the perpendicular negative vector adjustment. + - Python dtype: `comfy.model_management.Model` + - Comfy dtype: `MODEL` +- **`empty_conditioning`** + - An empty conditioning input that serves as a baseline for calculating the perpendicular negative vector. It's used to encode and process model conditions without any positive conditioning. + - Python dtype: `torch.Tensor` + - Comfy dtype: `CONDITIONING` +- **`neg_scale`** + - A scaling factor for the perpendicular negative vector. This scale adjusts the influence of the negative conditioning on the model's output, allowing for fine-tuning of the generation process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The transformed model with a modified sampling configuration to incorporate the perpendicular negative vector adjustment. This model is expected to have enhanced generation capabilities by effectively utilizing negative conditioning. + - Python dtype: `comfy.model_management.Model` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class PerpNeg: + @classmethod + def INPUT_TYPES(s): + return {"required": {"model": ("MODEL", ), + "empty_conditioning": ("CONDITIONING", ), + "neg_scale": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "_for_testing" + + def patch(self, model, empty_conditioning, neg_scale): + m = model.clone() + nocond = comfy.sample.convert_cond(empty_conditioning) + + def cfg_function(args): + model = args["model"] + noise_pred_pos = args["cond_denoised"] + noise_pred_neg = args["uncond_denoised"] + cond_scale = args["cond_scale"] + x = args["input"] + sigma = args["sigma"] + model_options = args["model_options"] + nocond_processed = comfy.samplers.encode_model_conds(model.extra_conds, nocond, x, x.device, "negative") + + (noise_pred_nocond, _) = comfy.samplers.calc_cond_uncond_batch(model, nocond_processed, None, x, sigma, model_options) + + pos = noise_pred_pos - noise_pred_nocond + neg = noise_pred_neg - noise_pred_nocond + perp = neg - ((torch.mul(neg, pos).sum())/(torch.norm(pos)**2)) * pos + perp_neg = perp * neg_scale + cfg_result = noise_pred_nocond + cond_scale*(pos - perp_neg) + cfg_result = x - cfg_result + return cfg_result + + m.set_model_sampler_cfg_function(cfg_function) + + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/PhotoMakerEncode.md b/docs/md/Comfy/Nodes/PhotoMakerEncode.md new file mode 100644 index 0000000000..c5f977431b --- /dev/null +++ b/docs/md/Comfy/Nodes/PhotoMakerEncode.md @@ -0,0 +1,82 @@ +# PhotoMakerEncode +## Documentation +- Class name: `PhotoMakerEncode` +- Category: `_for_testing/photomaker` +- Output node: `False` + +The `PhotoMakerEncode` node is designed to enhance the embedding of text prompts by incorporating visual information from images. It processes image pixel values and text prompt embeddings, fusing them together to produce updated prompt embeddings that are enriched with visual context. This process involves projecting image embeddings into a shared space with the text embeddings and then combining them, aiming to improve the relevance and specificity of the generated content based on the visual input. +## Input types +### Required +- **`photomaker`** + - Represents the model used for incorporating visual information into text prompt embeddings. It is essential for the process of enhancing text prompts with visual context. + - Python dtype: `PhotoMakerIDEncoder` + - Comfy dtype: `PHOTOMAKER` +- **`image`** + - The input image whose visual information is to be fused with the text prompt embeddings. This visual context aims to enrich the text prompts, making them more relevant and specific. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`clip`** + - The CLIP model used for processing the text and possibly the image. It plays a crucial role in encoding the text and generating embeddings that are later fused with visual information. + - Python dtype: `CLIPModel` + - Comfy dtype: `CLIP` +- **`text`** + - The text prompt to be enhanced with visual information. The process aims to make the generated content more relevant and specific by incorporating visual context. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`conditioning`** + - The updated text prompt embeddings, enriched with visual information from the input images. This enhanced representation aims to improve the generated content's relevance and specificity based on the visual context. + - Python dtype: `torch.Tensor` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class PhotoMakerEncode: + @classmethod + def INPUT_TYPES(s): + return {"required": { "photomaker": ("PHOTOMAKER",), + "image": ("IMAGE",), + "clip": ("CLIP", ), + "text": ("STRING", {"multiline": True, "default": "photograph of photomaker"}), + }} + + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "apply_photomaker" + + CATEGORY = "_for_testing/photomaker" + + def apply_photomaker(self, photomaker, image, clip, text): + special_token = "photomaker" + pixel_values = comfy.clip_vision.clip_preprocess(image.to(photomaker.load_device)).float() + try: + index = text.split(" ").index(special_token) + 1 + except ValueError: + index = -1 + tokens = clip.tokenize(text, return_word_ids=True) + out_tokens = {} + for k in tokens: + out_tokens[k] = [] + for t in tokens[k]: + f = list(filter(lambda x: x[2] != index, t)) + while len(f) < len(t): + f.append(t[-1]) + out_tokens[k].append(f) + + cond, pooled = clip.encode_from_tokens(out_tokens, return_pooled=True) + + if index > 0: + token_index = index - 1 + num_id_images = 1 + class_tokens_mask = [True if token_index <= i < token_index+num_id_images else False for i in range(77)] + out = photomaker(id_pixel_values=pixel_values.unsqueeze(0), prompt_embeds=cond.to(photomaker.load_device), + class_tokens_mask=torch.tensor(class_tokens_mask, dtype=torch.bool, device=photomaker.load_device).unsqueeze(0)) + else: + out = cond + + return ([[out, {"pooled_output": pooled}]], ) + +``` diff --git a/docs/md/Comfy/Nodes/PhotoMakerLoader.md b/docs/md/Comfy/Nodes/PhotoMakerLoader.md new file mode 100644 index 0000000000..eb66fd50ff --- /dev/null +++ b/docs/md/Comfy/Nodes/PhotoMakerLoader.md @@ -0,0 +1,45 @@ +# PhotoMakerLoader +## Documentation +- Class name: `PhotoMakerLoader` +- Category: `_for_testing/photomaker` +- Output node: `False` + +This node is responsible for loading a specific PhotoMaker model by its name. It retrieves the model's path, loads the model's state from a file, and optionally extracts a specific component if present, making the model ready for further operations. +## Input types +### Required +- **`photomaker_model_name`** + - The name of the PhotoMaker model to load. This name is used to locate the model file within a predefined directory structure. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`photomaker`** + - The loaded PhotoMaker model, ready for use in image processing tasks. + - Python dtype: `PhotoMakerIDEncoder` + - Comfy dtype: `PHOTOMAKER` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class PhotoMakerLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "photomaker_model_name": (folder_paths.get_filename_list("photomaker"), )}} + + RETURN_TYPES = ("PHOTOMAKER",) + FUNCTION = "load_photomaker_model" + + CATEGORY = "_for_testing/photomaker" + + def load_photomaker_model(self, photomaker_model_name): + photomaker_model_path = folder_paths.get_full_path("photomaker", photomaker_model_name) + photomaker_model = PhotoMakerIDEncoder() + data = comfy.utils.load_torch_file(photomaker_model_path, safe_load=True) + if "id_encoder" in data: + data = data["id_encoder"] + photomaker_model.load_state_dict(data) + return (photomaker_model,) + +``` diff --git a/docs/md/Comfy/Nodes/PolyexponentialScheduler.md b/docs/md/Comfy/Nodes/PolyexponentialScheduler.md new file mode 100644 index 0000000000..d97934cde8 --- /dev/null +++ b/docs/md/Comfy/Nodes/PolyexponentialScheduler.md @@ -0,0 +1,57 @@ +# PolyexponentialScheduler +## Documentation +- Class name: `PolyexponentialScheduler` +- Category: `sampling/custom_sampling/schedulers` +- Output node: `False` + +This node generates a sequence of noise levels (sigmas) based on a polyexponential schedule for diffusion processes. It allows for the customization of the noise schedule by adjusting parameters such as the number of steps and the minimum and maximum sigma values, as well as the rho parameter which controls the shape of the noise schedule. +## Input types +### Required +- **`steps`** + - Specifies the number of steps in the diffusion process. It determines the length of the generated sequence of noise levels. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`sigma_max`** + - The maximum sigma value in the noise schedule. It defines the upper bound of the noise levels. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sigma_min`** + - The minimum sigma value in the noise schedule. It sets the lower limit of the noise levels. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`rho`** + - Controls the shape of the polyexponential noise schedule, affecting how the noise levels progress between the minimum and maximum sigma values. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`sigmas`** + - The generated sequence of noise levels based on the polyexponential schedule. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class PolyexponentialScheduler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "sigma_max": ("FLOAT", {"default": 14.614642, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + "sigma_min": ("FLOAT", {"default": 0.0291675, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + "rho": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/schedulers" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, steps, sigma_max, sigma_min, rho): + sigmas = k_diffusion_sampling.get_sigmas_polyexponential(n=steps, sigma_min=sigma_min, sigma_max=sigma_max, rho=rho) + return (sigmas, ) + +``` diff --git a/docs/md/Comfy/Nodes/PorterDuffImageComposite.md b/docs/md/Comfy/Nodes/PorterDuffImageComposite.md new file mode 100644 index 0000000000..42a0fca94c --- /dev/null +++ b/docs/md/Comfy/Nodes/PorterDuffImageComposite.md @@ -0,0 +1,98 @@ +# Porter-Duff Image Composite +## Documentation +- Class name: `PorterDuffImageComposite` +- Category: `mask/compositing` +- Output node: `False` + +The `PorterDuffImageComposite` node applies various Porter-Duff compositing modes to combine source and destination images and their respective alpha channels. It supports a range of compositing operations such as ADD, CLEAR, DARKEN, etc., each affecting the output image and alpha channel differently based on the specified mode. +## Input types +### Required +- **`source`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `IMAGE` +- **`source_alpha`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `MASK` +- **`destination`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `IMAGE` +- **`destination_alpha`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `MASK` +- **`mode`** + - Specifies the Porter-Duff compositing mode to use for combining the images. + - Python dtype: `PorterDuffMode` + - Comfy dtype: `STRING` +## Output types +- **`image`** + - The composited image resulting from the applied Porter-Duff mode. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`mask`** + - The alpha channel of the composited image, indicating its transparency. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PorterDuffImageComposite` + + +## Source code +```python +class PorterDuffImageComposite: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "source": ("IMAGE",), + "source_alpha": ("MASK",), + "destination": ("IMAGE",), + "destination_alpha": ("MASK",), + "mode": ([mode.name for mode in PorterDuffMode], {"default": PorterDuffMode.DST.name}), + }, + } + + RETURN_TYPES = ("IMAGE", "MASK") + FUNCTION = "composite" + CATEGORY = "mask/compositing" + + def composite(self, source: torch.Tensor, source_alpha: torch.Tensor, destination: torch.Tensor, destination_alpha: torch.Tensor, mode): + batch_size = min(len(source), len(source_alpha), len(destination), len(destination_alpha)) + out_images = [] + out_alphas = [] + + for i in range(batch_size): + src_image = source[i] + dst_image = destination[i] + + assert src_image.shape[2] == dst_image.shape[2] # inputs need to have same number of channels + + src_alpha = source_alpha[i].unsqueeze(2) + dst_alpha = destination_alpha[i].unsqueeze(2) + + if dst_alpha.shape[:2] != dst_image.shape[:2]: + upscale_input = dst_alpha.unsqueeze(0).permute(0, 3, 1, 2) + upscale_output = comfy.utils.common_upscale(upscale_input, dst_image.shape[1], dst_image.shape[0], upscale_method='bicubic', crop='center') + dst_alpha = upscale_output.permute(0, 2, 3, 1).squeeze(0) + if src_image.shape != dst_image.shape: + upscale_input = src_image.unsqueeze(0).permute(0, 3, 1, 2) + upscale_output = comfy.utils.common_upscale(upscale_input, dst_image.shape[1], dst_image.shape[0], upscale_method='bicubic', crop='center') + src_image = upscale_output.permute(0, 2, 3, 1).squeeze(0) + if src_alpha.shape != dst_alpha.shape: + upscale_input = src_alpha.unsqueeze(0).permute(0, 3, 1, 2) + upscale_output = comfy.utils.common_upscale(upscale_input, dst_alpha.shape[1], dst_alpha.shape[0], upscale_method='bicubic', crop='center') + src_alpha = upscale_output.permute(0, 2, 3, 1).squeeze(0) + + out_image, out_alpha = porter_duff_composite(src_image, src_alpha, dst_image, dst_alpha, PorterDuffMode[mode]) + + out_images.append(out_image) + out_alphas.append(out_alpha.squeeze(2)) + + result = (torch.stack(out_images), torch.stack(out_alphas)) + return result + +``` diff --git a/docs/md/Comfy/Nodes/PreviewImage.md b/docs/md/Comfy/Nodes/PreviewImage.md new file mode 100644 index 0000000000..27c4338a14 --- /dev/null +++ b/docs/md/Comfy/Nodes/PreviewImage.md @@ -0,0 +1,37 @@ +# Preview Image +## Documentation +- Class name: `PreviewImage` +- Category: `image` +- Output node: `True` + +The `PreviewImage` node is designed for creating temporary preview images. It generates a unique temporary file name for each image, allows for image compression, and saves the images in a temporary directory. +## Input types +### Required +- **`images`** + - The 'images' parameter is crucial as it directly influences the node's core functionality, which is to process and save temporary preview images. The quality and characteristics of the input images significantly affect the output. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class PreviewImage(SaveImage): + def __init__(self): + self.output_dir = folder_paths.get_temp_directory() + self.type = "temp" + self.prefix_append = "_temp_" + ''.join(random.choice("abcdefghijklmnopqrstupvxyz") for x in range(5)) + self.compress_level = 1 + + @classmethod + def INPUT_TYPES(s): + return {"required": + {"images": ("IMAGE", ), }, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"}, + } + +``` diff --git a/docs/md/Comfy/Nodes/RebatchImages.md b/docs/md/Comfy/Nodes/RebatchImages.md new file mode 100644 index 0000000000..f31ba09889 --- /dev/null +++ b/docs/md/Comfy/Nodes/RebatchImages.md @@ -0,0 +1,58 @@ +# Rebatch Images +## Documentation +- Class name: `RebatchImages` +- Category: `image/batch` +- Output node: `False` + +The `RebatchImages` node is designed to reorganize a list of image batches into a new list of image batches with a specified batch size. It iterates through each image in the input batches, flattens them into a single list of images, and then regroups these images into new batches according to the provided batch size. This process is useful for adjusting the batch size of image data for processing or inference in machine learning models. +## Input types +### Required +- **`images`** + - A list of image batches to be rebatched. This input is crucial for determining the new organization of images across the output batches. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `IMAGE` +- **`batch_size`** + - The desired size for the new image batches. This parameter directly influences the number of images in each output batch, thereby affecting the granularity of processing or inference. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - A list of new image batches, each containing a number of images specified by the `batch_size` parameter. This output facilitates flexible batch processing in subsequent operations. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class ImageRebatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "images": ("IMAGE",), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + }} + RETURN_TYPES = ("IMAGE",) + INPUT_IS_LIST = True + OUTPUT_IS_LIST = (True, ) + + FUNCTION = "rebatch" + + CATEGORY = "image/batch" + + def rebatch(self, images, batch_size): + batch_size = batch_size[0] + + output_list = [] + all_images = [] + for img in images: + for i in range(img.shape[0]): + all_images.append(img[i:i+1]) + + for i in range(0, len(all_images), batch_size): + output_list.append(torch.cat(all_images[i:i+batch_size], dim=0)) + + return (output_list,) + +``` diff --git a/docs/md/Comfy/Nodes/RebatchLatents.md b/docs/md/Comfy/Nodes/RebatchLatents.md new file mode 100644 index 0000000000..2866ca4e8c --- /dev/null +++ b/docs/md/Comfy/Nodes/RebatchLatents.md @@ -0,0 +1,129 @@ +# Rebatch Latents +## Documentation +- Class name: `RebatchLatents` +- Category: `latent/batch` +- Output node: `False` + +The `RebatchLatents` node is designed to reorganize a batch of latent representations into a new batch configuration based on a specified batch size. It processes each latent entry, adjusting the batch size as necessary, and handles different dimensions by slicing or concatenating batches to meet the target batch size. +## Input types +### Required +- **`latents`** + - The input latents to be rebatched. This parameter is crucial for determining how the latent representations are to be reorganized into the new batch configuration. + - Python dtype: `List[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +- **`batch_size`** + - Specifies the target batch size for the rebatching process. It influences how the input latents are divided and combined to form the new batches. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The rebatched latents, organized according to the specified batch size. + - Python dtype: `List[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEDecode` + + +## Source code +```python +class LatentRebatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "latents": ("LATENT",), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + }} + RETURN_TYPES = ("LATENT",) + INPUT_IS_LIST = True + OUTPUT_IS_LIST = (True, ) + + FUNCTION = "rebatch" + + CATEGORY = "latent/batch" + + @staticmethod + def get_batch(latents, list_ind, offset): + '''prepare a batch out of the list of latents''' + samples = latents[list_ind]['samples'] + shape = samples.shape + mask = latents[list_ind]['noise_mask'] if 'noise_mask' in latents[list_ind] else torch.ones((shape[0], 1, shape[2]*8, shape[3]*8), device='cpu') + if mask.shape[-1] != shape[-1] * 8 or mask.shape[-2] != shape[-2]: + torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(shape[-2]*8, shape[-1]*8), mode="bilinear") + if mask.shape[0] < samples.shape[0]: + mask = mask.repeat((shape[0] - 1) // mask.shape[0] + 1, 1, 1, 1)[:shape[0]] + if 'batch_index' in latents[list_ind]: + batch_inds = latents[list_ind]['batch_index'] + else: + batch_inds = [x+offset for x in range(shape[0])] + return samples, mask, batch_inds + + @staticmethod + def get_slices(indexable, num, batch_size): + '''divides an indexable object into num slices of length batch_size, and a remainder''' + slices = [] + for i in range(num): + slices.append(indexable[i*batch_size:(i+1)*batch_size]) + if num * batch_size < len(indexable): + return slices, indexable[num * batch_size:] + else: + return slices, None + + @staticmethod + def slice_batch(batch, num, batch_size): + result = [LatentRebatch.get_slices(x, num, batch_size) for x in batch] + return list(zip(*result)) + + @staticmethod + def cat_batch(batch1, batch2): + if batch1[0] is None: + return batch2 + result = [torch.cat((b1, b2)) if torch.is_tensor(b1) else b1 + b2 for b1, b2 in zip(batch1, batch2)] + return result + + def rebatch(self, latents, batch_size): + batch_size = batch_size[0] + + output_list = [] + current_batch = (None, None, None) + processed = 0 + + for i in range(len(latents)): + # fetch new entry of list + #samples, masks, indices = self.get_batch(latents, i) + next_batch = self.get_batch(latents, i, processed) + processed += len(next_batch[2]) + # set to current if current is None + if current_batch[0] is None: + current_batch = next_batch + # add previous to list if dimensions do not match + elif next_batch[0].shape[-1] != current_batch[0].shape[-1] or next_batch[0].shape[-2] != current_batch[0].shape[-2]: + sliced, _ = self.slice_batch(current_batch, 1, batch_size) + output_list.append({'samples': sliced[0][0], 'noise_mask': sliced[1][0], 'batch_index': sliced[2][0]}) + current_batch = next_batch + # cat if everything checks out + else: + current_batch = self.cat_batch(current_batch, next_batch) + + # add to list if dimensions gone above target batch size + if current_batch[0].shape[0] > batch_size: + num = current_batch[0].shape[0] // batch_size + sliced, remainder = self.slice_batch(current_batch, num, batch_size) + + for i in range(num): + output_list.append({'samples': sliced[0][i], 'noise_mask': sliced[1][i], 'batch_index': sliced[2][i]}) + + current_batch = remainder + + #add remainder + if current_batch[0] is not None: + sliced, _ = self.slice_batch(current_batch, 1, batch_size) + output_list.append({'samples': sliced[0][0], 'noise_mask': sliced[1][0], 'batch_index': sliced[2][0]}) + + #get rid of empty masks + for s in output_list: + if s['noise_mask'].mean() == 1.0: + del s['noise_mask'] + + return (output_list,) + +``` diff --git a/docs/md/Comfy/Nodes/RepeatImageBatch.md b/docs/md/Comfy/Nodes/RepeatImageBatch.md new file mode 100644 index 0000000000..5ca0d5d805 --- /dev/null +++ b/docs/md/Comfy/Nodes/RepeatImageBatch.md @@ -0,0 +1,45 @@ +# RepeatImageBatch +## Documentation +- Class name: `RepeatImageBatch` +- Category: `image/batch` +- Output node: `False` + +This node repeats an input image a specified number of times along the batch dimension, effectively creating a batch of identical images. This can be useful for operations that require batch processing of the same image. +## Input types +### Required +- **`image`** + - The image to be repeated. This parameter is crucial for defining the input that will be processed multiple times in a batch. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`amount`** + - Specifies how many times the input image should be repeated. This affects the size of the output batch, allowing for batch processing of a single image. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - A batch of images, where each image in the batch is an identical copy of the input image. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class RepeatImageBatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "image": ("IMAGE",), + "amount": ("INT", {"default": 1, "min": 1, "max": 64}), + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "repeat" + + CATEGORY = "image/batch" + + def repeat(self, image, amount): + s = image.repeat((amount, 1,1,1)) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/RepeatLatentBatch.md b/docs/md/Comfy/Nodes/RepeatLatentBatch.md new file mode 100644 index 0000000000..45634e48c6 --- /dev/null +++ b/docs/md/Comfy/Nodes/RepeatLatentBatch.md @@ -0,0 +1,56 @@ +# Repeat Latent Batch +## Documentation +- Class name: `RepeatLatentBatch` +- Category: `latent/batch` +- Output node: `False` + +This node is designed to repeat the latent representations of samples a specified number of times. It can also handle repeating associated noise masks and adjusting batch indices accordingly, ensuring the repeated latents maintain their batch structure. +## Input types +### Required +- **`samples`** + - The latent representations to be repeated. This input is crucial for generating multiple copies of the same latent structure, which can be useful in various generative tasks. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`amount`** + - Specifies how many times the input samples should be repeated. This parameter directly influences the size of the output latent batch, allowing for flexible batch size adjustments. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The output is a modified version of the input latent representations, repeated the specified number of times along with any associated noise masks and batch indices. + - Python dtype: `Tuple[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSamplerAdvanced` + + +## Source code +```python +class RepeatLatentBatch: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "amount": ("INT", {"default": 1, "min": 1, "max": 64}), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "repeat" + + CATEGORY = "latent/batch" + + def repeat(self, samples, amount): + s = samples.copy() + s_in = samples["samples"] + + s["samples"] = s_in.repeat((amount, 1,1,1)) + if "noise_mask" in samples and samples["noise_mask"].shape[0] > 1: + masks = samples["noise_mask"] + if masks.shape[0] < s_in.shape[0]: + masks = masks.repeat(math.ceil(s_in.shape[0] / masks.shape[0]), 1, 1, 1)[:s_in.shape[0]] + s["noise_mask"] = samples["noise_mask"].repeat((amount, 1,1,1)) + if "batch_index" in s: + offset = max(s["batch_index"]) - min(s["batch_index"]) + 1 + s["batch_index"] = s["batch_index"] + [x + (i * offset) for i in range(1, amount) for x in s["batch_index"]] + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/RescaleCFG.md b/docs/md/Comfy/Nodes/RescaleCFG.md new file mode 100644 index 0000000000..c0be2d56b4 --- /dev/null +++ b/docs/md/Comfy/Nodes/RescaleCFG.md @@ -0,0 +1,69 @@ +# RescaleCFG +## Documentation +- Class name: `RescaleCFG` +- Category: `advanced/model` +- Output node: `False` + +The `RescaleCFG` node applies a rescaling operation to the conditional guidance scale (CFG) of a given model. This operation adjusts the influence of the conditional guidance on the model's output, aiming to balance between the conditioned and unconditioned generation. The rescaling is controlled by a multiplier, allowing for fine-tuning of the model's behavior. +## Input types +### Required +- **`model`** + - The model to which the CFG rescaling will be applied. This operation clones the model and modifies its sampler configuration to incorporate the rescaling logic. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`multiplier`** + - A scaling factor that determines the degree of rescaling applied to the CFG. It allows for adjusting the balance between conditioned and unconditioned generation, with values closer to 1 increasing the influence of the conditional guidance. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The modified model with the CFG rescaling applied. This model is a clone of the input model, with its sampler configuration adjusted to incorporate the rescaling logic. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class RescaleCFG: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "multiplier": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "advanced/model" + + def patch(self, model, multiplier): + def rescale_cfg(args): + cond = args["cond"] + uncond = args["uncond"] + cond_scale = args["cond_scale"] + sigma = args["sigma"] + sigma = sigma.view(sigma.shape[:1] + (1,) * (cond.ndim - 1)) + x_orig = args["input"] + + #rescale cfg has to be done on v-pred model output + x = x_orig / (sigma * sigma + 1.0) + cond = ((x - (x_orig - cond)) * (sigma ** 2 + 1.0) ** 0.5) / (sigma) + uncond = ((x - (x_orig - uncond)) * (sigma ** 2 + 1.0) ** 0.5) / (sigma) + + #rescalecfg + x_cfg = uncond + cond_scale * (cond - uncond) + ro_pos = torch.std(cond, dim=(1,2,3), keepdim=True) + ro_cfg = torch.std(x_cfg, dim=(1,2,3), keepdim=True) + + x_rescaled = x_cfg * (ro_pos / ro_cfg) + x_final = multiplier * x_rescaled + (1.0 - multiplier) * x_cfg + + return x_orig - (x - x_final * sigma / (sigma * sigma + 1.0) ** 0.5) + + m = model.clone() + m.set_model_sampler_cfg_function(rescale_cfg) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/SDTurboScheduler.md b/docs/md/Comfy/Nodes/SDTurboScheduler.md new file mode 100644 index 0000000000..0a517edff2 --- /dev/null +++ b/docs/md/Comfy/Nodes/SDTurboScheduler.md @@ -0,0 +1,56 @@ +# SDTurboScheduler +## Documentation +- Class name: `SDTurboScheduler` +- Category: `sampling/custom_sampling/schedulers` +- Output node: `False` + +The `SDTurboScheduler` node is designed to generate a sequence of sigma values for diffusion models, specifically tailored to adjust the noise level based on the `denoise` parameter and the number of steps. It calculates a specific range of timesteps and uses the model's sigma function to produce the sigma values, which are crucial for controlling the noise level during the diffusion process. +## Input types +### Required +- **`model`** + - The model parameter is crucial as it specifies the diffusion model for which the sigma values are to be generated. The choice of model affects the characteristics of the generated sigmas, directly influencing the noise control in the diffusion process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`steps`** + - The steps parameter determines the length of the sigma sequence to be generated. It directly influences the granularity of the noise control during the diffusion process. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`denoise`** + - The denoise parameter adjusts the starting point of the sigma sequence, allowing for finer control over the noise level in the generated images. It essentially modifies the range of timesteps used to calculate the sigmas. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`sigmas`** + - The output sigmas are a sequence of values crucial for controlling the noise level during the diffusion process. They are calculated based on the model's sigma function and the specified timesteps. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: `SamplerCustom,SplitSigmas` + + +## Source code +```python +class SDTurboScheduler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "steps": ("INT", {"default": 1, "min": 1, "max": 10}), + "denoise": ("FLOAT", {"default": 1.0, "min": 0, "max": 1.0, "step": 0.01}), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/schedulers" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, model, steps, denoise): + start_step = 10 - int(10 * denoise) + timesteps = torch.flip(torch.arange(1, 11) * 100 - 1, (0,))[start_step:start_step + steps] + comfy.model_management.load_models_gpu([model]) + sigmas = model.model.model_sampling.sigma(timesteps) + sigmas = torch.cat([sigmas, sigmas.new_zeros([1])]) + return (sigmas, ) + +``` diff --git a/docs/md/Comfy/Nodes/SD_4XUpscale_Conditioning.md b/docs/md/Comfy/Nodes/SD_4XUpscale_Conditioning.md new file mode 100644 index 0000000000..cfa72e69ad --- /dev/null +++ b/docs/md/Comfy/Nodes/SD_4XUpscale_Conditioning.md @@ -0,0 +1,86 @@ +# SD_4XUpscale_Conditioning +## Documentation +- Class name: `SD_4XUpscale_Conditioning` +- Category: `conditioning/upscale_diffusion` +- Output node: `False` + +This node is designed for conditioning and upscaling images by a factor of 4x using diffusion models. It takes images along with positive and negative conditioning phrases, applies noise augmentation, and scales the images, returning enhanced conditioning and a latent representation. +## Input types +### Required +- **`images`** + - The input images to be upscaled. These images are the primary subject for the upscaling process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`positive`** + - Positive conditioning phrases that guide the upscaling process towards desired attributes or features in the output images. + - Python dtype: `List[str]` + - Comfy dtype: `CONDITIONING` +- **`negative`** + - Negative conditioning phrases used to steer away the upscaling process from undesired attributes or features. + - Python dtype: `List[str]` + - Comfy dtype: `CONDITIONING` +- **`scale_ratio`** + - Determines the scaling factor for the upscaling process. A higher value results in a larger output image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`noise_augmentation`** + - Controls the level of noise augmentation applied to the images during the upscaling process, affecting the final image quality. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - Enhanced negative conditioning phrases after the upscaling process. + - Python dtype: `List[str]` + - Comfy dtype: `CONDITIONING` +- **`latent`** + - A latent representation of the upscaled images, useful for further processing or analysis. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler` + + +## Source code +```python +class SD_4XUpscale_Conditioning: + @classmethod + def INPUT_TYPES(s): + return {"required": { "images": ("IMAGE",), + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "scale_ratio": ("FLOAT", {"default": 4.0, "min": 0.0, "max": 10.0, "step": 0.01}), + "noise_augmentation": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}), + }} + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + + FUNCTION = "encode" + + CATEGORY = "conditioning/upscale_diffusion" + + def encode(self, images, positive, negative, scale_ratio, noise_augmentation): + width = max(1, round(images.shape[-2] * scale_ratio)) + height = max(1, round(images.shape[-3] * scale_ratio)) + + pixels = comfy.utils.common_upscale((images.movedim(-1,1) * 2.0) - 1.0, width // 4, height // 4, "bilinear", "center") + + out_cp = [] + out_cn = [] + + for t in positive: + n = [t[0], t[1].copy()] + n[1]['concat_image'] = pixels + n[1]['noise_augmentation'] = noise_augmentation + out_cp.append(n) + + for t in negative: + n = [t[0], t[1].copy()] + n[1]['concat_image'] = pixels + n[1]['noise_augmentation'] = noise_augmentation + out_cn.append(n) + + latent = torch.zeros([images.shape[0], 4, height // 4, width // 4]) + return (out_cp, out_cn, {"samples":latent}) + +``` diff --git a/docs/md/Comfy/Nodes/SVD_img2vid_Conditioning.md b/docs/md/Comfy/Nodes/SVD_img2vid_Conditioning.md new file mode 100644 index 0000000000..9eb4a64676 --- /dev/null +++ b/docs/md/Comfy/Nodes/SVD_img2vid_Conditioning.md @@ -0,0 +1,95 @@ +# SVD_img2vid_Conditioning +## Documentation +- Class name: `SVD_img2vid_Conditioning` +- Category: `conditioning/video_models` +- Output node: `False` + +This node is designed for video generation conditioning, utilizing a combination of CLIP vision embeddings, an initial image, and a VAE model to generate conditioning data for video frames. It adjusts the dimensions of the input image and applies video-specific parameters such as frame count, motion bucket ID, and frames per second (fps) to create a tailored conditioning environment for generating videos. +## Input types +### Required +- **`clip_vision`** + - Represents the CLIP vision model used to extract visual features from the initial image, playing a crucial role in understanding the content and context of the image for video generation. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP_VISION` +- **`init_image`** + - The initial image serves as the starting point for video generation, providing the visual context and content that will be expanded upon in the video frames. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - A Variational Autoencoder (VAE) model used to encode the initial image and any subsequent frames into a latent space, facilitating the generation of coherent and continuous video content. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`width`** + - Specifies the width of the video frames to be generated, allowing for customization of the video's resolution. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Specifies the height of the video frames to be generated, complementing the width parameter to define the video's resolution. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`video_frames`** + - Determines the number of frames to be generated for the video, directly influencing the video's length and content. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`motion_bucket_id`** + - A unique identifier for the motion bucket, which categorizes the type of motion to be applied to the video frames, enhancing the dynamism and realism of the generated video. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`fps`** + - Frames per second (fps) setting for the video, dictating the playback speed and smoothness of the generated video content. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`augmentation_level`** + - Controls the level of augmentation applied to the initial image, introducing variability and enhancing the realism of the generated video frames. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The negative conditioning data, serving as a contrast to the positive data, potentially used for refining the generation process. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`latent`** + - The latent representation of the video frames, encoded into a compact form that can be used for further processing or generation steps. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,Prompts Everywhere,KSamplerAdvanced,SeargeSDXLSamplerV3` + + +## Source code +```python +class SVD_img2vid_Conditioning: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_vision": ("CLIP_VISION",), + "init_image": ("IMAGE",), + "vae": ("VAE",), + "width": ("INT", {"default": 1024, "min": 16, "max": nodes.MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 576, "min": 16, "max": nodes.MAX_RESOLUTION, "step": 8}), + "video_frames": ("INT", {"default": 14, "min": 1, "max": 4096}), + "motion_bucket_id": ("INT", {"default": 127, "min": 1, "max": 1023}), + "fps": ("INT", {"default": 6, "min": 1, "max": 1024}), + "augmentation_level": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.01}) + }} + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + + FUNCTION = "encode" + + CATEGORY = "conditioning/video_models" + + def encode(self, clip_vision, init_image, vae, width, height, video_frames, motion_bucket_id, fps, augmentation_level): + output = clip_vision.encode_image(init_image) + pooled = output.image_embeds.unsqueeze(0) + pixels = comfy.utils.common_upscale(init_image.movedim(-1,1), width, height, "bilinear", "center").movedim(1,-1) + encode_pixels = pixels[:,:,:,:3] + if augmentation_level > 0: + encode_pixels += torch.randn_like(pixels) * augmentation_level + t = vae.encode(encode_pixels) + positive = [[pooled, {"motion_bucket_id": motion_bucket_id, "fps": fps, "augmentation_level": augmentation_level, "concat_latent_image": t}]] + negative = [[torch.zeros_like(pooled), {"motion_bucket_id": motion_bucket_id, "fps": fps, "augmentation_level": augmentation_level, "concat_latent_image": torch.zeros_like(t)}]] + latent = torch.zeros([video_frames, 4, height // 8, width // 8]) + return (positive, negative, {"samples":latent}) + +``` diff --git a/docs/md/Comfy/Nodes/SamplerCustom.md b/docs/md/Comfy/Nodes/SamplerCustom.md new file mode 100644 index 0000000000..c38cbe9344 --- /dev/null +++ b/docs/md/Comfy/Nodes/SamplerCustom.md @@ -0,0 +1,109 @@ +# SamplerCustom +## Documentation +- Class name: `SamplerCustom` +- Category: `sampling/custom_sampling` +- Output node: `False` + +This node represents a custom sampling process, which involves generating or modifying latent images based on various parameters such as model, conditioning, and specific sampler configurations. It's designed to work with different types of samplers and configurations to produce customized outputs. +## Input types +### Required +- **`model`** + - Specifies the generative model used for sampling. It's crucial for determining the behavior and capabilities of the sampling process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`add_noise`** + - Determines whether noise should be added to the sampling process, affecting the randomness and variability of the output. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`noise_seed`** + - Sets the seed for noise generation, ensuring reproducibility of the results when adding noise. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`cfg`** + - Controls the conditioning factor, influencing the strength of the conditioning on the sampling process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`positive`** + - Positive conditioning to guide the sampling towards desired attributes or features. + - Python dtype: `dict` + - Comfy dtype: `CONDITIONING` +- **`negative`** + - Negative conditioning to steer the sampling away from certain attributes or features. + - Python dtype: `dict` + - Comfy dtype: `CONDITIONING` +- **`sampler`** + - The specific sampler algorithm used for generating or modifying the latent images. + - Python dtype: `str` + - Comfy dtype: `SAMPLER` +- **`sigmas`** + - Specifies the noise levels for each step of the sampling process, affecting the detail and quality of the output. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +- **`latent_image`** + - The initial latent image to be modified or used as a basis for generation in the sampling process. + - Python dtype: `dict` + - Comfy dtype: `LATENT` +## Output types +- **`latent`** + - The denoised version of the output latent image, providing a cleaner result. + - Python dtype: `dict` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEDecode,Preview Chooser,LatentUpscaleBy` + + +## Source code +```python +class SamplerCustom: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "add_noise": ("BOOLEAN", {"default": True}), + "noise_seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}), + "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}), + "positive": ("CONDITIONING", ), + "negative": ("CONDITIONING", ), + "sampler": ("SAMPLER", ), + "sigmas": ("SIGMAS", ), + "latent_image": ("LATENT", ), + } + } + + RETURN_TYPES = ("LATENT","LATENT") + RETURN_NAMES = ("output", "denoised_output") + + FUNCTION = "sample" + + CATEGORY = "sampling/custom_sampling" + + def sample(self, model, add_noise, noise_seed, cfg, positive, negative, sampler, sigmas, latent_image): + latent = latent_image + latent_image = latent["samples"] + if not add_noise: + noise = torch.zeros(latent_image.size(), dtype=latent_image.dtype, layout=latent_image.layout, device="cpu") + else: + batch_inds = latent["batch_index"] if "batch_index" in latent else None + noise = comfy.sample.prepare_noise(latent_image, noise_seed, batch_inds) + + noise_mask = None + if "noise_mask" in latent: + noise_mask = latent["noise_mask"] + + x0_output = {} + callback = latent_preview.prepare_callback(model, sigmas.shape[-1] - 1, x0_output) + + disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED + samples = comfy.sample.sample_custom(model, noise, cfg, sampler, sigmas, positive, negative, latent_image, noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise_seed) + + out = latent.copy() + out["samples"] = samples + if "x0" in x0_output: + out_denoised = latent.copy() + out_denoised["samples"] = model.model.process_latent_out(x0_output["x0"].cpu()) + else: + out_denoised = out + return (out, out_denoised) + +``` diff --git a/docs/md/Comfy/Nodes/SamplerDPMPP_2M_SDE.md b/docs/md/Comfy/Nodes/SamplerDPMPP_2M_SDE.md new file mode 100644 index 0000000000..4c75c34f42 --- /dev/null +++ b/docs/md/Comfy/Nodes/SamplerDPMPP_2M_SDE.md @@ -0,0 +1,61 @@ +# SamplerDPMPP_2M_SDE +## Documentation +- Class name: `SamplerDPMPP_2M_SDE` +- Category: `sampling/custom_sampling/samplers` +- Output node: `False` + +This node generates a sampler for the DPM-Solver++(2M) SDE model, selecting between CPU and GPU based on the noise device input. It configures the sampler with specific parameters like solver type, eta, and s_noise. +## Input types +### Required +- **`solver_type`** + - Determines the numerical solver used for the SDE. Options are 'midpoint' and 'heun', affecting the numerical stability and accuracy of the sampling process. + - Python dtype: `List[str]` + - Comfy dtype: `['STRING']` +- **`eta`** + - Controls the step size in the SDE solver, influencing the trade-off between sampling quality and computational cost. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`s_noise`** + - Adjusts the scale of the noise used in the sampling process, impacting the diversity of the generated samples. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`noise_device`** + - Specifies the computational device (CPU or GPU) for noise generation, directly affecting performance and compatibility. + - Python dtype: `str` + - Comfy dtype: `['STRING']` +## Output types +- **`sampler`** + - The configured sampler for the DPM-Solver++(2M) SDE model, ready for use in sampling tasks. + - Python dtype: `comfy.samplers.ksampler` + - Comfy dtype: `SAMPLER` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class SamplerDPMPP_2M_SDE: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"solver_type": (['midpoint', 'heun'], ), + "eta": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + "s_noise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + "noise_device": (['gpu', 'cpu'], ), + } + } + RETURN_TYPES = ("SAMPLER",) + CATEGORY = "sampling/custom_sampling/samplers" + + FUNCTION = "get_sampler" + + def get_sampler(self, solver_type, eta, s_noise, noise_device): + if noise_device == 'cpu': + sampler_name = "dpmpp_2m_sde" + else: + sampler_name = "dpmpp_2m_sde_gpu" + sampler = comfy.samplers.ksampler(sampler_name, {"eta": eta, "s_noise": s_noise, "solver_type": solver_type}) + return (sampler, ) + +``` diff --git a/docs/md/Comfy/Nodes/SamplerDPMPP_SDE.md b/docs/md/Comfy/Nodes/SamplerDPMPP_SDE.md new file mode 100644 index 0000000000..5ea4ad7c05 --- /dev/null +++ b/docs/md/Comfy/Nodes/SamplerDPMPP_SDE.md @@ -0,0 +1,61 @@ +# SamplerDPMPP_SDE +## Documentation +- Class name: `SamplerDPMPP_SDE` +- Category: `sampling/custom_sampling/samplers` +- Output node: `False` + +This node defines a custom sampler for DPM-Solver++ SDE (Stochastic Differential Equations) models, specifically tailored for scenarios where different noise levels and reduction factors are considered. It dynamically selects between CPU and GPU implementations based on the specified noise device, enhancing flexibility and performance optimization. +## Input types +### Required +- **`eta`** + - Controls the strength of the noise applied during the sampling process, influencing the model's ability to generate diverse outputs. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`s_noise`** + - Determines the scale of noise used in the sampling process, affecting the detail and quality of generated samples. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`r`** + - A factor that adjusts the rate at which noise is reduced throughout the sampling process, impacting the smoothness and coherence of the output. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`noise_device`** + - Specifies the computational device (CPU or GPU) for noise generation, allowing for performance optimization based on available hardware. + - Python dtype: `str` + - Comfy dtype: `['gpu', 'cpu']` +## Output types +- **`sampler`** + - The configured sampler object ready for use in generating samples from the model. + - Python dtype: `comfy.samplers.ksampler` + - Comfy dtype: `SAMPLER` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class SamplerDPMPP_SDE: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"eta": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + "s_noise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + "r": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 100.0, "step":0.01, "round": False}), + "noise_device": (['gpu', 'cpu'], ), + } + } + RETURN_TYPES = ("SAMPLER",) + CATEGORY = "sampling/custom_sampling/samplers" + + FUNCTION = "get_sampler" + + def get_sampler(self, eta, s_noise, r, noise_device): + if noise_device == 'cpu': + sampler_name = "dpmpp_sde" + else: + sampler_name = "dpmpp_sde_gpu" + sampler = comfy.samplers.ksampler(sampler_name, {"eta": eta, "s_noise": s_noise, "r": r}) + return (sampler, ) + +``` diff --git a/docs/md/Comfy/Nodes/SaveAnimatedPNG.md b/docs/md/Comfy/Nodes/SaveAnimatedPNG.md new file mode 100644 index 0000000000..029a1e1f58 --- /dev/null +++ b/docs/md/Comfy/Nodes/SaveAnimatedPNG.md @@ -0,0 +1,89 @@ +# SaveAnimatedPNG +## Documentation +- Class name: `SaveAnimatedPNG` +- Category: `image/animation` +- Output node: `True` + +This node is responsible for saving a sequence of images as an animated PNG file. It iterates through a collection of PIL images, saving them together in a single file with specified animation settings such as frame duration, quality, and whether the animation should be lossless. Metadata can also be included in the saved file. +## Input types +### Required +- **`images`** + - The sequence of images to be saved as an animated PNG. This parameter is crucial as it directly contains the visual content that will be animated. + - Python dtype: `List[PIL.Image.Image]` + - Comfy dtype: `IMAGE` +- **`filename_prefix`** + - The prefix for the output file names. This parameter helps in organizing and identifying the output files easily. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`fps`** + - Frames per second for the animation, determining the playback speed of the animated PNG. A higher value results in a faster animation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`compress_level`** + - The compression level for the PNG files, affecting the file size and quality. A higher value results in better compression but might affect image quality. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`ui`** + - The UI representation of the saved animated PNG, including a list of generated files and whether the animation is single-frame or multi-frame. +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class SaveAnimatedPNG: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + self.type = "output" + self.prefix_append = "" + + @classmethod + def INPUT_TYPES(s): + return {"required": + {"images": ("IMAGE", ), + "filename_prefix": ("STRING", {"default": "ComfyUI"}), + "fps": ("FLOAT", {"default": 6.0, "min": 0.01, "max": 1000.0, "step": 0.01}), + "compress_level": ("INT", {"default": 4, "min": 0, "max": 9}) + }, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"}, + } + + RETURN_TYPES = () + FUNCTION = "save_images" + + OUTPUT_NODE = True + + CATEGORY = "image/animation" + + def save_images(self, images, fps, compress_level, filename_prefix="ComfyUI", prompt=None, extra_pnginfo=None): + filename_prefix += self.prefix_append + full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0]) + results = list() + pil_images = [] + for image in images: + i = 255. * image.cpu().numpy() + img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8)) + pil_images.append(img) + + metadata = None + if not args.disable_metadata: + metadata = PngInfo() + if prompt is not None: + metadata.add(b"comf", "prompt".encode("latin-1", "strict") + b"\0" + json.dumps(prompt).encode("latin-1", "strict"), after_idat=True) + if extra_pnginfo is not None: + for x in extra_pnginfo: + metadata.add(b"comf", x.encode("latin-1", "strict") + b"\0" + json.dumps(extra_pnginfo[x]).encode("latin-1", "strict"), after_idat=True) + + file = f"{filename}_{counter:05}_.png" + pil_images[0].save(os.path.join(full_output_folder, file), pnginfo=metadata, compress_level=compress_level, save_all=True, duration=int(1000.0/fps), append_images=pil_images[1:]) + results.append({ + "filename": file, + "subfolder": subfolder, + "type": self.type + }) + + return { "ui": { "images": results, "animated": (True,)} } + +``` diff --git a/docs/md/Comfy/Nodes/SaveAnimatedWEBP.md b/docs/md/Comfy/Nodes/SaveAnimatedWEBP.md new file mode 100644 index 0000000000..d264879757 --- /dev/null +++ b/docs/md/Comfy/Nodes/SaveAnimatedWEBP.md @@ -0,0 +1,110 @@ +# SaveAnimatedWEBP +## Documentation +- Class name: `SaveAnimatedWEBP` +- Category: `image/animation` +- Output node: `True` + +This node is responsible for saving a sequence of PIL images as an animated WEBP file. It iterates through given frames, compiles them into a single WEBP file with specified animation settings such as frame duration, quality, and losslessness, and returns metadata about the saved file. +## Input types +### Required +- **`images`** + - A list of PIL Image objects to be saved as an animated WEBP. These images represent the frames of the animation. + - Python dtype: `List[PIL.Image.Image]` + - Comfy dtype: `IMAGE` +- **`filename_prefix`** + - The base name for the output file. The node appends a counter to this base name for each frame sequence saved. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`fps`** + - Frames per second for the animation. This affects the duration each frame is displayed in the animated WEBP. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`lossless`** + - Determines whether the WEBP animation should be saved in a lossless format. Affects the file size and quality of the animation. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`quality`** + - The compression quality for the WEBP file, ranging from 0 (worst) to 100 (best). Higher values result in better image quality but larger file sizes. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`method`** + - The encoding method to be used for generating the WEBP file. Different methods can affect the encoding speed and file size. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`ui`** + - Returns a UI component containing the list of saved files with their metadata, and a flag indicating if the output is animated. +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class SaveAnimatedWEBP: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + self.type = "output" + self.prefix_append = "" + + methods = {"default": 4, "fastest": 0, "slowest": 6} + @classmethod + def INPUT_TYPES(s): + return {"required": + {"images": ("IMAGE", ), + "filename_prefix": ("STRING", {"default": "ComfyUI"}), + "fps": ("FLOAT", {"default": 6.0, "min": 0.01, "max": 1000.0, "step": 0.01}), + "lossless": ("BOOLEAN", {"default": True}), + "quality": ("INT", {"default": 80, "min": 0, "max": 100}), + "method": (list(s.methods.keys()),), + # "num_frames": ("INT", {"default": 0, "min": 0, "max": 8192}), + }, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"}, + } + + RETURN_TYPES = () + FUNCTION = "save_images" + + OUTPUT_NODE = True + + CATEGORY = "image/animation" + + def save_images(self, images, fps, filename_prefix, lossless, quality, method, num_frames=0, prompt=None, extra_pnginfo=None): + method = self.methods.get(method) + filename_prefix += self.prefix_append + full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0]) + results = list() + pil_images = [] + for image in images: + i = 255. * image.cpu().numpy() + img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8)) + pil_images.append(img) + + metadata = pil_images[0].getexif() + if not args.disable_metadata: + if prompt is not None: + metadata[0x0110] = "prompt:{}".format(json.dumps(prompt)) + if extra_pnginfo is not None: + inital_exif = 0x010f + for x in extra_pnginfo: + metadata[inital_exif] = "{}:{}".format(x, json.dumps(extra_pnginfo[x])) + inital_exif -= 1 + + if num_frames == 0: + num_frames = len(pil_images) + + c = len(pil_images) + for i in range(0, c, num_frames): + file = f"{filename}_{counter:05}_.webp" + pil_images[i].save(os.path.join(full_output_folder, file), save_all=True, duration=int(1000.0/fps), append_images=pil_images[i + 1:i + num_frames], exif=metadata, lossless=lossless, quality=quality, method=method) + results.append({ + "filename": file, + "subfolder": subfolder, + "type": self.type + }) + counter += 1 + + animated = num_frames != 1 + return { "ui": { "images": results, "animated": (animated,) } } + +``` diff --git a/docs/md/Comfy/Nodes/SaveImage.md b/docs/md/Comfy/Nodes/SaveImage.md new file mode 100644 index 0000000000..e713e6824c --- /dev/null +++ b/docs/md/Comfy/Nodes/SaveImage.md @@ -0,0 +1,78 @@ +# Save Image +## Documentation +- Class name: `SaveImage` +- Category: `image` +- Output node: `True` + +The SaveImage node is designed to save a batch of images to disk, optionally including metadata such as prompts and additional PNG information. It supports customization of the filename prefix, compression level, and the inclusion of metadata in the saved images. +## Input types +### Required +- **`images`** + - A batch of images to be saved. The images are expected to be in a tensor format, ready for saving. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`filename_prefix`** + - A prefix to be added to the filename of each saved image, allowing for easy identification and organization of saved images. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`ui`** + - A UI component displaying the saved images, including their filenames, subfolders, and types. +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class SaveImage: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + self.type = "output" + self.prefix_append = "" + self.compress_level = 4 + + @classmethod + def INPUT_TYPES(s): + return {"required": + {"images": ("IMAGE", ), + "filename_prefix": ("STRING", {"default": "ComfyUI"})}, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"}, + } + + RETURN_TYPES = () + FUNCTION = "save_images" + + OUTPUT_NODE = True + + CATEGORY = "image" + + def save_images(self, images, filename_prefix="ComfyUI", prompt=None, extra_pnginfo=None): + filename_prefix += self.prefix_append + full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0]) + results = list() + for (batch_number, image) in enumerate(images): + i = 255. * image.cpu().numpy() + img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8)) + metadata = None + if not args.disable_metadata: + metadata = PngInfo() + if prompt is not None: + metadata.add_text("prompt", json.dumps(prompt)) + if extra_pnginfo is not None: + for x in extra_pnginfo: + metadata.add_text(x, json.dumps(extra_pnginfo[x])) + + filename_with_batch_num = filename.replace("%batch_num%", str(batch_number)) + file = f"{filename_with_batch_num}_{counter:05}_.png" + img.save(os.path.join(full_output_folder, file), pnginfo=metadata, compress_level=self.compress_level) + results.append({ + "filename": file, + "subfolder": subfolder, + "type": self.type + }) + counter += 1 + + return { "ui": { "images": results } } + +``` diff --git a/docs/md/Comfy/Nodes/SaveLatent.md b/docs/md/Comfy/Nodes/SaveLatent.md new file mode 100644 index 0000000000..374f97d643 --- /dev/null +++ b/docs/md/Comfy/Nodes/SaveLatent.md @@ -0,0 +1,77 @@ +# SaveLatent +## Documentation +- Class name: `SaveLatent` +- Category: `_for_testing` +- Output node: `True` + +The `SaveLatent` node is not explicitly detailed in the provided context. However, based on the naming convention and typical operations associated with saving latent representations, it can be inferred that this node likely involves storing latent vectors or images to disk. This process would involve specifying a filename or prefix and the latent data to be saved. The operation is crucial for persisting generated latent representations for future use or analysis. +## Input types +### Required +- **`samples`** + - Represents the latent vectors or images to be saved. This is crucial for the operation as it specifies the data that needs to be persisted for future use. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +- **`filename_prefix`** + - Specifies the prefix for the filename under which the latent data will be saved. This allows for organized storage and easy retrieval of saved latents. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class SaveLatent: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT", ), + "filename_prefix": ("STRING", {"default": "latents/ComfyUI"})}, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"}, + } + RETURN_TYPES = () + FUNCTION = "save" + + OUTPUT_NODE = True + + CATEGORY = "_for_testing" + + def save(self, samples, filename_prefix="ComfyUI", prompt=None, extra_pnginfo=None): + full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir) + + # support save metadata for latent sharing + prompt_info = "" + if prompt is not None: + prompt_info = json.dumps(prompt) + + metadata = None + if not args.disable_metadata: + metadata = {"prompt": prompt_info} + if extra_pnginfo is not None: + for x in extra_pnginfo: + metadata[x] = json.dumps(extra_pnginfo[x]) + + file = f"{filename}_{counter:05}_.latent" + + results = list() + results.append({ + "filename": file, + "subfolder": subfolder, + "type": "output" + }) + + file = os.path.join(full_output_folder, file) + + output = {} + output["latent_tensor"] = samples["samples"] + output["latent_format_version_0"] = torch.tensor([]) + + comfy.utils.save_torch_file(output, file, metadata=metadata) + return { "ui": { "latents": results } } + +``` diff --git a/docs/md/Comfy/Nodes/SelfAttentionGuidance.md b/docs/md/Comfy/Nodes/SelfAttentionGuidance.md new file mode 100644 index 0000000000..0cc70a3f33 --- /dev/null +++ b/docs/md/Comfy/Nodes/SelfAttentionGuidance.md @@ -0,0 +1,102 @@ +# Self-Attention Guidance +## Documentation +- Class name: `SelfAttentionGuidance` +- Category: `_for_testing` +- Output node: `False` + +The `SelfAttentionGuidance` node likely encapsulates functionality related to guiding the attention mechanism in neural networks, specifically focusing on self-attention. This could involve modifying or enhancing the attention scores or outputs based on certain criteria or additional inputs to improve the model's performance or interpretability. +## Input types +### Required +- **`model`** + - The model parameter represents the neural network model that will be guided or modified by the SelfAttentionGuidance functionality. It is crucial for defining the structure and behavior of the attention mechanism within the model. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`scale`** + - The scale parameter likely adjusts the magnitude of guidance applied to the attention mechanism, influencing how strongly the guidance affects the model's attention scores or outputs. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`blur_sigma`** + - The blur_sigma parameter probably controls the amount of blurring applied in the guidance process, potentially used to smooth or soften the guidance effects on the attention mechanism. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The modified neural network model with adjusted attention mechanism, reflecting the guidance applied through the SelfAttentionGuidance process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,Reroute,ADE_AnimateDiffLoaderWithContext,Attention couple,UltimateSDUpscale` + + +## Source code +```python +class SelfAttentionGuidance: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "scale": ("FLOAT", {"default": 0.5, "min": -2.0, "max": 5.0, "step": 0.1}), + "blur_sigma": ("FLOAT", {"default": 2.0, "min": 0.0, "max": 10.0, "step": 0.1}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "_for_testing" + + def patch(self, model, scale, blur_sigma): + m = model.clone() + + attn_scores = None + + # TODO: make this work properly with chunked batches + # currently, we can only save the attn from one UNet call + def attn_and_record(q, k, v, extra_options): + nonlocal attn_scores + # if uncond, save the attention scores + heads = extra_options["n_heads"] + cond_or_uncond = extra_options["cond_or_uncond"] + b = q.shape[0] // len(cond_or_uncond) + if 1 in cond_or_uncond: + uncond_index = cond_or_uncond.index(1) + # do the entire attention operation, but save the attention scores to attn_scores + (out, sim) = attention_basic_with_sim(q, k, v, heads=heads) + # when using a higher batch size, I BELIEVE the result batch dimension is [uc1, ... ucn, c1, ... cn] + n_slices = heads * b + attn_scores = sim[n_slices * uncond_index:n_slices * (uncond_index+1)] + return out + else: + return optimized_attention(q, k, v, heads=heads) + + def post_cfg_function(args): + nonlocal attn_scores + uncond_attn = attn_scores + + sag_scale = scale + sag_sigma = blur_sigma + sag_threshold = 1.0 + model = args["model"] + uncond_pred = args["uncond_denoised"] + uncond = args["uncond"] + cfg_result = args["denoised"] + sigma = args["sigma"] + model_options = args["model_options"] + x = args["input"] + if min(cfg_result.shape[2:]) <= 4: #skip when too small to add padding + return cfg_result + + # create the adversarially blurred image + degraded = create_blur_map(uncond_pred, uncond_attn, sag_sigma, sag_threshold) + degraded_noised = degraded + x - uncond_pred + # call into the UNet + (sag, _) = comfy.samplers.calc_cond_uncond_batch(model, uncond, None, degraded_noised, sigma, model_options) + return cfg_result + (degraded - sag) * sag_scale + + m.set_model_sampler_post_cfg_function(post_cfg_function, disable_cfg1_optimization=True) + + # from diffusers: + # unet.mid_block.attentions[0].transformer_blocks[0].attn1.patch + m.set_model_attn1_replace(attn_and_record, "middle", 0, 0) + + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/SetLatentNoiseMask.md b/docs/md/Comfy/Nodes/SetLatentNoiseMask.md new file mode 100644 index 0000000000..81db1625c7 --- /dev/null +++ b/docs/md/Comfy/Nodes/SetLatentNoiseMask.md @@ -0,0 +1,46 @@ +# Set Latent Noise Mask +## Documentation +- Class name: `SetLatentNoiseMask` +- Category: `latent/inpaint` +- Output node: `False` + +This node applies a noise mask to a given set of latent samples. It modifies the input samples by adding a 'noise_mask' field, which is reshaped to match the dimensions of the latent samples. +## Input types +### Required +- **`samples`** + - The latent samples to which the noise mask will be applied. This parameter is crucial for defining the data that will be modified. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +- **`mask`** + - The mask to be applied to the latent samples. It is reshaped to fit the dimensions of the samples, thereby affecting the final output by selectively applying noise. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`latent`** + - The modified latent samples with the applied noise mask. This output retains the original structure of the input samples but includes the added 'noise_mask' field. + - Python dtype: `Tuple[Dict[str, torch.Tensor]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,KSampler (Efficient),Mute / Bypass Repeater (rgthree)` + + +## Source code +```python +class SetLatentNoiseMask: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT",), + "mask": ("MASK",), + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "set_mask" + + CATEGORY = "latent/inpaint" + + def set_mask(self, samples, mask): + s = samples.copy() + s["noise_mask"] = mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])) + return (s,) + +``` diff --git a/docs/md/Comfy/Nodes/SolidMask.md b/docs/md/Comfy/Nodes/SolidMask.md new file mode 100644 index 0000000000..7f46bed395 --- /dev/null +++ b/docs/md/Comfy/Nodes/SolidMask.md @@ -0,0 +1,55 @@ +# SolidMask +## Documentation +- Class name: `SolidMask` +- Category: `mask` +- Output node: `False` + +The SolidMask node generates a solid mask with a uniform value across its dimensions. It creates a mask of specified width and height, filled with a given value. +## Input types +### Required +- **`value`** + - Specifies the uniform value to fill the mask with. It determines the intensity or opacity of the mask. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`width`** + - Determines the width of the generated mask. It specifies how wide the mask will be. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Determines the height of the generated mask. It specifies how tall the mask will be. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`mask`** + - The output is a solid mask with uniform value, specified width, and height. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `MaskComposite,MaskToImage,FeatherMask` + + +## Source code +```python +class SolidMask: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "value": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "width": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + "height": ("INT", {"default": 512, "min": 1, "max": MAX_RESOLUTION, "step": 1}), + } + } + + CATEGORY = "mask" + + RETURN_TYPES = ("MASK",) + + FUNCTION = "solid" + + def solid(self, value, width, height): + out = torch.full((1, height, width), value, dtype=torch.float32, device="cpu") + return (out,) + +``` diff --git a/docs/md/Comfy/Nodes/SplitImageWithAlpha.md b/docs/md/Comfy/Nodes/SplitImageWithAlpha.md new file mode 100644 index 0000000000..743f378162 --- /dev/null +++ b/docs/md/Comfy/Nodes/SplitImageWithAlpha.md @@ -0,0 +1,49 @@ +# Split Image with Alpha +## Documentation +- Class name: `SplitImageWithAlpha` +- Category: `mask/compositing` +- Output node: `False` + +The `SplitImageWithAlpha` node is designed to separate an image into its color components and its alpha (transparency) channel. It processes each image in a batch, extracting the RGB channels as output images and computing the alpha channel, which is then inverted to represent transparency. +## Input types +### Required +- **`image`** + - The input image tensor, expected to contain an alpha channel alongside the RGB channels. This tensor is split into RGB components and the alpha channel, facilitating operations that require separate handling of image content and transparency. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +- **`image`** + - The RGB components of the input images, extracted as a separate tensor. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`mask`** + - The inverted alpha channel of the input images, representing transparency. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PorterDuffImageComposite` + + +## Source code +```python +class SplitImageWithAlpha: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "image": ("IMAGE",), + } + } + + CATEGORY = "mask/compositing" + RETURN_TYPES = ("IMAGE", "MASK") + FUNCTION = "split_image_with_alpha" + + def split_image_with_alpha(self, image: torch.Tensor): + out_images = [i[:,:,:3] for i in image] + out_alphas = [i[:,:,3] if i.shape[2] > 3 else torch.ones_like(i[:,:,0]) for i in image] + result = (torch.stack(out_images), 1.0 - torch.stack(out_alphas)) + return result + +``` diff --git a/docs/md/Comfy/Nodes/SplitSigmas.md b/docs/md/Comfy/Nodes/SplitSigmas.md new file mode 100644 index 0000000000..3895cc77e4 --- /dev/null +++ b/docs/md/Comfy/Nodes/SplitSigmas.md @@ -0,0 +1,48 @@ +# SplitSigmas +## Documentation +- Class name: `SplitSigmas` +- Category: `sampling/custom_sampling/sigmas` +- Output node: `False` + +The `SplitSigmas` node is designed to divide a sequence of sigmas into two parts based on a specified step. This operation is useful in scenarios where different processing or analysis is required for distinct segments of the sigma sequence. +## Input types +### Required +- **`sigmas`** + - The sequence of sigmas to be split. This parameter is crucial for determining how the sequence will be divided into two parts. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +- **`step`** + - The index at which the sigma sequence is split. The sequence is divided such that the first part includes sigmas up to and including this index, and the second part contains the remaining sigmas. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`sigmas`** + - The second part of the split sigma sequence, starting from the specified step. + - Python dtype: `torch.Tensor` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `CPU` +- Common nodes: `SamplerCustom` + + +## Source code +```python +class SplitSigmas: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"sigmas": ("SIGMAS", ), + "step": ("INT", {"default": 0, "min": 0, "max": 10000}), + } + } + RETURN_TYPES = ("SIGMAS","SIGMAS") + CATEGORY = "sampling/custom_sampling/sigmas" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, sigmas, step): + sigmas1 = sigmas[:step + 1] + sigmas2 = sigmas[step:] + return (sigmas1, sigmas2) + +``` diff --git a/docs/md/Comfy/Nodes/StableCascade_EmptyLatentImage.md b/docs/md/Comfy/Nodes/StableCascade_EmptyLatentImage.md new file mode 100644 index 0000000000..36f8226cf6 --- /dev/null +++ b/docs/md/Comfy/Nodes/StableCascade_EmptyLatentImage.md @@ -0,0 +1,65 @@ +# StableCascade_EmptyLatentImage +## Documentation +- Class name: `StableCascade_EmptyLatentImage` +- Category: `_for_testing/stable_cascade` +- Output node: `False` + +The `StableCascade_EmptyLatentImage` node generates two sets of empty latent images (c_latent and b_latent) with specified dimensions and batch size. The dimensions of these latent images are determined by the input width, height, compression factor, and batch size. This node is primarily used for initializing the latent space for further processing in a stable cascade model pipeline. +## Input types +### Required +- **`width`** + - Specifies the width of the generated latent images. It directly influences the dimensions of the output latent tensors. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Determines the height of the generated latent images, affecting the dimensions of the output latent tensors. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`compression`** + - The compression factor used to calculate the dimensions of the c_latent images. A higher compression value results in smaller c_latent dimensions. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`batch_size`** + - Defines the number of latent images to generate in one batch. It directly affects the first dimension of the output latent tensors. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The second set of generated empty latent images, with dimensions reduced by a factor of 4 regardless of the compression input. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class StableCascade_EmptyLatentImage: + def __init__(self, device="cpu"): + self.device = device + + @classmethod + def INPUT_TYPES(s): + return {"required": { + "width": ("INT", {"default": 1024, "min": 256, "max": nodes.MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 1024, "min": 256, "max": nodes.MAX_RESOLUTION, "step": 8}), + "compression": ("INT", {"default": 42, "min": 4, "max": 128, "step": 1}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}) + }} + RETURN_TYPES = ("LATENT", "LATENT") + RETURN_NAMES = ("stage_c", "stage_b") + FUNCTION = "generate" + + CATEGORY = "_for_testing/stable_cascade" + + def generate(self, width, height, compression, batch_size=1): + c_latent = torch.zeros([batch_size, 16, height // compression, width // compression]) + b_latent = torch.zeros([batch_size, 4, height // 4, width // 4]) + return ({ + "samples": c_latent, + }, { + "samples": b_latent, + }) + +``` diff --git a/docs/md/Comfy/Nodes/StableCascade_StageB_Conditioning.md b/docs/md/Comfy/Nodes/StableCascade_StageB_Conditioning.md new file mode 100644 index 0000000000..c20f5140c4 --- /dev/null +++ b/docs/md/Comfy/Nodes/StableCascade_StageB_Conditioning.md @@ -0,0 +1,51 @@ +# StableCascade_StageB_Conditioning +## Documentation +- Class name: `StableCascade_StageB_Conditioning` +- Category: `_for_testing/stable_cascade` +- Output node: `False` + +The `set_prior` method updates the conditioning data by incorporating a 'stable_cascade_prior' derived from the 'stage_c' latent representation into each element of the conditioning data. This process enriches the conditioning context with information from a later stage in the cascade, potentially enhancing the generation quality in subsequent steps. +## Input types +### Required +- **`conditioning`** + - The conditioning data to be updated. It is crucial for guiding the generation process by providing context and constraints. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`stage_c`** + - The latent representation from a later stage in the cascade. It is used to enrich the conditioning data, providing additional context for the generation process. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Output types +- **`conditioning`** + - The updated conditioning data, now enriched with 'stable_cascade_prior' information from the 'stage_c' latent representation. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class StableCascade_StageB_Conditioning: + @classmethod + def INPUT_TYPES(s): + return {"required": { "conditioning": ("CONDITIONING",), + "stage_c": ("LATENT",), + }} + RETURN_TYPES = ("CONDITIONING",) + + FUNCTION = "set_prior" + + CATEGORY = "_for_testing/stable_cascade" + + def set_prior(self, conditioning, stage_c): + c = [] + for t in conditioning: + d = t[1].copy() + d['stable_cascade_prior'] = stage_c['samples'] + n = [t[0], d] + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/StableCascade_StageC_VAEEncode.md b/docs/md/Comfy/Nodes/StableCascade_StageC_VAEEncode.md new file mode 100644 index 0000000000..7b8280cdb3 --- /dev/null +++ b/docs/md/Comfy/Nodes/StableCascade_StageC_VAEEncode.md @@ -0,0 +1,67 @@ +# StableCascade_StageC_VAEEncode +## Documentation +- Class name: `StableCascade_StageC_VAEEncode` +- Category: `_for_testing/stable_cascade` +- Output node: `False` + +This node is responsible for encoding an image into a latent representation using a specified VAE model, with an adjustable compression factor. It performs an initial resizing of the input image based on the compression factor and the VAE's downscale ratio, then encodes the resized image into a latent space representation. Additionally, it generates a zero-initialized latent representation for stage B. +## Input types +### Required +- **`image`** + - The input image to be encoded into a latent representation. It is crucial for the encoding process as it directly influences the generated latent representation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - The VAE model used for encoding the image. It determines the encoding mechanism and the characteristics of the generated latent representation. + - Python dtype: `StageC_coder` + - Comfy dtype: `VAE` +- **`compression`** + - A factor that adjusts the size of the input image before encoding. It affects the resolution of the generated latent representation and allows for control over the level of detail. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - A zero-initialized latent representation for use in stage B of the cascade process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class StableCascade_StageC_VAEEncode: + def __init__(self, device="cpu"): + self.device = device + + @classmethod + def INPUT_TYPES(s): + return {"required": { + "image": ("IMAGE",), + "vae": ("VAE", ), + "compression": ("INT", {"default": 42, "min": 4, "max": 128, "step": 1}), + }} + RETURN_TYPES = ("LATENT", "LATENT") + RETURN_NAMES = ("stage_c", "stage_b") + FUNCTION = "generate" + + CATEGORY = "_for_testing/stable_cascade" + + def generate(self, image, vae, compression): + width = image.shape[-2] + height = image.shape[-3] + out_width = (width // compression) * vae.downscale_ratio + out_height = (height // compression) * vae.downscale_ratio + + s = comfy.utils.common_upscale(image.movedim(-1,1), out_width, out_height, "bicubic", "center").movedim(1,-1) + + c_latent = vae.encode(s[:,:,:,:3]) + b_latent = torch.zeros([c_latent.shape[0], 4, height // 4, width // 4]) + return ({ + "samples": c_latent, + }, { + "samples": b_latent, + }) + +``` diff --git a/docs/md/Comfy/Nodes/StableZero123_Conditioning.md b/docs/md/Comfy/Nodes/StableZero123_Conditioning.md new file mode 100644 index 0000000000..48d998fae7 --- /dev/null +++ b/docs/md/Comfy/Nodes/StableZero123_Conditioning.md @@ -0,0 +1,91 @@ +# StableZero123_Conditioning +## Documentation +- Class name: `StableZero123_Conditioning` +- Category: `conditioning/3d_models` +- Output node: `False` + +This node is designed to process conditioning data for 3D models, specifically focusing on adjusting the conditioning based on the input parameters such as clip vision, initial image, and VAE model. It plays a crucial role in preparing the conditioning for further processing or generation tasks within the context of 3D model conditioning. +## Input types +### Required +- **`clip_vision`** + - Specifies the CLIP model's vision to be used for conditioning, influencing the direction and focus of the generated content. + - Python dtype: `torch.Tensor` + - Comfy dtype: `CLIP_VISION` +- **`init_image`** + - The initial image to start the conditioning process, serving as a base or reference for further modifications. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - The VAE model used for encoding or decoding images, integral to the conditioning process for 3D models. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`width`** + - Defines the width of the output image, affecting the dimensionality of the conditioned output. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Specifies the height of the output image, impacting the dimensionality of the conditioned output. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`batch_size`** + - Determines the number of conditioning operations to be processed in a single batch, influencing the efficiency and speed of the node's execution. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`elevation`** + - Sets the elevation angle for the 3D model conditioning, affecting the perspective and orientation of the generated content. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`azimuth`** + - Defines the azimuth angle for the 3D model conditioning, altering the direction and viewpoint of the generated content. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The negative conditioning output, designed for suppressing certain features or aspects in the generated content. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`latent`** + - The latent representation derived from the conditioning process, essential for further processing or generation tasks. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,SamplerCustom` + + +## Source code +```python +class StableZero123_Conditioning: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_vision": ("CLIP_VISION",), + "init_image": ("IMAGE",), + "vae": ("VAE",), + "width": ("INT", {"default": 256, "min": 16, "max": nodes.MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 256, "min": 16, "max": nodes.MAX_RESOLUTION, "step": 8}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "elevation": ("FLOAT", {"default": 0.0, "min": -180.0, "max": 180.0}), + "azimuth": ("FLOAT", {"default": 0.0, "min": -180.0, "max": 180.0}), + }} + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + + FUNCTION = "encode" + + CATEGORY = "conditioning/3d_models" + + def encode(self, clip_vision, init_image, vae, width, height, batch_size, elevation, azimuth): + output = clip_vision.encode_image(init_image) + pooled = output.image_embeds.unsqueeze(0) + pixels = comfy.utils.common_upscale(init_image.movedim(-1,1), width, height, "bilinear", "center").movedim(1,-1) + encode_pixels = pixels[:,:,:,:3] + t = vae.encode(encode_pixels) + cam_embeds = camera_embeddings(elevation, azimuth) + cond = torch.cat([pooled, cam_embeds.to(pooled.device).repeat((pooled.shape[0], 1, 1))], dim=-1) + + positive = [[cond, {"concat_latent_image": t}]] + negative = [[torch.zeros_like(pooled), {"concat_latent_image": torch.zeros_like(t)}]] + latent = torch.zeros([batch_size, 4, height // 8, width // 8]) + return (positive, negative, {"samples":latent}) + +``` diff --git a/docs/md/Comfy/Nodes/StableZero123_Conditioning_Batched.md b/docs/md/Comfy/Nodes/StableZero123_Conditioning_Batched.md new file mode 100644 index 0000000000..25c9272fd6 --- /dev/null +++ b/docs/md/Comfy/Nodes/StableZero123_Conditioning_Batched.md @@ -0,0 +1,108 @@ +# StableZero123_Conditioning_Batched +## Documentation +- Class name: `StableZero123_Conditioning_Batched` +- Category: `conditioning/3d_models` +- Output node: `False` + +The functionality of the 'StableZero123_Conditioning_Batched' node cannot be directly inferred from the provided context, as specific details about its methods and operations are not given. However, based on the naming convention and the context of other nodes, it might be related to processing or modifying conditioning data in batches for stable generative models. +## Input types +### Required +- **`clip_vision`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `CLIP_VISION` +- **`init_image`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `IMAGE` +- **`vae`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `VAE` +- **`width`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`height`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`batch_size`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`elevation`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`azimuth`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`elevation_batch_increment`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`azimuth_batch_increment`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `CONDITIONING` +- **`latent`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class StableZero123_Conditioning_Batched: + @classmethod + def INPUT_TYPES(s): + return {"required": { "clip_vision": ("CLIP_VISION",), + "init_image": ("IMAGE",), + "vae": ("VAE",), + "width": ("INT", {"default": 256, "min": 16, "max": nodes.MAX_RESOLUTION, "step": 8}), + "height": ("INT", {"default": 256, "min": 16, "max": nodes.MAX_RESOLUTION, "step": 8}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "elevation": ("FLOAT", {"default": 0.0, "min": -180.0, "max": 180.0}), + "azimuth": ("FLOAT", {"default": 0.0, "min": -180.0, "max": 180.0}), + "elevation_batch_increment": ("FLOAT", {"default": 0.0, "min": -180.0, "max": 180.0}), + "azimuth_batch_increment": ("FLOAT", {"default": 0.0, "min": -180.0, "max": 180.0}), + }} + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + + FUNCTION = "encode" + + CATEGORY = "conditioning/3d_models" + + def encode(self, clip_vision, init_image, vae, width, height, batch_size, elevation, azimuth, elevation_batch_increment, azimuth_batch_increment): + output = clip_vision.encode_image(init_image) + pooled = output.image_embeds.unsqueeze(0) + pixels = comfy.utils.common_upscale(init_image.movedim(-1,1), width, height, "bilinear", "center").movedim(1,-1) + encode_pixels = pixels[:,:,:,:3] + t = vae.encode(encode_pixels) + + cam_embeds = [] + for i in range(batch_size): + cam_embeds.append(camera_embeddings(elevation, azimuth)) + elevation += elevation_batch_increment + azimuth += azimuth_batch_increment + + cam_embeds = torch.cat(cam_embeds, dim=0) + cond = torch.cat([comfy.utils.repeat_to_batch_size(pooled, batch_size), cam_embeds], dim=-1) + + positive = [[cond, {"concat_latent_image": t}]] + negative = [[torch.zeros_like(pooled), {"concat_latent_image": torch.zeros_like(t)}]] + latent = torch.zeros([batch_size, 4, height // 8, width // 8]) + return (positive, negative, {"samples":latent, "batch_index": [0] * batch_size}) + +``` diff --git a/docs/md/Comfy/Nodes/StyleModelApply.md b/docs/md/Comfy/Nodes/StyleModelApply.md new file mode 100644 index 0000000000..292858e26d --- /dev/null +++ b/docs/md/Comfy/Nodes/StyleModelApply.md @@ -0,0 +1,54 @@ +# Apply Style Model +## Documentation +- Class name: `StyleModelApply` +- Category: `conditioning/style_model` +- Output node: `False` + +This node applies a style model to a given conditioning, enhancing or altering its style based on the output of a CLIP vision model. It integrates the style model's conditions into the conditioning, preparing it for further processing or generation tasks. +## Input types +### Required +- **`conditioning`** + - The base conditioning data to which the style will be applied. It's crucial for defining the initial context or content that will be styled. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`style_model`** + - The style model used to generate style conditions based on the CLIP vision model's output. It plays a key role in determining the final style applied to the conditioning. + - Python dtype: `StyleModel` + - Comfy dtype: `STYLE_MODEL` +- **`clip_vision_output`** + - The output from a CLIP vision model, which is used by the style model to generate style conditions. It's essential for tailoring the style to the specific content represented in the conditioning. + - Python dtype: `torch.Tensor` + - Comfy dtype: `CLIP_VISION_OUTPUT` +## Output types +- **`conditioning`** + - The enhanced or altered conditioning data, now integrated with style conditions from the style model. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class StyleModelApply: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "style_model": ("STYLE_MODEL", ), + "clip_vision_output": ("CLIP_VISION_OUTPUT", ), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "apply_stylemodel" + + CATEGORY = "conditioning/style_model" + + def apply_stylemodel(self, clip_vision_output, style_model, conditioning): + cond = style_model.get_cond(clip_vision_output).flatten(start_dim=0, end_dim=1).unsqueeze(dim=0) + c = [] + for t in conditioning: + n = [torch.cat((t[0], cond), dim=1), t[1].copy()] + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/Nodes/StyleModelLoader.md b/docs/md/Comfy/Nodes/StyleModelLoader.md new file mode 100644 index 0000000000..7c7c5f6e04 --- /dev/null +++ b/docs/md/Comfy/Nodes/StyleModelLoader.md @@ -0,0 +1,41 @@ +# Load Style Model +## Documentation +- Class name: `StyleModelLoader` +- Category: `loaders` +- Output node: `False` + +This node is responsible for loading a style model from a specified file. It retrieves the style model's path based on the given name, loads the model, and returns it. This process involves checking the model data for a specific key to determine the correct model configuration before initializing and returning the style model instance. +## Input types +### Required +- **`style_model_name`** + - Specifies the name of the style model to be loaded. This name is used to retrieve the full path of the style model file from a predefined list of available style models. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`style_model`** + - The loaded style model instance, ready to be used for applying styles to inputs. + - Python dtype: `StyleModel` + - Comfy dtype: `STYLE_MODEL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class StyleModelLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "style_model_name": (folder_paths.get_filename_list("style_models"), )}} + + RETURN_TYPES = ("STYLE_MODEL",) + FUNCTION = "load_style_model" + + CATEGORY = "loaders" + + def load_style_model(self, style_model_name): + style_model_path = folder_paths.get_full_path("style_models", style_model_name) + style_model = comfy.sd.load_style_model(style_model_path) + return (style_model,) + +``` diff --git a/docs/md/Comfy/Nodes/TomePatchModel.md b/docs/md/Comfy/Nodes/TomePatchModel.md new file mode 100644 index 0000000000..a8927bce2a --- /dev/null +++ b/docs/md/Comfy/Nodes/TomePatchModel.md @@ -0,0 +1,56 @@ +# TomePatchModel +## Documentation +- Class name: `TomePatchModel` +- Category: `_for_testing` +- Output node: `False` + +The `TomePatchModel` node applies a patch to a given model, modifying its attention mechanism with a custom function to potentially enhance its performance. This patching process involves cloning the original model and applying modifications to the attention mechanism based on the provided ratio and the original shape of the input, aiming to improve the model's handling of input data. +## Input types +### Required +- **`model`** + - The model to be patched. This parameter is crucial as it determines the base model that will undergo the patching process to potentially enhance its performance. + - Python dtype: `ModelPatcher` + - Comfy dtype: `MODEL` +- **`ratio`** + - A floating-point value that influences the patching process by adjusting the degree of modification applied to the model's attention mechanism. It plays a significant role in tailoring the patch to achieve desired enhancements. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The patched model, which has undergone modifications to its attention mechanism based on the provided ratio and the original shape of the input. This output is ready for further use or evaluation. + - Python dtype: `ModelPatcher` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class TomePatchModel: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "ratio": ("FLOAT", {"default": 0.3, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "_for_testing" + + def patch(self, model, ratio): + self.u = None + def tomesd_m(q, k, v, extra_options): + #NOTE: In the reference code get_functions takes x (input of the transformer block) as the argument instead of q + #however from my basic testing it seems that using q instead gives better results + m, self.u = get_functions(q, ratio, extra_options["original_shape"]) + return m(q), k, v + def tomesd_u(n, extra_options): + return self.u(n) + + m = model.clone() + m.set_model_attn1_patch(tomesd_m) + m.set_model_attn1_output_patch(tomesd_u) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/UNETLoader.md b/docs/md/Comfy/Nodes/UNETLoader.md new file mode 100644 index 0000000000..e9580c5e78 --- /dev/null +++ b/docs/md/Comfy/Nodes/UNETLoader.md @@ -0,0 +1,41 @@ +# UNETLoader +## Documentation +- Class name: `UNETLoader` +- Category: `advanced/loaders` +- Output node: `False` + +Loads a U-Net model by name from a specified directory. This method is crucial for loading pre-trained U-Net models, which are often used in image segmentation tasks. +## Input types +### Required +- **`unet_name`** + - Specifies the name of the U-Net model to be loaded. This name is used to locate the model file within a predefined directory structure. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - The loaded U-Net model, ready for use in subsequent processing or inference tasks. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PatchModelAddDownscale,Reroute` + + +## Source code +```python +class UNETLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "unet_name": (folder_paths.get_filename_list("unet"), ), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "load_unet" + + CATEGORY = "advanced/loaders" + + def load_unet(self, unet_name): + unet_path = folder_paths.get_full_path("unet", unet_name) + model = comfy.sd.load_unet(unet_path) + return (model,) + +``` diff --git a/docs/md/Comfy/Nodes/UpscaleModelLoader.md b/docs/md/Comfy/Nodes/UpscaleModelLoader.md new file mode 100644 index 0000000000..0c1e00cb67 --- /dev/null +++ b/docs/md/Comfy/Nodes/UpscaleModelLoader.md @@ -0,0 +1,44 @@ +# Load Upscale Model +## Documentation +- Class name: `UpscaleModelLoader` +- Category: `loaders` +- Output node: `False` + +This node is responsible for loading an upscale model by name. It retrieves the model's state dictionary from a specified path, optionally adjusts the state dictionary if necessary, and returns the model in evaluation mode. +## Input types +### Required +- **`model_name`** + - Specifies the name of the upscale model to load. The available model names are retrieved from a predefined list of upscale models, allowing the user to select from a variety of upscale models tailored to different upscaling needs. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`upscale_model`** + - The loaded upscale model, ready for use in evaluation mode. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `UPSCALE_MODEL` +## Usage tips +- Infra type: `CPU` +- Common nodes: `ImageUpscaleWithModel,UltimateSDUpscale,Reroute,LatentPixelScale,Anything Everywhere,PixelKSampleUpscalerProvider` + + +## Source code +```python +class UpscaleModelLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model_name": (folder_paths.get_filename_list("upscale_models"), ), + }} + RETURN_TYPES = ("UPSCALE_MODEL",) + FUNCTION = "load_model" + + CATEGORY = "loaders" + + def load_model(self, model_name): + model_path = folder_paths.get_full_path("upscale_models", model_name) + sd = comfy.utils.load_torch_file(model_path, safe_load=True) + if "module.layers.0.residual_group.blocks.0.norm1.weight" in sd: + sd = comfy.utils.state_dict_prefix_replace(sd, {"module.":""}) + out = model_loading.load_state_dict(sd).eval() + return (out, ) + +``` diff --git a/docs/md/Comfy/Nodes/VAEDecode.md b/docs/md/Comfy/Nodes/VAEDecode.md new file mode 100644 index 0000000000..d690b6c748 --- /dev/null +++ b/docs/md/Comfy/Nodes/VAEDecode.md @@ -0,0 +1,42 @@ +# VAE Decode +## Documentation +- Class name: `VAEDecode` +- Category: `latent` +- Output node: `False` + +This node decodes latent representations into images using a specified VAE model. It takes latent samples as input and utilizes the VAE's decode method to transform these samples back into image space. +## Input types +### Required +- **`samples`** + - Latent representations to be decoded into images. These are essential for generating the final output images from the compressed latent space. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +- **`vae`** + - The VAE model used for decoding the latent samples into images. It defines the specific architecture and parameters for the decoding process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +## Output types +- **`image`** + - The decoded images generated from the latent samples using the VAE model. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PreviewImage,SaveImage,VHS_VideoCombine,Reroute,ImageUpscaleWithModel,UltimateSDUpscale,RIFE VFI,FaceDetailer,FILM VFI,ColorMatch` + + +## Source code +```python +class VAEDecode: + @classmethod + def INPUT_TYPES(s): + return {"required": { "samples": ("LATENT", ), "vae": ("VAE", )}} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "decode" + + CATEGORY = "latent" + + def decode(self, vae, samples): + return (vae.decode(samples["samples"]), ) + +``` diff --git a/docs/md/Comfy/Nodes/VAEDecodeTiled.md b/docs/md/Comfy/Nodes/VAEDecodeTiled.md new file mode 100644 index 0000000000..d6f48f3079 --- /dev/null +++ b/docs/md/Comfy/Nodes/VAEDecodeTiled.md @@ -0,0 +1,48 @@ +# VAE Decode (Tiled) +## Documentation +- Class name: `VAEDecodeTiled` +- Category: `_for_testing` +- Output node: `False` + +The VAEDecodeTiled node is designed for decoding latent representations into images using a tiled approach. This method is particularly useful for handling large images by breaking them down into smaller tiles, decoding each tile separately, and then stitching them back together. This approach can help manage memory usage and computational load more efficiently. +## Input types +### Required +- **`samples`** + - The latent representation of an image or a set of images that the VAE model will decode. This input is crucial for generating the final image output. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +- **`vae`** + - The Variational Autoencoder (VAE) model used for decoding the latent representations into images. The choice of VAE model can significantly affect the quality and characteristics of the generated images. + - Python dtype: `comfy.sd.VAE` + - Comfy dtype: `VAE` +- **`tile_size`** + - Specifies the size of the tiles used in the decoding process. Adjusting the tile size can influence the balance between computational efficiency and the quality of the generated images. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`image`** + - The decoded image or set of images generated from the latent representations. This output is the direct result of the tiled decoding process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `SaveImage,ImageListToImageBatch,ImageUpscaleWithModel,Anything Everywhere,AlphaChanelAdd,GetImageSize,VHS_VideoCombine,ImageSender,ControlNetApplyAdvanced,PreviewImage` + + +## Source code +```python +class VAEDecodeTiled: + @classmethod + def INPUT_TYPES(s): + return {"required": {"samples": ("LATENT", ), "vae": ("VAE", ), + "tile_size": ("INT", {"default": 512, "min": 320, "max": 4096, "step": 64}) + }} + RETURN_TYPES = ("IMAGE",) + FUNCTION = "decode" + + CATEGORY = "_for_testing" + + def decode(self, vae, samples, tile_size): + return (vae.decode_tiled(samples["samples"], tile_x=tile_size // 8, tile_y=tile_size // 8, ), ) + +``` diff --git a/docs/md/Comfy/Nodes/VAEEncode.md b/docs/md/Comfy/Nodes/VAEEncode.md new file mode 100644 index 0000000000..98cdc683ae --- /dev/null +++ b/docs/md/Comfy/Nodes/VAEEncode.md @@ -0,0 +1,43 @@ +# VAE Encode +## Documentation +- Class name: `VAEEncode` +- Category: `latent` +- Output node: `False` + +This node encodes an image into a latent representation using a specified VAE model. The encoding process involves transforming the input image's pixel values to a latent space, which can be used for various generative tasks. +## Input types +### Required +- **`pixels`** + - The input image to be encoded into a latent representation. This image is processed and its pixel values are transformed into a latent space by the VAE model. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - The VAE model used for encoding the input image into a latent representation. This model is responsible for the transformation of pixel values into a latent space. + - Python dtype: `comfy.sd.VAE` + - Comfy dtype: `VAE` +## Output types +- **`latent`** + - The encoded latent representation of the input image. This output can be used for various generative tasks that require a latent space representation. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,KSamplerAdvanced,SetLatentNoiseMask,ImpactKSamplerBasicPipe,KSampler (Efficient),BNK_Unsampler,LatentUpscale,KSampler //Inspire,DZ_Face_Detailer,LatentUpscaleBy` + + +## Source code +```python +class VAEEncode: + @classmethod + def INPUT_TYPES(s): + return {"required": { "pixels": ("IMAGE", ), "vae": ("VAE", )}} + RETURN_TYPES = ("LATENT",) + FUNCTION = "encode" + + CATEGORY = "latent" + + def encode(self, vae, pixels): + t = vae.encode(pixels[:,:,:,:3]) + return ({"samples":t}, ) + +``` diff --git a/docs/md/Comfy/Nodes/VAEEncodeForInpaint.md b/docs/md/Comfy/Nodes/VAEEncodeForInpaint.md new file mode 100644 index 0000000000..38c639f236 --- /dev/null +++ b/docs/md/Comfy/Nodes/VAEEncodeForInpaint.md @@ -0,0 +1,77 @@ +# VAE Encode (for Inpainting) +## Documentation +- Class name: `VAEEncodeForInpaint` +- Category: `latent/inpaint` +- Output node: `False` + +This node is designed for encoding images for inpainting tasks using a Variational Autoencoder (VAE). Inpainting involves filling in missing or corrupted parts of images. The node processes images, applies a mask to identify the areas to inpaint, optionally grows the mask to ensure seamless inpainting, and encodes the modified image into a latent representation using a VAE. +## Input types +### Required +- **`pixels`** + - The input image to be encoded for inpainting. It is the primary data that undergoes the inpainting process. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - The Variational Autoencoder model used for encoding the image into a latent representation. + - Python dtype: `VAE` + - Comfy dtype: `VAE` +- **`mask`** + - A mask indicating the areas of the image to be inpainted. The mask guides the inpainting process by highlighting the regions that need reconstruction. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`grow_mask_by`** + - An optional parameter to expand the mask by a certain number of pixels. Growing the mask can help achieve smoother transitions between inpainted areas and the original image. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The encoded latent representation of the inpainted image. This output is used for further processing or generation tasks. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VHS_DuplicateLatents,KSampler,Reroute,KSampler Adv. (Efficient),Fast Muter (rgthree),KSampler (Efficient),VAEDecode` + + +## Source code +```python +class VAEEncodeForInpaint: + @classmethod + def INPUT_TYPES(s): + return {"required": { "pixels": ("IMAGE", ), "vae": ("VAE", ), "mask": ("MASK", ), "grow_mask_by": ("INT", {"default": 6, "min": 0, "max": 64, "step": 1}),}} + RETURN_TYPES = ("LATENT",) + FUNCTION = "encode" + + CATEGORY = "latent/inpaint" + + def encode(self, vae, pixels, mask, grow_mask_by=6): + x = (pixels.shape[1] // vae.downscale_ratio) * vae.downscale_ratio + y = (pixels.shape[2] // vae.downscale_ratio) * vae.downscale_ratio + mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(pixels.shape[1], pixels.shape[2]), mode="bilinear") + + pixels = pixels.clone() + if pixels.shape[1] != x or pixels.shape[2] != y: + x_offset = (pixels.shape[1] % vae.downscale_ratio) // 2 + y_offset = (pixels.shape[2] % vae.downscale_ratio) // 2 + pixels = pixels[:,x_offset:x + x_offset, y_offset:y + y_offset,:] + mask = mask[:,:,x_offset:x + x_offset, y_offset:y + y_offset] + + #grow mask by a few pixels to keep things seamless in latent space + if grow_mask_by == 0: + mask_erosion = mask + else: + kernel_tensor = torch.ones((1, 1, grow_mask_by, grow_mask_by)) + padding = math.ceil((grow_mask_by - 1) / 2) + + mask_erosion = torch.clamp(torch.nn.functional.conv2d(mask.round(), kernel_tensor, padding=padding), 0, 1) + + m = (1.0 - mask.round()).squeeze(1) + for i in range(3): + pixels[:,:,:,i] -= 0.5 + pixels[:,:,:,i] *= m + pixels[:,:,:,i] += 0.5 + t = vae.encode(pixels) + + return ({"samples":t, "noise_mask": (mask_erosion[:,:,:x,:y].round())}, ) + +``` diff --git a/docs/md/Comfy/Nodes/VAEEncodeTiled.md b/docs/md/Comfy/Nodes/VAEEncodeTiled.md new file mode 100644 index 0000000000..b78663679d --- /dev/null +++ b/docs/md/Comfy/Nodes/VAEEncodeTiled.md @@ -0,0 +1,49 @@ +# VAE Encode (Tiled) +## Documentation +- Class name: `VAEEncodeTiled` +- Category: `_for_testing` +- Output node: `False` + +This node performs tiled encoding of an image using a Variational Autoencoder (VAE). It divides the input image into tiles of a specified size and encodes each tile separately. This approach is useful for handling large images that may not fit into memory if processed as a whole. +## Input types +### Required +- **`pixels`** + - The input image to be encoded. The tiling process allows for efficient handling of large images by breaking them down into smaller, manageable pieces. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - The Variational Autoencoder model used for encoding the image. This model is responsible for transforming the input image into a latent representation. + - Python dtype: `comfy.sd.VAE` + - Comfy dtype: `VAE` +- **`tile_size`** + - The size of the tiles into which the input image is divided. This parameter allows for flexibility in managing the trade-off between memory usage and granularity of encoding. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The encoded representation of the input image in the latent space. This output captures the essential features of the image in a compressed form. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,CR Interpolate Latents,KSampler //Inspire,BNK_TiledKSampler,KSampler (Efficient)` + + +## Source code +```python +class VAEEncodeTiled: + @classmethod + def INPUT_TYPES(s): + return {"required": {"pixels": ("IMAGE", ), "vae": ("VAE", ), + "tile_size": ("INT", {"default": 512, "min": 320, "max": 4096, "step": 64}) + }} + RETURN_TYPES = ("LATENT",) + FUNCTION = "encode" + + CATEGORY = "_for_testing" + + def encode(self, vae, pixels, tile_size): + t = vae.encode_tiled(pixels[:,:,:,:3], tile_x=tile_size, tile_y=tile_size, ) + return ({"samples":t}, ) + +``` diff --git a/docs/md/Comfy/Nodes/VAELoader.md b/docs/md/Comfy/Nodes/VAELoader.md new file mode 100644 index 0000000000..ebd9b354a6 --- /dev/null +++ b/docs/md/Comfy/Nodes/VAELoader.md @@ -0,0 +1,91 @@ +# Load VAE +## Documentation +- Class name: `VAELoader` +- Category: `loaders` +- Output node: `False` + +The VAELoader node is responsible for loading a Variational Autoencoder (VAE) model. It supports loading specific pre-trained VAE models by name, including handling special cases for 'taesd' and 'taesdxl' models by loading their respective encoder and decoder components and setting a scale factor. +## Input types +### Required +- **`vae_name`** + - Specifies the name of the VAE model to load. The node supports a predefined list of VAE models, including 'taesd' and 'taesdxl', and loads the corresponding model components. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`vae`** + - The loaded VAE model, ready for encoding or decoding operations. + - Python dtype: `comfy.sd.VAE` + - Comfy dtype: `VAE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `VAEDecode,VAEEncode,Reroute,CR Module Pipe Loader,VAEDecodeTiled,KSampler Adv. (Efficient),FaceDetailer,Anything Everywhere3,VAEEncodeTiled` + + +## Source code +```python +class VAELoader: + @staticmethod + def vae_list(): + vaes = folder_paths.get_filename_list("vae") + approx_vaes = folder_paths.get_filename_list("vae_approx") + sdxl_taesd_enc = False + sdxl_taesd_dec = False + sd1_taesd_enc = False + sd1_taesd_dec = False + + for v in approx_vaes: + if v.startswith("taesd_decoder."): + sd1_taesd_dec = True + elif v.startswith("taesd_encoder."): + sd1_taesd_enc = True + elif v.startswith("taesdxl_decoder."): + sdxl_taesd_dec = True + elif v.startswith("taesdxl_encoder."): + sdxl_taesd_enc = True + if sd1_taesd_dec and sd1_taesd_enc: + vaes.append("taesd") + if sdxl_taesd_dec and sdxl_taesd_enc: + vaes.append("taesdxl") + return vaes + + @staticmethod + def load_taesd(name): + sd = {} + approx_vaes = folder_paths.get_filename_list("vae_approx") + + encoder = next(filter(lambda a: a.startswith("{}_encoder.".format(name)), approx_vaes)) + decoder = next(filter(lambda a: a.startswith("{}_decoder.".format(name)), approx_vaes)) + + enc = comfy.utils.load_torch_file(folder_paths.get_full_path("vae_approx", encoder)) + for k in enc: + sd["taesd_encoder.{}".format(k)] = enc[k] + + dec = comfy.utils.load_torch_file(folder_paths.get_full_path("vae_approx", decoder)) + for k in dec: + sd["taesd_decoder.{}".format(k)] = dec[k] + + if name == "taesd": + sd["vae_scale"] = torch.tensor(0.18215) + elif name == "taesdxl": + sd["vae_scale"] = torch.tensor(0.13025) + return sd + + @classmethod + def INPUT_TYPES(s): + return {"required": { "vae_name": (s.vae_list(), )}} + RETURN_TYPES = ("VAE",) + FUNCTION = "load_vae" + + CATEGORY = "loaders" + + #TODO: scale factor? + def load_vae(self, vae_name): + if vae_name in ["taesd", "taesdxl"]: + sd = self.load_taesd(vae_name) + else: + vae_path = folder_paths.get_full_path("vae", vae_name) + sd = comfy.utils.load_torch_file(vae_path) + vae = comfy.sd.VAE(sd=sd) + return (vae,) + +``` diff --git a/docs/md/Comfy/Nodes/VAESave.md b/docs/md/Comfy/Nodes/VAESave.md new file mode 100644 index 0000000000..9a45f1539b --- /dev/null +++ b/docs/md/Comfy/Nodes/VAESave.md @@ -0,0 +1,61 @@ +# VAESave +## Documentation +- Class name: `VAESave` +- Category: `advanced/model_merging` +- Output node: `True` + +The VAESave node is designed for saving a VAE model's state dictionary along with optional metadata such as prompts and extra PNG information. It generates a unique filename for the saved model based on a prefix and counter, and stores it in a specified output directory. +## Input types +### Required +- **`vae`** + - The VAE model whose state dictionary is to be saved. This is crucial for preserving the model's learned parameters for future use. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`filename_prefix`** + - A prefix for the filename under which the model and its metadata are saved. This helps in organizing and identifying model files easily. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class VAESave: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + + @classmethod + def INPUT_TYPES(s): + return {"required": { "vae": ("VAE",), + "filename_prefix": ("STRING", {"default": "vae/ComfyUI_vae"}),}, + "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},} + RETURN_TYPES = () + FUNCTION = "save" + OUTPUT_NODE = True + + CATEGORY = "advanced/model_merging" + + def save(self, vae, filename_prefix, prompt=None, extra_pnginfo=None): + full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir) + prompt_info = "" + if prompt is not None: + prompt_info = json.dumps(prompt) + + metadata = {} + if not args.disable_metadata: + metadata["prompt"] = prompt_info + if extra_pnginfo is not None: + for x in extra_pnginfo: + metadata[x] = json.dumps(extra_pnginfo[x]) + + output_checkpoint = f"{filename}_{counter:05}_.safetensors" + output_checkpoint = os.path.join(full_output_folder, output_checkpoint) + + comfy.utils.save_torch_file(vae.get_sd(), output_checkpoint, metadata=metadata) + return {} + +``` diff --git a/docs/md/Comfy/Nodes/VPScheduler.md b/docs/md/Comfy/Nodes/VPScheduler.md new file mode 100644 index 0000000000..4ccf30c1e5 --- /dev/null +++ b/docs/md/Comfy/Nodes/VPScheduler.md @@ -0,0 +1,57 @@ +# VPScheduler +## Documentation +- Class name: `VPScheduler` +- Category: `sampling/custom_sampling/schedulers` +- Output node: `False` + +The VPScheduler node is designed to generate a sequence of noise levels (sigmas) based on the Variance Preserving (VP) SDE approach. It calculates these noise levels using specific parameters that influence the diffusion process, making it a crucial component for custom sampling strategies in generative models. +## Input types +### Required +- **`steps`** + - Defines the number of steps in the diffusion process, directly affecting the granularity of the generated noise levels. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`beta_d`** + - Specifies the beta decay parameter, influencing the rate at which noise levels decrease throughout the diffusion steps. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`beta_min`** + - Sets the minimum beta value, determining the lowest noise level achievable in the diffusion process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`eps_s`** + - Controls the epsilon scale, a parameter that fine-tunes the noise levels for more precise control over the diffusion process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`sigmas`** + - Outputs a sequence of noise levels (sigmas) calculated based on the input parameters, essential for guiding the diffusion process in generative models. + - Python dtype: `Tuple[torch.Tensor]` + - Comfy dtype: `SIGMAS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class VPScheduler: + @classmethod + def INPUT_TYPES(s): + return {"required": + {"steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "beta_d": ("FLOAT", {"default": 19.9, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), #TODO: fix default values + "beta_min": ("FLOAT", {"default": 0.1, "min": 0.0, "max": 1000.0, "step":0.01, "round": False}), + "eps_s": ("FLOAT", {"default": 0.001, "min": 0.0, "max": 1.0, "step":0.0001, "round": False}), + } + } + RETURN_TYPES = ("SIGMAS",) + CATEGORY = "sampling/custom_sampling/schedulers" + + FUNCTION = "get_sigmas" + + def get_sigmas(self, steps, beta_d, beta_min, eps_s): + sigmas = k_diffusion_sampling.get_sigmas_vp(n=steps, beta_d=beta_d, beta_min=beta_min, eps_s=eps_s) + return (sigmas, ) + +``` diff --git a/docs/md/Comfy/Nodes/VideoLinearCFGGuidance.md b/docs/md/Comfy/Nodes/VideoLinearCFGGuidance.md new file mode 100644 index 0000000000..9dd64e8a4f --- /dev/null +++ b/docs/md/Comfy/Nodes/VideoLinearCFGGuidance.md @@ -0,0 +1,54 @@ +# VideoLinearCFGGuidance +## Documentation +- Class name: `VideoLinearCFGGuidance` +- Category: `sampling/video_models` +- Output node: `False` + +This node applies a linear conditioning guidance scale to a video model. It modifies the model's sampler configuration function to adjust the influence of conditioned and unconditioned inputs based on a linear scale between a minimum configuration scale and the provided condition scale. +## Input types +### Required +- **`model`** + - The video model to which the linear CFG guidance will be applied. This model is cloned and modified. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`min_cfg`** + - The minimum conditioning scale used as the starting point of the linear scale applied to the model's conditioning guidance. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`model`** + - The modified video model with the linear CFG guidance applied. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,FreeU_V2,KSamplerAdvanced` + + +## Source code +```python +class VideoLinearCFGGuidance: + @classmethod + def INPUT_TYPES(s): + return {"required": { "model": ("MODEL",), + "min_cfg": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 100.0, "step":0.5, "round": 0.01}), + }} + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + + CATEGORY = "sampling/video_models" + + def patch(self, model, min_cfg): + def linear_cfg(args): + cond = args["cond"] + uncond = args["uncond"] + cond_scale = args["cond_scale"] + + scale = torch.linspace(min_cfg, cond_scale, cond.shape[0], device=cond.device).reshape((cond.shape[0], 1, 1, 1)) + return uncond + scale * (cond - uncond) + + m = model.clone() + m.set_model_sampler_cfg_function(linear_cfg) + return (m, ) + +``` diff --git a/docs/md/Comfy/Nodes/unCLIPCheckpointLoader.md b/docs/md/Comfy/Nodes/unCLIPCheckpointLoader.md new file mode 100644 index 0000000000..431ec1f5ba --- /dev/null +++ b/docs/md/Comfy/Nodes/unCLIPCheckpointLoader.md @@ -0,0 +1,53 @@ +# unCLIPCheckpointLoader +## Documentation +- Class name: `unCLIPCheckpointLoader` +- Category: `loaders` +- Output node: `False` + +This node is responsible for loading checkpoints specifically designed for the unCLIP model. It retrieves the model's checkpoint from a specified path and configures it for use, including loading various components such as VAE, CLIP, and CLIP Vision models. +## Input types +### Required +- **`ckpt_name`** + - Specifies the name of the checkpoint to be loaded. This is crucial for identifying the correct file within the 'checkpoints' directory and loading the associated model. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`model`** + - Returns the main model loaded from the checkpoint. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`clip`** + - Returns the CLIP model component loaded from the checkpoint. + - Python dtype: `comfy.sd.CLIP` + - Comfy dtype: `CLIP` +- **`vae`** + - Returns the VAE model component loaded from the checkpoint. + - Python dtype: `comfy.sd.VAE` + - Comfy dtype: `VAE` +- **`clip_vision`** + - Returns the CLIP Vision model component loaded from the checkpoint. + - Python dtype: `comfy.sd.CLIPVision` + - Comfy dtype: `CLIP_VISION` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class unCLIPCheckpointLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "ckpt_name": (folder_paths.get_filename_list("checkpoints"), ), + }} + RETURN_TYPES = ("MODEL", "CLIP", "VAE", "CLIP_VISION") + FUNCTION = "load_checkpoint" + + CATEGORY = "loaders" + + def load_checkpoint(self, ckpt_name, output_vae=True, output_clip=True): + ckpt_path = folder_paths.get_full_path("checkpoints", ckpt_name) + out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, output_clipvision=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) + return out + +``` diff --git a/docs/md/Comfy/Nodes/unCLIPConditioning.md b/docs/md/Comfy/Nodes/unCLIPConditioning.md new file mode 100644 index 0000000000..9c11f2f38f --- /dev/null +++ b/docs/md/Comfy/Nodes/unCLIPConditioning.md @@ -0,0 +1,67 @@ +# unCLIPConditioning +## Documentation +- Class name: `unCLIPConditioning` +- Category: `conditioning` +- Output node: `False` + +This node applies additional conditioning to the input conditioning data based on CLIP vision output, strength, and noise augmentation parameters. It enhances the conditioning by incorporating visual features and adjustable parameters to influence the generation process. +## Input types +### Required +- **`conditioning`** + - The base conditioning data to which the enhancements are applied. It serves as the foundation for further modifications. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`clip_vision_output`** + - The output from a CLIP vision model, used to enrich the conditioning with visual features. + - Python dtype: `CLIPVisionOutput` + - Comfy dtype: `CLIP_VISION_OUTPUT` +- **`strength`** + - Determines the intensity of the applied enhancements, allowing for fine-tuning of the conditioning's influence. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`noise_augmentation`** + - Specifies the level of noise augmentation to apply, adding variability to the conditioning. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The enhanced conditioning data, ready for use in further processing or generation tasks. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: `ControlNetApplyAdvanced,ConditioningConcat` + + +## Source code +```python +class unCLIPConditioning: + @classmethod + def INPUT_TYPES(s): + return {"required": {"conditioning": ("CONDITIONING", ), + "clip_vision_output": ("CLIP_VISION_OUTPUT", ), + "strength": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + "noise_augmentation": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "apply_adm" + + CATEGORY = "conditioning" + + def apply_adm(self, conditioning, clip_vision_output, strength, noise_augmentation): + if strength == 0: + return (conditioning, ) + + c = [] + for t in conditioning: + o = t[1].copy() + x = {"clip_vision_output": clip_vision_output, "strength": strength, "noise_augmentation": noise_augmentation} + if "unclip_conditioning" in o: + o["unclip_conditioning"] = o["unclip_conditioning"][:] + [x] + else: + o["unclip_conditioning"] = [x] + n = [t[0], o] + c.append(n) + return (c, ) + +``` diff --git a/docs/md/Comfy/index.md b/docs/md/Comfy/index.md new file mode 100644 index 0000000000..5a35f4c4a9 --- /dev/null +++ b/docs/md/Comfy/index.md @@ -0,0 +1,3 @@ +# Comfy +- repo url: `https://github.com/comfyanonymous/ComfyUI` +- commit hash: `36f7face37320ec3c6ac85ec3c625738241deaa9` \ No newline at end of file diff --git a/docs/md/ComfyMath/Nodes/CM_BoolBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_BoolBinaryOperation.md new file mode 100644 index 0000000000..5d0c6a577d --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_BoolBinaryOperation.md @@ -0,0 +1,52 @@ +# BoolBinaryOperation +## Documentation +- Class name: `CM_BoolBinaryOperation` +- Category: `math/bool` +- Output node: `False` + +Performs a binary operation on two boolean inputs based on a specified operation. This node allows for the execution of various boolean logic operations such as AND, OR, XOR, etc., on the input booleans, producing a boolean result. +## Input types +### Required +- **`op`** + - Specifies the boolean binary operation to be performed. The choice of operation affects the logic applied to the input booleans and consequently the result. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first boolean input for the binary operation. Acts as one of the operands in the boolean logic operation. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +- **`b`** + - The second boolean input for the binary operation. Acts as the other operand in the boolean logic operation. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Output types +- **`bool`** + - The result of applying the specified boolean binary operation on the input booleans. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used with conditional logic nodes to apply boolean operations like AND, OR, XOR on two input booleans, resulting in a single boolean output that can influence the flow or decisions within the AI pipeline. +## Source code +```python +class BoolBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(BOOL_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_BOOL, + "b": DEFAULT_BOOL, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/bool" + + def op(self, op: str, a: bool, b: bool) -> tuple[bool]: + return (BOOL_BINARY_OPERATIONS[op](a, b),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_BoolToInt.md b/docs/md/ComfyMath/Nodes/CM_BoolToInt.md new file mode 100644 index 0000000000..a97dcf1b52 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_BoolToInt.md @@ -0,0 +1,38 @@ +# BoolToInt +## Documentation +- Class name: `CM_BoolToInt` +- Category: `math/conversion` +- Output node: `False` + +Converts a boolean value to an integer. This operation is a basic type conversion in mathematical computations, allowing for the transformation of true/false values into 0/1 integer representations. +## Input types +### Required +- **`a`** + - The boolean input to be converted to an integer. This parameter is fundamental for the conversion process, determining the output based on its truth value. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Output types +- **`int`** + - The integer representation of the input boolean value. This output is crucial for further mathematical operations that require integer inputs. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and logical operations where a boolean input needs to be converted to an integer for further processing, enabling seamless integration with nodes that require integer inputs for computations. +## Source code +```python +class BoolToInt: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("BOOL", {"default": False})}} + + RETURN_TYPES = ("INT",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: bool) -> tuple[int]: + return (int(a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_BoolUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_BoolUnaryOperation.md new file mode 100644 index 0000000000..20433af3ee --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_BoolUnaryOperation.md @@ -0,0 +1,44 @@ +# BoolUnaryOperation +## Documentation +- Class name: `CM_BoolUnaryOperation` +- Category: `math/bool` +- Output node: `False` + +Performs a unary boolean operation on a single boolean input. The operation is determined by the 'op' parameter, which selects from a predefined set of boolean operations such as logical NOT. +## Input types +### Required +- **`op`** + - Specifies the unary boolean operation to be performed. The choice of operation affects the outcome of the node's execution. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The boolean input on which the unary operation will be performed. The operation's effect is directly applied to this input. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Output types +- **`bool`** + - The result of applying the specified unary boolean operation on the input boolean value. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in logical flow control and decision-making processes within the pipeline, the CM_BoolUnaryOperation node performs a specified unary boolean operation, such as logical NOT, on a single boolean input to produce a boolean output. +## Source code +```python +class BoolUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": {"op": (list(BOOL_UNARY_OPERATIONS.keys()),), "a": DEFAULT_BOOL} + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/bool" + + def op(self, op: str, a: bool) -> tuple[bool]: + return (BOOL_UNARY_OPERATIONS[op](a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_BreakoutVec2.md b/docs/md/ComfyMath/Nodes/CM_BreakoutVec2.md new file mode 100644 index 0000000000..218abc6803 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_BreakoutVec2.md @@ -0,0 +1,38 @@ +# BreakoutVec2 +## Documentation +- Class name: `CM_BreakoutVec2` +- Category: `math/conversion` +- Output node: `False` + +This node breaks down a 2D vector into its individual components, providing a straightforward way to access and manipulate the x and y coordinates separately. +## Input types +### Required +- **`a`** + - The 2D vector to be broken down. This allows for the individual manipulation of the x and y components of the vector. + - Python dtype: `tuple[float, float]` + - Comfy dtype: `VEC2` +## Output types +- **`float`** + - The y component of the input 2D vector. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and conversion operations within the pipeline to separate a 2D vector into its x and y components, allowing for individual manipulation and analysis of these coordinates. +## Source code +```python +class BreakoutVec2: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("VEC2", {"default": VEC2_ZERO})}} + + RETURN_TYPES = ("FLOAT", "FLOAT") + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: Vec2) -> tuple[float, float]: + return (a[0], a[1]) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_BreakoutVec3.md b/docs/md/ComfyMath/Nodes/CM_BreakoutVec3.md new file mode 100644 index 0000000000..a635c174f5 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_BreakoutVec3.md @@ -0,0 +1,38 @@ +# BreakoutVec3 +## Documentation +- Class name: `CM_BreakoutVec3` +- Category: `math/conversion` +- Output node: `False` + +The `CM_BreakoutVec3` node is designed for decomposing a 3-dimensional vector into its individual components. This operation is useful in scenarios where the separate values of a vector are needed for further calculations or analysis. +## Input types +### Required +- **`a`** + - The input vector to be decomposed. It is essential for operations that require access to the individual components of a vector for further processing. + - Python dtype: `tuple[float, float, float]` + - Comfy dtype: `VEC3` +## Output types +- **`float`** + - The third component of the input vector. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in scenarios requiring the manipulation or analysis of individual vector components, such as adjusting positions in 3D space or performing calculations on specific dimensions. The `CM_BreakoutVec3` node takes a 3-dimensional vector as input and outputs its three components separately for further processing or analysis. +## Source code +```python +class BreakoutVec3: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("VEC3", {"default": VEC3_ZERO})}} + + RETURN_TYPES = ("FLOAT", "FLOAT", "FLOAT") + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: Vec3) -> tuple[float, float, float]: + return (a[0], a[1], a[2]) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_BreakoutVec4.md b/docs/md/ComfyMath/Nodes/CM_BreakoutVec4.md new file mode 100644 index 0000000000..6af648ad8d --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_BreakoutVec4.md @@ -0,0 +1,38 @@ +# BreakoutVec4 +## Documentation +- Class name: `CM_BreakoutVec4` +- Category: `math/conversion` +- Output node: `False` + +The `CM_BreakoutVec4` node is designed for decomposing a 4-dimensional vector into its individual components. This operation is useful in scenarios where the separate values of a vector are needed for further individual processing or analysis. +## Input types +### Required +- **`a`** + - This parameter represents the 4-dimensional vector to be decomposed. It is essential for the operation as it provides the data that will be broken down into individual float values. + - Python dtype: `tuple[float, float, float, float]` + - Comfy dtype: `VEC4` +## Output types +- **`float`** + - The fourth component of the input vector. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in scenarios requiring individual manipulation or analysis of the components of a 4-dimensional vector, such as adjusting specific properties in graphics or data visualization tasks. +## Source code +```python +class BreakoutVec4: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("VEC4", {"default": VEC4_ZERO})}} + + RETURN_TYPES = ("FLOAT", "FLOAT", "FLOAT", "FLOAT") + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: Vec4) -> tuple[float, float, float, float]: + return (a[0], a[1], a[2], a[3]) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_ComposeVec2.md b/docs/md/ComfyMath/Nodes/CM_ComposeVec2.md new file mode 100644 index 0000000000..eb07f95649 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_ComposeVec2.md @@ -0,0 +1,47 @@ +# ComposeVec2 +## Documentation +- Class name: `CM_ComposeVec2` +- Category: `math/conversion` +- Output node: `False` + +The `CM_ComposeVec2` node constructs a 2D vector (Vec2) from two provided float values, representing the x and y components of the vector. This operation is fundamental in vector mathematics and is used for converting scalar values into a vector format for further vector-based calculations. +## Input types +### Required +- **`x`** + - The x component of the Vec2. It defines the horizontal position or value in the 2D vector space. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`y`** + - The y component of the Vec2. It defines the vertical position or value in the 2D vector space. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`vec2`** + - The resulting 2D vector constructed from the input x and y components. + - Python dtype: `tuple[float, float]` + - Comfy dtype: `VEC2` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in vector-based calculations and transformations, the `CM_ComposeVec2` node takes two float values as input to construct a 2D vector, representing the x and y components, which is essential for operations requiring vector inputs or for converting scalar values into a vector format. +## Source code +```python +class ComposeVec2: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "x": ("FLOAT", {"default": 0.0}), + "y": ("FLOAT", {"default": 0.0}), + } + } + + RETURN_TYPES = ("VEC2",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, x: float, y: float) -> tuple[Vec2]: + return ((x, y),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_ComposeVec3.md b/docs/md/ComfyMath/Nodes/CM_ComposeVec3.md new file mode 100644 index 0000000000..3f9be92ded --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_ComposeVec3.md @@ -0,0 +1,52 @@ +# ComposeVec3 +## Documentation +- Class name: `CM_ComposeVec3` +- Category: `math/conversion` +- Output node: `False` + +The `ComposeVec3` node is designed for creating a 3-dimensional vector (Vec3) from three separate float values representing the x, y, and z coordinates. This operation is fundamental in vector mathematics and is often used in graphics programming, physics simulations, and other computational fields requiring spatial representation. +## Input types +### Required +- **`x`** + - The 'x' input represents the x-coordinate of the Vec3 to be composed. It is a crucial component in defining the spatial position or direction of the vector in 3D space. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`y`** + - The 'y' input signifies the y-coordinate of the Vec3. Along with 'x' and 'z', it helps in accurately defining the vector's position or orientation in a three-dimensional environment. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`z`** + - The 'z' input denotes the z-coordinate of the Vec3. It completes the 3D spatial information required to fully specify the vector, working in conjunction with 'x' and 'y'. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`vec3`** + - The output is a Vec3, a 3-dimensional vector composed of the input x, y, and z coordinates. It's a fundamental data structure in 3D computations, useful for representing points, directions, or velocities in space. + - Python dtype: `tuple[Vec3]` + - Comfy dtype: `VEC3` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The `CM_ComposeVec3` node is essential for constructing a 3-dimensional vector from individual x, y, and z float values, crucial for applications in 3D graphics, physics simulations, and spatial analysis. It is often used with nodes that require precise spatial positioning or directionality in a three-dimensional environment. +## Source code +```python +class ComposeVec3: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "x": ("FLOAT", {"default": 0.0}), + "y": ("FLOAT", {"default": 0.0}), + "z": ("FLOAT", {"default": 0.0}), + } + } + + RETURN_TYPES = ("VEC3",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, x: float, y: float, z: float) -> tuple[Vec3]: + return ((x, y, z),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_ComposeVec4.md b/docs/md/ComfyMath/Nodes/CM_ComposeVec4.md new file mode 100644 index 0000000000..9204c3264e --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_ComposeVec4.md @@ -0,0 +1,57 @@ +# ComposeVec4 +## Documentation +- Class name: `CM_ComposeVec4` +- Category: `math/conversion` +- Output node: `False` + +The `CM_ComposeVec4` node is designed for creating a 4-dimensional vector (Vec4) from four individual floating-point values. This operation is fundamental in mathematical and graphical computations where a Vec4 is required. +## Input types +### Required +- **`x`** + - Represents the first component of the Vec4. It's crucial for defining the vector's position or value along the x-axis. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`y`** + - Defines the second component of the Vec4, essential for specifying the vector's position or value along the y-axis. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`z`** + - Serves as the third component, vital for determining the vector's position or value along the z-axis. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`w`** + - Represents the fourth component, often used for additional data like transparency in graphical applications. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`vec4`** + - The resulting 4-dimensional vector composed from the input values. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in graphical computations and shader programming, the `CM_ComposeVec4` node allows for the creation of a 4-dimensional vector from individual x, y, z, and w floating-point values, essential for defining positions, colors, or other vector-based attributes in a scene. +## Source code +```python +class ComposeVec4: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "x": ("FLOAT", {"default": 0.0}), + "y": ("FLOAT", {"default": 0.0}), + "z": ("FLOAT", {"default": 0.0}), + "w": ("FLOAT", {"default": 0.0}), + } + } + + RETURN_TYPES = ("VEC4",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, x: float, y: float, z: float, w: float) -> tuple[Vec4]: + return ((x, y, z, w),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_FloatBinaryCondition.md b/docs/md/ComfyMath/Nodes/CM_FloatBinaryCondition.md new file mode 100644 index 0000000000..e967cc0ad3 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_FloatBinaryCondition.md @@ -0,0 +1,52 @@ +# FloatBinaryCondition +## Documentation +- Class name: `CM_FloatBinaryCondition` +- Category: `math/float` +- Output node: `False` + +This node evaluates a binary condition between two float values based on a specified operation. It supports a variety of conditions such as equality, inequality, greater than, less than, etc., and returns a boolean result indicating whether the condition holds. +## Input types +### Required +- **`op`** + - Specifies the binary condition operation to be applied between the two float values. The choice of operation determines how the values are compared and affects the boolean result of the node. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first float value to be compared. It plays a crucial role in the binary condition evaluation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`b`** + - The second float value to be compared. Together with the first value, it determines the outcome of the binary condition. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`bool`** + - The boolean result of the binary condition evaluation, indicating whether the specified condition holds between the two float values. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The CM_FloatBinaryCondition node is utilized for evaluating a binary condition (such as equality, inequality, greater than, less than, etc.) between two float values, returning a boolean result that indicates whether the specified condition is met. It is often used in decision-making processes within the pipeline, where the outcome of the condition can influence subsequent actions or the flow of data. +## Source code +```python +class FloatBinaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_BINARY_CONDITIONS.keys()),), + "a": DEFAULT_FLOAT, + "b": DEFAULT_FLOAT, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/float" + + def op(self, op: str, a: float, b: float) -> tuple[bool]: + return (FLOAT_BINARY_CONDITIONS[op](a, b),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_FloatBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_FloatBinaryOperation.md new file mode 100644 index 0000000000..15b22cb7a1 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_FloatBinaryOperation.md @@ -0,0 +1,52 @@ +# FloatBinaryOperation +## Documentation +- Class name: `CM_FloatBinaryOperation` +- Category: `math/float` +- Output node: `False` + +Executes a binary operation on two float inputs based on a specified operation. It supports a variety of operations such as addition, subtraction, multiplication, and division, allowing for flexible mathematical manipulations of float values. +## Input types +### Required +- **`op`** + - Specifies the binary operation to be performed on the inputs. The choice of operation directly influences the result, enabling diverse mathematical computations. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first float input for the binary operation. Acts as one of the operands in the mathematical computation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`b`** + - The second float input for the binary operation. Together with the first input, it completes the operands needed for the computation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`float`** + - The result of the binary operation performed on the two float inputs, returned as a float value. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical computations and data transformations, the CM_FloatBinaryOperation node performs a specified binary operation (addition, subtraction, multiplication, or division) on two float inputs, producing a single float output. +## Source code +```python +class FloatBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_FLOAT, + "b": DEFAULT_FLOAT, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/float" + + def op(self, op: str, a: float, b: float) -> tuple[float]: + return (FLOAT_BINARY_OPERATIONS[op](a, b),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_FloatToInt.md b/docs/md/ComfyMath/Nodes/CM_FloatToInt.md new file mode 100644 index 0000000000..59b83a1aab --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_FloatToInt.md @@ -0,0 +1,38 @@ +# FloatToInt +## Documentation +- Class name: `CM_FloatToInt` +- Category: `math/conversion` +- Output node: `False` + +Converts a floating-point number to an integer by truncating the decimal part. This operation is useful for scenarios where integer values are required from floating-point computations. +## Input types +### Required +- **`a`** + - The floating-point number to be converted to an integer. This parameter is crucial as it directly determines the output integer value by truncating any decimal part. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`int`** + - The integer result obtained by truncating the decimal part of the input floating-point number. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in scenarios requiring integer values from floating-point computations, such as setting discrete parameters or indexing, by truncating the decimal part of the input floating-point number. +## Source code +```python +class FloatToInt: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("FLOAT", {"default": 0.0})}} + + RETURN_TYPES = ("INT",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: float) -> tuple[int]: + return (int(a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_FloatToNumber.md b/docs/md/ComfyMath/Nodes/CM_FloatToNumber.md new file mode 100644 index 0000000000..1a4372e5b1 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_FloatToNumber.md @@ -0,0 +1,38 @@ +# FloatToNumber +## Documentation +- Class name: `CM_FloatToNumber` +- Category: `math/conversion` +- Output node: `False` + +The `CM_FloatToNumber` node converts a floating-point value to a generic number type. This operation is fundamental in scenarios where a uniform number type is required across different mathematical operations. +## Input types +### Required +- **`a`** + - The floating-point value to be converted into a generic number type. This parameter is essential for the conversion process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`number`** + - The result of the conversion, represented as a generic number type. + - Python dtype: `float` + - Comfy dtype: `NUMBER` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical operations and data normalization processes where a uniform number type is essential, the `CM_FloatToNumber` node converts a floating-point value into a generic number type, ensuring compatibility across various computational tasks. +## Source code +```python +class FloatToNumber: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("FLOAT", {"default": 0.0})}} + + RETURN_TYPES = ("NUMBER",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: float) -> tuple[number]: + return (a,) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_FloatUnaryCondition.md b/docs/md/ComfyMath/Nodes/CM_FloatUnaryCondition.md new file mode 100644 index 0000000000..4f3eab8de0 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_FloatUnaryCondition.md @@ -0,0 +1,47 @@ +# FloatUnaryCondition +## Documentation +- Class name: `CM_FloatUnaryCondition` +- Category: `math/float` +- Output node: `False` + +This node performs unary operations on a single float value based on a specified condition. It evaluates the condition and returns a boolean indicating whether the condition holds for the given float value. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed. The operation is chosen from a predefined list of conditions such as 'IsZero', 'IsPositive', 'IsNegative', 'IsNonZero', 'IsPositiveInfinity', 'IsNegativeInfinity', 'IsNaN', 'IsFinite', 'IsInfinite', 'IsEven', and 'IsOdd', which evaluate to either true or false. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The float value to be evaluated by the specified unary condition. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`bool`** + - The result of evaluating the specified unary condition on the input float value. It indicates whether the condition holds (true) or not (false). + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in decision-making processes within the pipeline to evaluate a single float value against a predefined condition, such as 'IsZero' or 'IsPositive', and returns a boolean indicating whether the condition is met. This node is crucial for filtering data or triggering specific pathways based on numerical conditions. +## Source code +```python +class FloatUnaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_UNARY_CONDITIONS.keys()),), + "a": DEFAULT_FLOAT, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/float" + + def op(self, op: str, a: float) -> tuple[bool]: + return (FLOAT_UNARY_CONDITIONS[op](a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_FloatUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_FloatUnaryOperation.md new file mode 100644 index 0000000000..cea5b52d07 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_FloatUnaryOperation.md @@ -0,0 +1,47 @@ +# FloatUnaryOperation +## Documentation +- Class name: `CM_FloatUnaryOperation` +- Category: `math/float` +- Output node: `False` + +Performs a unary operation on a single float value. The operation is selected from a predefined list of float unary operations, and applied to the input float, producing a single float result. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the input float. The operation is chosen from a predefined list of available float unary operations, which determines how the input float is processed. The choice of operation directly influences the computation performed on the input, affecting the node's output in a specific manner. Understanding the specific operation selected is crucial as it dictates the mathematical transformation applied to the input, thereby shaping the final output in a distinct way. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The input float value to which the specified unary operation is applied. This value is the subject of the unary operation, determining the output of the node. The nature of this input is crucial as it directly impacts the result of the operation, influencing the computation's outcome. The magnitude and sign of this input float significantly affect the operation's effect, thereby altering the computation's final result. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`float`** + - The result of applying the specified unary operation to the input float value. This output is a single float, representing the outcome of the operation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The CM_FloatUnaryOperation node is utilized for applying a specific unary operation, chosen from a predefined list, to a single float input value, resulting in a single float output. This node is often used in mathematical and data transformation pipelines to modify the magnitude or sign of an input float, thereby altering its value according to the selected operation. +## Source code +```python +class FloatUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_UNARY_OPERATIONS.keys()),), + "a": DEFAULT_FLOAT, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/float" + + def op(self, op: str, a: float) -> tuple[float]: + return (FLOAT_UNARY_OPERATIONS[op](a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntBinaryCondition.md b/docs/md/ComfyMath/Nodes/CM_IntBinaryCondition.md new file mode 100644 index 0000000000..60929b6dac --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntBinaryCondition.md @@ -0,0 +1,52 @@ +# IntBinaryCondition +## Documentation +- Class name: `CM_IntBinaryCondition` +- Category: `math/int` +- Output node: `False` + +This node performs a binary condition operation on two integer inputs based on a specified operation (e.g., equality, greater than). It evaluates the condition and returns a boolean result indicating whether the condition holds. +## Input types +### Required +- **`op`** + - Specifies the binary condition operation to perform on the inputs, such as equality or comparison. Its choice directly influences the result of the node, determining the nature of the condition being evaluated and thus affecting the boolean outcome. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first integer input for the binary condition operation. This value serves as one of the operands in the binary condition, playing a crucial role in the evaluation process and influencing the final boolean result. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`b`** + - The second integer input for the binary condition operation. Similar to the first operand, this value is essential in the condition evaluation, contributing to the determination of the boolean outcome. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`bool`** + - The boolean result of the binary condition operation, indicating whether the specified condition between the two integer inputs holds. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in decision-making processes within the pipeline to compare two integer values based on a specified operation (e.g., equality, greater than), and outputs a boolean result indicating the outcome of the comparison. +## Source code +```python +class IntBinaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(INT_BINARY_CONDITIONS.keys()),), + "a": DEFAULT_INT, + "b": DEFAULT_INT, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/int" + + def op(self, op: str, a: int, b: int) -> tuple[bool]: + return (INT_BINARY_CONDITIONS[op](a, b),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_IntBinaryOperation.md new file mode 100644 index 0000000000..67bc00539e --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntBinaryOperation.md @@ -0,0 +1,52 @@ +# IntBinaryOperation +## Documentation +- Class name: `CM_IntBinaryOperation` +- Category: `math/int` +- Output node: `False` + +Performs a binary operation on two integer inputs based on a specified operation. This node allows for basic arithmetic or comparison operations between integers, producing an integer result. +## Input types +### Required +- **`op`** + - Determines the specific binary operation (e.g., addition, subtraction) to apply to the integer inputs. The choice of operation directly influences the nature of the computation and the final outcome, making it a crucial aspect of the node's functionality. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - Represents the first operand in the binary operation. Its value is essential as it serves as one of the two primary inputs upon which the specified operation is executed, directly impacting the result. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`b`** + - Serves as the second operand in the binary operation. Similar to the first operand, its value is critical as it contributes to the computation process, affecting the overall result of the operation. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`int`** + - The integer result of the specified binary operation on the inputs. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and logical operations within the pipeline, the CM_IntBinaryOperation node performs specified arithmetic or comparison operations on two integer inputs, producing an integer result. +## Source code +```python +class IntBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(INT_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_INT, + "b": DEFAULT_INT, + } + } + + RETURN_TYPES = ("INT",) + FUNCTION = "op" + CATEGORY = "math/int" + + def op(self, op: str, a: int, b: int) -> tuple[int]: + return (INT_BINARY_OPERATIONS[op](a, b),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntToBool.md b/docs/md/ComfyMath/Nodes/CM_IntToBool.md new file mode 100644 index 0000000000..f3cf99d56d --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntToBool.md @@ -0,0 +1,38 @@ +# IntToBool +## Documentation +- Class name: `CM_IntToBool` +- Category: `math/conversion` +- Output node: `False` + +The `CM_IntToBool` node converts an integer input to a boolean output. This operation is based on the principle that any non-zero integer will be considered `True`, while zero is considered `False`. +## Input types +### Required +- **`a`** + - The integer input to be converted to a boolean. This parameter determines the boolean output based on its value being zero (False) or non-zero (True). + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`bool`** + - The boolean result of the conversion, indicating whether the input integer was non-zero (True) or zero (False). + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in decision-making processes within the pipeline, where the need to convert numerical thresholds into boolean flags is essential, such as enabling or disabling subsequent AI model nodes based on integer inputs. +## Source code +```python +class IntToBool: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("INT", {"default": 0})}} + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: int) -> tuple[bool]: + return (a != 0,) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntToFloat.md b/docs/md/ComfyMath/Nodes/CM_IntToFloat.md new file mode 100644 index 0000000000..c30b5f25fa --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntToFloat.md @@ -0,0 +1,38 @@ +# IntToFloat +## Documentation +- Class name: `CM_IntToFloat` +- Category: `math/conversion` +- Output node: `False` + +Converts an integer value to a floating-point number. This operation is a basic type conversion in mathematical computations, allowing for more precise calculations that require decimal points. +## Input types +### Required +- **`a`** + - The integer value to be converted into a floating-point number. This conversion is essential for operations that require decimal precision. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`float`** + - The resulting floating-point number after conversion. This allows for more precise mathematical operations that involve decimals. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and computational pipelines requiring decimal precision, the CM_IntToFloat node converts an integer input ('a') into a floating-point number, facilitating operations that involve decimals. +## Source code +```python +class IntToFloat: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("INT", {"default": 0})}} + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: int) -> tuple[float]: + return (float(a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntToNumber.md b/docs/md/ComfyMath/Nodes/CM_IntToNumber.md new file mode 100644 index 0000000000..769fa25596 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntToNumber.md @@ -0,0 +1,38 @@ +# IntToNumber +## Documentation +- Class name: `CM_IntToNumber` +- Category: `math/conversion` +- Output node: `False` + +Converts an integer to a generic number format. This operation is a basic type conversion that allows integers to be used in contexts where a more general number type is expected. +## Input types +### Required +- **`a`** + - The integer value to be converted into a generic number format. This conversion is straightforward and preserves the value. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`number`** + - The result of converting the input integer to a generic number format. The output is directly equivalent to the input value, but in a more flexible number format. + - Python dtype: `int | float` + - Comfy dtype: `NUMBER` +## Usage tips +- Infra type: `CPU` +- Common nodes: `CM_NumberBinaryOperation` + +Often used with nodes requiring numerical inputs in non-integer formats, such as the Eff. Loader SDXL for combining model outputs and configurations, ensuring seamless data type compatibility and processing efficiency within the pipeline. +## Source code +```python +class IntToNumber: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("INT", {"default": 0})}} + + RETURN_TYPES = ("NUMBER",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: int) -> tuple[number]: + return (a,) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntUnaryCondition.md b/docs/md/ComfyMath/Nodes/CM_IntUnaryCondition.md new file mode 100644 index 0000000000..75295f7ce6 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntUnaryCondition.md @@ -0,0 +1,44 @@ +# IntUnaryCondition +## Documentation +- Class name: `CM_IntUnaryCondition` +- Category: `math/int` +- Output node: `False` + +This node performs unary operations on integers based on a specified condition. It evaluates a single integer input against a unary condition (e.g., checking if the integer is positive, negative, or zero) and returns a boolean indicating the result of the condition. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the integer. The choice of operation determines how the integer is evaluated against the condition. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The integer to be evaluated against the specified unary condition. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`bool`** + - A boolean indicating the result of evaluating the integer against the specified unary condition. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in decision-making processes within the pipeline to evaluate an integer's property, such as checking if it is positive, negative, or zero, and outputs a boolean result based on the specified condition. +## Source code +```python +class IntUnaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": {"op": (list(INT_UNARY_CONDITIONS.keys()),), "a": DEFAULT_INT} + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/int" + + def op(self, op: str, a: int) -> tuple[bool]: + return (INT_UNARY_CONDITIONS[op](a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_IntUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_IntUnaryOperation.md new file mode 100644 index 0000000000..a2ae8b7505 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_IntUnaryOperation.md @@ -0,0 +1,44 @@ +# IntUnaryOperation +## Documentation +- Class name: `CM_IntUnaryOperation` +- Category: `math/int` +- Output node: `False` + +Performs a unary operation on an integer. This operation is defined by a specific operation (op) applied to an integer (a), resulting in a modified integer value. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the integer. The choice of operation significantly affects the modification of the input integer, altering its value based on the operation selected. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The integer on which the unary operation is to be performed. It serves as the input to the operation defined by 'op'. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`int`** + - The result of applying the specified unary operation on the input integer. It is an integer reflecting the outcome of the operation. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and data transformation pipelines to apply a specific unary operation, such as incrementing or negating, on an input integer `a`, resulting in a modified integer output. +## Source code +```python +class IntUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": {"op": (list(INT_UNARY_OPERATIONS.keys()),), "a": DEFAULT_INT} + } + + RETURN_TYPES = ("INT",) + FUNCTION = "op" + CATEGORY = "math/int" + + def op(self, op: str, a: int) -> tuple[int]: + return (INT_UNARY_OPERATIONS[op](a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NearestSDXLResolution.md b/docs/md/ComfyMath/Nodes/CM_NearestSDXLResolution.md new file mode 100644 index 0000000000..08d81cb78d --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NearestSDXLResolution.md @@ -0,0 +1,61 @@ +# NearestSDXLResolution +## Documentation +- Class name: `CM_NearestSDXLResolution` +- Category: `math/graphics` +- Output node: `False` + +This node finds the nearest supported SDXL resolution to the input image's resolution. It calculates the aspect ratio of the input image and selects the SDXL resolution with the closest aspect ratio, defaulting to 1024x1024 if no closer match is found. +## Input types +### Required +- **`image`** + - The input image for which the nearest SDXL resolution is to be found. The image's aspect ratio is used to determine the closest supported SDXL resolution. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Output types +- **`int`** + - The height component of the nearest SDXL resolution. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `GPU` +- Common nodes: `ConstrainImage|pysssss,Reroute,ImageScale` + +Often used with image preprocessing to ensure the input image is compatible with SDXL model requirements by adjusting its resolution to the nearest supported SDXL resolution, ensuring optimal performance and quality in AI-driven graphics and image processing tasks. +## Source code +```python +class NearestSDXLResolution: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"image": ("IMAGE",)}} + + RETURN_TYPES = ("INT", "INT") + RETURN_NAMES = ("width", "height") + FUNCTION = "op" + CATEGORY = "math/graphics" + + def op(self, image) -> tuple[int, int]: + image_width = image.size()[2] + image_height = image.size()[1] + print(f"Input image resolution: {image_width}x{image_height}") + image_ratio = image_width / image_height + differences = [ + (abs(image_ratio - resolution[2]), resolution) + for resolution in SDXL_SUPPORTED_RESOLUTIONS + ] + smallest = None + for difference in differences: + if smallest is None: + smallest = difference + else: + if difference[0] < smallest[0]: + smallest = difference + if smallest is not None: + width = smallest[1][0] + height = smallest[1][1] + else: + width = 1024 + height = 1024 + print(f"Selected SDXL resolution: {width}x{height}") + return (width, height) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NumberBinaryCondition.md b/docs/md/ComfyMath/Nodes/CM_NumberBinaryCondition.md new file mode 100644 index 0000000000..a1eac283d1 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NumberBinaryCondition.md @@ -0,0 +1,52 @@ +# NumberBinaryCondition +## Documentation +- Class name: `CM_NumberBinaryCondition` +- Category: `math/float` +- Output node: `False` + +This node performs a binary condition operation on two numbers, evaluating a specified condition between them. It supports a variety of conditions, such as equality, inequality, and comparison operations, determined by the 'op' parameter. +## Input types +### Required +- **`op`** + - Specifies the binary condition operation to be performed, such as equality or inequality. The choice of operation directly influences the comparison or evaluation process between the two input numbers, impacting the node's execution and the nature of the result. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first number to be compared or evaluated in the binary condition operation. Its value plays a crucial role in determining the outcome of the specified condition, affecting the node's execution and results. + - Python dtype: `float` + - Comfy dtype: `NUMBER` +- **`b`** + - The second number to be compared or evaluated, alongside the first number, in the binary condition operation. Its value is essential in evaluating the specified condition, influencing the node's execution and the final outcome. + - Python dtype: `float` + - Comfy dtype: `NUMBER` +## Output types +- **`bool`** + - The result of the binary condition operation, indicating whether the specified condition holds true between the two input numbers. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in decision-making processes within the pipeline to compare two numbers based on a specified condition, such as equality, greater than, or less than, influencing the flow or outcome of the AI application. +## Source code +```python +class NumberBinaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_BINARY_CONDITIONS.keys()),), + "a": DEFAULT_NUMBER, + "b": DEFAULT_NUMBER, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/float" + + def op(self, op: str, a: number, b: number) -> tuple[bool]: + return (FLOAT_BINARY_CONDITIONS[op](float(a), float(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NumberBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_NumberBinaryOperation.md new file mode 100644 index 0000000000..6177df1451 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NumberBinaryOperation.md @@ -0,0 +1,52 @@ +# NumberBinaryOperation +## Documentation +- Class name: `CM_NumberBinaryOperation` +- Category: `math/number` +- Output node: `False` + +Performs a binary operation on two numbers based on a specified operation. This operation can be any function defined in FLOAT_BINARY_OPERATIONS, allowing for a wide range of mathematical computations between two numbers. +## Input types +### Required +- **`op`** + - Specifies the binary operation to perform. The choice of operation significantly influences the result, as it determines how the two input numbers are mathematically combined. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first input number for the binary operation. Acts as one of the operands in the mathematical computation. + - Python dtype: `float or int` + - Comfy dtype: `NUMBER` +- **`b`** + - The second input number for the binary operation. Serves as the other operand, together with 'a', in the computation. + - Python dtype: `float or int` + - Comfy dtype: `NUMBER` +## Output types +- **`number`** + - The result of the binary operation performed on the two input numbers. + - Python dtype: `float` + - Comfy dtype: `NUMBER` +## Usage tips +- Infra type: `CPU` +- Common nodes: `CM_NumberToInt` + +The CM_NumberBinaryOperation node is crucial for performing mathematical adjustments or calculations between numerical values within the pipeline, such as adjusting image dimensions or calculating aspect ratios. It accepts two numerical inputs, applies a specified binary operation (addition, subtraction, multiplication, or division), and outputs the resulting number, facilitating precise control over numerical parameters in AI-driven video content creation. +## Source code +```python +class NumberBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_NUMBER, + "b": DEFAULT_NUMBER, + } + } + + RETURN_TYPES = ("NUMBER",) + FUNCTION = "op" + CATEGORY = "math/number" + + def op(self, op: str, a: number, b: number) -> tuple[float]: + return (FLOAT_BINARY_OPERATIONS[op](float(a), float(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NumberToFloat.md b/docs/md/ComfyMath/Nodes/CM_NumberToFloat.md new file mode 100644 index 0000000000..71d7248559 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NumberToFloat.md @@ -0,0 +1,38 @@ +# NumberToFloat +## Documentation +- Class name: `CM_NumberToFloat` +- Category: `math/conversion` +- Output node: `False` + +Converts a number (either integer or float) to a floating-point representation. This operation is useful for ensuring that numerical inputs are in a consistent format for further mathematical operations. +## Input types +### Required +- **`a`** + - The number to be converted to a floating-point representation. This conversion is essential for maintaining numerical consistency across operations that require floating-point inputs. + - Python dtype: `Union[int, float]` + - Comfy dtype: `NUMBER` +## Output types +- **`float`** + - The floating-point representation of the input number. This output is crucial for operations that specifically require floating-point numbers. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `CPU` +- Common nodes: `FaceDetailer,CR Integer Multiple` + +The CM_NumberToFloat node is essential for converting numerical input data, whether integer or another number format, to a floating-point format. This conversion is crucial for ensuring numerical consistency and precision in operations that require floating-point inputs, such as scaling factors or detailed adjustments in advanced facial enhancement and transformation pipelines. +## Source code +```python +class NumberToFloat: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("NUMBER", {"default": 0.0})}} + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: number) -> tuple[float]: + return (float(a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NumberToInt.md b/docs/md/ComfyMath/Nodes/CM_NumberToInt.md new file mode 100644 index 0000000000..f2437a4a99 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NumberToInt.md @@ -0,0 +1,38 @@ +# NumberToInt +## Documentation +- Class name: `CM_NumberToInt` +- Category: `math/conversion` +- Output node: `False` + +Converts a number (either integer or float) to an integer. This operation is useful for scenarios where integer values are required, such as indexing or discrete operations. +## Input types +### Required +- **`a`** + - The number to be converted to an integer. This can be either an integer or a float, making the node versatile for different numeric types. + - Python dtype: `Union[int, float]` + - Comfy dtype: `NUMBER` +## Output types +- **`int`** + - The integer representation of the input number. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `CPU` +- Common nodes: `SVD_img2vid_Conditioning` + +The CM_NumberToInt node is crucial for converting numerical values from floating-point to integer format, often used in scenarios requiring integer inputs, such as indexing or discrete operations within the pipeline. +## Source code +```python +class NumberToInt: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return {"required": {"a": ("NUMBER", {"default": 0.0})}} + + RETURN_TYPES = ("INT",) + FUNCTION = "op" + CATEGORY = "math/conversion" + + def op(self, a: number) -> tuple[int]: + return (int(a),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NumberUnaryCondition.md b/docs/md/ComfyMath/Nodes/CM_NumberUnaryCondition.md new file mode 100644 index 0000000000..4ff67ec6e7 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NumberUnaryCondition.md @@ -0,0 +1,47 @@ +# NumberUnaryCondition +## Documentation +- Class name: `CM_NumberUnaryCondition` +- Category: `math/number` +- Output node: `False` + +This node performs unary operations on numbers based on predefined conditions. It takes a number and applies a specified unary condition operation to it, returning a boolean result indicating whether the condition is met. +## Input types +### Required +- **`op`** + - Specifies the unary condition operation to be applied to the input number. The choice of operation determines how the input number is evaluated against the condition. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The input number to which the unary condition operation is applied. The operation's outcome depends on this number's value. + - Python dtype: `Union[int, float]` + - Comfy dtype: `NUMBER` +## Output types +- **`bool`** + - The result of applying the unary condition operation to the input number, indicating whether the condition is met. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The CM_NumberUnaryCondition node is primarily utilized for evaluating numerical values against specific conditions, such as checking if a number is greater than a certain value. It accepts a number and a condition operation as input, and outputs a boolean indicating whether the number meets the specified condition. +## Source code +```python +class NumberUnaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_UNARY_CONDITIONS.keys()),), + "a": DEFAULT_NUMBER, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/number" + + def op(self, op: str, a: number) -> tuple[bool]: + return (FLOAT_UNARY_CONDITIONS[op](float(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_NumberUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_NumberUnaryOperation.md new file mode 100644 index 0000000000..3432f62f43 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_NumberUnaryOperation.md @@ -0,0 +1,47 @@ +# NumberUnaryOperation +## Documentation +- Class name: `CM_NumberUnaryOperation` +- Category: `math/number` +- Output node: `False` + +Performs a unary operation on a number, where the operation is one of the predefined operations. This method applies the specified operation to the input number and returns the result as a number. +## Input types +### Required +- **`op`** + - Specifies the operation to be performed on the input number, influencing the computation result. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The input number on which the unary operation will be performed. It can be an integer or a float, and it significantly influences the outcome of the operation. + - Python dtype: `Union[int, float]` + - Comfy dtype: `NUMBER` +## Output types +- **`number`** + - The result of applying the specified unary operation on the input number, returned as a number. + - Python dtype: `float` + - Comfy dtype: `NUMBER` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The CM_NumberUnaryOperation node is primarily utilized for performing a specified unary mathematical operation on a single input number, such as negation or reciprocal, and outputs the result as a number. It is often used in conjunction with nodes requiring mathematical transformations or adjustments to numerical inputs for further processing or analysis. +## Source code +```python +class NumberUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(FLOAT_UNARY_OPERATIONS.keys()),), + "a": DEFAULT_NUMBER, + } + } + + RETURN_TYPES = ("NUMBER",) + FUNCTION = "op" + CATEGORY = "math/number" + + def op(self, op: str, a: number) -> tuple[float]: + return (FLOAT_UNARY_OPERATIONS[op](float(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_SDXLResolution.md b/docs/md/ComfyMath/Nodes/CM_SDXLResolution.md new file mode 100644 index 0000000000..0886e9260a --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_SDXLResolution.md @@ -0,0 +1,46 @@ +# SDXLResolution +## Documentation +- Class name: `CM_SDXLResolution` +- Category: `math/graphics` +- Output node: `False` + +This node takes a resolution string as input and returns the width and height as separate integers. It is designed to work with specific resolutions supported by the SDXL framework, parsing the input string to extract the numerical values. +## Input types +### Required +- **`resolution`** + - Specifies the desired resolution in a 'width x height' format string. It must match one of the predefined resolutions supported by the SDXL framework, affecting the output width and height directly. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`int`** + - The height component of the specified resolution, extracted from the input string. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `CPU` +- Common nodes: `Eff. Loader SDXL,intToFloat _O` + +The CM_SDXLResolution node is primarily utilized for setting the target resolution for image processing tasks within the pipeline, by taking a resolution string as input and outputting the corresponding width and height as integers. This node ensures that AI models and custom nodes operate on images at consistent and optimal resolutions, as defined by the user or the pipeline's requirements. +## Source code +```python +class SDXLResolution: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "resolution": ( + [f"{res[0]}x{res[1]}" for res in SDXL_SUPPORTED_RESOLUTIONS], + ) + } + } + + RETURN_TYPES = ("INT", "INT") + RETURN_NAMES = ("width", "height") + FUNCTION = "op" + CATEGORY = "math/graphics" + + def op(self, resolution: str) -> tuple[int, int]: + width, height = resolution.split("x") + return (int(width), int(height)) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2BinaryCondition.md b/docs/md/ComfyMath/Nodes/CM_Vec2BinaryCondition.md new file mode 100644 index 0000000000..6f099b3b65 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2BinaryCondition.md @@ -0,0 +1,52 @@ +# Vec2BinaryCondition +## Documentation +- Class name: `CM_Vec2BinaryCondition` +- Category: `math/vec2` +- Output node: `False` + +Performs a binary condition operation between two 2-dimensional vectors based on a specified operation. This operation evaluates to a boolean value indicating the result of the condition. +## Input types +### Required +- **`op`** + - Specifies the binary condition operation to be performed between the two vectors. The choice of operation determines how the vectors are compared. + - Python dtype: `List[str]` + - Comfy dtype: `STRING` +- **`a`** + - The first 2-dimensional vector involved in the binary condition operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +- **`b`** + - The second 2-dimensional vector involved in the binary condition operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +## Output types +- **`bool`** + - The result of the binary condition operation as a boolean value, indicating whether the specified condition holds true between the two vectors. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in decision-making processes within the pipeline, where comparing two 2-dimensional vectors is necessary to determine the flow based on a specified condition, such as equality or magnitude comparison. The CM_Vec2BinaryCondition node takes two vectors as input and outputs a boolean value indicating whether the specified condition holds true between them. +## Source code +```python +class Vec2BinaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_BINARY_CONDITIONS.keys()),), + "a": DEFAULT_VEC2, + "b": DEFAULT_VEC2, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2, b: Vec2) -> tuple[bool]: + return (VEC_BINARY_CONDITIONS[op](numpy.array(a), numpy.array(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2BinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec2BinaryOperation.md new file mode 100644 index 0000000000..bdbe5c9c6f --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2BinaryOperation.md @@ -0,0 +1,54 @@ +# Vec2BinaryOperation +## Documentation +- Class name: `CM_Vec2BinaryOperation` +- Category: `math/vec2` +- Output node: `False` + +Performs a binary operation on two Vec2 vectors, resulting in a new Vec2 vector. This operation is defined by a specific operation ('op') applied to the input vectors 'a' and 'b'. +## Input types +### Required +- **`op`** + - Specifies the binary operation to be applied to the vectors. The operation is chosen from a predefined list of vector operations, affecting the result of the node's execution. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first input Vec2 vector for the binary operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +- **`b`** + - The second input Vec2 vector for the binary operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +## Output types +- **`vec2`** + - The resulting Vec2 vector after applying the specified binary operation on the input vectors. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and geometric operations requiring the manipulation of vector pairs, the CM_Vec2BinaryOperation node applies a specified binary operation (such as addition, subtraction, etc.) on two input Vec2 vectors 'a' and 'b', resulting in a new Vec2 vector as output. +## Source code +```python +class Vec2BinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_VEC2, + "b": DEFAULT_VEC2, + } + } + + RETURN_TYPES = ("VEC2",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2, b: Vec2) -> tuple[Vec2]: + return ( + _vec2_from_numpy(VEC_BINARY_OPERATIONS[op](numpy.array(a), numpy.array(b))), + ) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2ScalarOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec2ScalarOperation.md new file mode 100644 index 0000000000..727c980d24 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2ScalarOperation.md @@ -0,0 +1,52 @@ +# Vec2ScalarOperation +## Documentation +- Class name: `CM_Vec2ScalarOperation` +- Category: `math/vec2` +- Output node: `False` + +Performs a scalar operation on a 2D vector. The operation is defined by a set of predefined scalar operations such as multiplication or division, applied between the 2D vector and a scalar value. +## Input types +### Required +- **`op`** + - Specifies the scalar operation to be performed. It determines how the 2D vector and the scalar value will interact, affecting the result of the operation. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 2D vector to which the scalar operation will be applied. It serves as the primary operand in the operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +- **`b`** + - The scalar value to be used in the operation. It interacts with the 2D vector according to the specified operation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`vec2`** + - The resulting 2D vector after applying the specified scalar operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and graphical computations, the CM_Vec2ScalarOperation node applies a specified scalar operation, such as multiplication or division, between a 2D vector and a scalar value, resulting in a modified 2D vector. +## Source code +```python +class Vec2ScalarOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_SCALAR_OPERATION.keys()),), + "a": DEFAULT_VEC2, + "b": ("FLOAT",), + } + } + + RETURN_TYPES = ("VEC2",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2, b: float) -> tuple[Vec2]: + return (_vec2_from_numpy(VEC_SCALAR_OPERATION[op](numpy.array(a), b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2ToScalarBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec2ToScalarBinaryOperation.md new file mode 100644 index 0000000000..a6ed0cd125 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2ToScalarBinaryOperation.md @@ -0,0 +1,52 @@ +# Vec2ToScalarBinaryOperation +## Documentation +- Class name: `CM_Vec2ToScalarBinaryOperation` +- Category: `math/vec2` +- Output node: `False` + +Performs a binary operation between two 2D vectors and returns a scalar value. The operation is defined by a set of predefined operations such as dot product or distance calculation. +## Input types +### Required +- **`op`** + - Specifies the binary operation to perform on the two vectors. It determines the nature of the computation, such as calculating the dot product or the distance between vectors. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first 2D vector involved in the binary operation. + - Python dtype: `tuple[float, float]` + - Comfy dtype: `VEC2` +- **`b`** + - The second 2D vector involved in the binary operation. + - Python dtype: `tuple[float, float]` + - Comfy dtype: `VEC2` +## Output types +- **`float`** + - The scalar result of the binary operation performed on the two vectors. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric computations within the pipeline, the CM_Vec2ToScalarBinaryOperation node performs a predefined binary operation, such as calculating the dot product or the distance, between two 2D vectors (inputs a and b) and outputs a scalar value. +## Source code +```python +class Vec2ToScalarBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_TO_SCALAR_BINARY_OPERATION.keys()),), + "a": DEFAULT_VEC2, + "b": DEFAULT_VEC2, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2, b: Vec2) -> tuple[float]: + return (VEC_TO_SCALAR_BINARY_OPERATION[op](numpy.array(a), numpy.array(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2ToScalarUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec2ToScalarUnaryOperation.md new file mode 100644 index 0000000000..debebc73b1 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2ToScalarUnaryOperation.md @@ -0,0 +1,47 @@ +# Vec2ToScalarUnaryOperation +## Documentation +- Class name: `CM_Vec2ToScalarUnaryOperation` +- Category: `math/vec2` +- Output node: `False` + +Performs a unary operation that transforms a 2D vector into a scalar value based on a specified operation. This operation is defined by a set of predefined functions that take a 2D vector as input and return a scalar value. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the 2D vector. The choice of operation significantly influences the transformation process, determining the mathematical manipulation applied to convert the vector into a scalar value. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 2D vector on which the unary operation is to be performed. It serves as the input to the chosen operation, and its structure is crucial for the operation's execution. + - Python dtype: `tuple[float, float]` + - Comfy dtype: `VEC2` +## Output types +- **`float`** + - The scalar result of the unary operation performed on the 2D vector. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric computations within the pipeline, the CM_Vec2ToScalarUnaryOperation node transforms a 2D vector into a scalar value by applying a specified unary operation, such as calculating the magnitude or performing a custom mathematical function. This node takes a 2D vector as input and outputs a scalar value, making it essential for operations that require dimensionality reduction or specific mathematical transformations. +## Source code +```python +class Vec2ToScalarUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_TO_SCALAR_UNARY_OPERATION.keys()),), + "a": DEFAULT_VEC2, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2) -> tuple[float]: + return (VEC_TO_SCALAR_UNARY_OPERATION[op](numpy.array(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2UnaryCondition.md b/docs/md/ComfyMath/Nodes/CM_Vec2UnaryCondition.md new file mode 100644 index 0000000000..2c3156fd6d --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2UnaryCondition.md @@ -0,0 +1,47 @@ +# Vec2UnaryCondition +## Documentation +- Class name: `CM_Vec2UnaryCondition` +- Category: `math/vec2` +- Output node: `False` + +Performs a unary condition operation on a 2-dimensional vector. It evaluates a condition specified by the operation on the vector and returns a boolean result. +## Input types +### Required +- **`op`** + - Specifies the unary condition operation to be performed on the vector. The choice of operation determines how the vector is evaluated to produce a boolean result. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 2-dimensional vector on which the unary condition operation is performed. It serves as the input vector for the condition evaluation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +## Output types +- **`bool`** + - The result of the unary condition operation, indicating whether the condition is met for the input vector. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in mathematical and logical operations within the pipeline to evaluate if a 2-dimensional vector meets a specific condition, resulting in a boolean output that can influence subsequent actions or decisions. +## Source code +```python +class Vec2UnaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_UNARY_CONDITIONS.keys()),), + "a": DEFAULT_VEC2, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2) -> tuple[bool]: + return (VEC_UNARY_CONDITIONS[op](numpy.array(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec2UnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec2UnaryOperation.md new file mode 100644 index 0000000000..fad08d69e3 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec2UnaryOperation.md @@ -0,0 +1,47 @@ +# Vec2UnaryOperation +## Documentation +- Class name: `CM_Vec2UnaryOperation` +- Category: `math/vec2` +- Output node: `False` + +Performs a unary operation on a 2D vector, transforming it based on a specified operation. This allows for the application of various mathematical operations to modify the vector's properties. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be applied to the vector. The choice of operation determines how the vector is transformed. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 2D vector to be transformed. It serves as the input for the unary operation. + - Python dtype: `Vec2` + - Comfy dtype: `VEC2` +## Output types +- **`vec2`** + - The transformed 2D vector after applying the specified unary operation. + - Python dtype: `tuple[Vec2]` + - Comfy dtype: `VEC2` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in vector manipulation tasks to apply mathematical transformations to 2D vectors, such as normalization or rotation, based on the specified unary operation. This node takes a 2D vector as input and outputs the transformed vector, enabling precise control over vector properties in the pipeline. +## Source code +```python +class Vec2UnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_UNARY_OPERATIONS.keys()),), + "a": DEFAULT_VEC2, + } + } + + RETURN_TYPES = ("VEC2",) + FUNCTION = "op" + CATEGORY = "math/vec2" + + def op(self, op: str, a: Vec2) -> tuple[Vec2]: + return (_vec2_from_numpy(VEC_UNARY_OPERATIONS[op](numpy.array(a))),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3BinaryCondition.md b/docs/md/ComfyMath/Nodes/CM_Vec3BinaryCondition.md new file mode 100644 index 0000000000..7c68792866 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3BinaryCondition.md @@ -0,0 +1,52 @@ +# Vec3BinaryCondition +## Documentation +- Class name: `CM_Vec3BinaryCondition` +- Category: `math/vec3` +- Output node: `False` + +This node performs a binary condition operation between two 3-dimensional vectors (Vec3) based on a specified operation. It evaluates the condition and returns a boolean result indicating whether the condition holds. +## Input types +### Required +- **`op`** + - Specifies the binary condition operation to be performed between the two vectors. The choice of operation affects how the condition is evaluated and ultimately determines the boolean result. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first 3-dimensional vector involved in the binary condition operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +- **`b`** + - The second 3-dimensional vector involved in the binary condition operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Output types +- **`bool`** + - The boolean outcome of the binary condition operation, indicating whether the specified condition holds between the two vectors. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in scenarios requiring comparison between two 3-dimensional vectors, such as determining spatial relationships or conditions in simulations and visualizations, by evaluating a specified binary condition and outputting a boolean result indicating whether the condition holds. +## Source code +```python +class Vec3BinaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_BINARY_CONDITIONS.keys()),), + "a": DEFAULT_VEC3, + "b": DEFAULT_VEC3, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3, b: Vec3) -> tuple[bool]: + return (VEC_BINARY_CONDITIONS[op](numpy.array(a), numpy.array(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3BinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec3BinaryOperation.md new file mode 100644 index 0000000000..19cf68188d --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3BinaryOperation.md @@ -0,0 +1,54 @@ +# Vec3BinaryOperation +## Documentation +- Class name: `CM_Vec3BinaryOperation` +- Category: `math/vec3` +- Output node: `False` + +Performs a binary operation on two Vec3 vectors, resulting in a new Vec3 vector. This operation is defined by a specific operation key which dictates the mathematical operation to be applied. +## Input types +### Required +- **`op`** + - Specifies the binary operation to be applied on the Vec3 vectors. The choice of operation affects the result of the computation. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first Vec3 vector to be operated on. It plays a crucial role in the operation as the primary vector. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +- **`b`** + - The second Vec3 vector to be operated on. It interacts with the first vector according to the specified operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Output types +- **`vec3`** + - The resulting Vec3 vector after applying the specified binary operation on the input vectors. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric computations within AI pipelines, the CM_Vec3BinaryOperation node performs a specified binary operation on two Vec3 vectors, resulting in a new Vec3 vector, which is essential for operations like vector addition, subtraction, or cross product in 3D space applications. +## Source code +```python +class Vec3BinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_VEC3, + "b": DEFAULT_VEC3, + } + } + + RETURN_TYPES = ("VEC3",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3, b: Vec3) -> tuple[Vec3]: + return ( + _vec3_from_numpy(VEC_BINARY_OPERATIONS[op](numpy.array(a), numpy.array(b))), + ) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3ScalarOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec3ScalarOperation.md new file mode 100644 index 0000000000..f167548e7c --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3ScalarOperation.md @@ -0,0 +1,52 @@ +# Vec3ScalarOperation +## Documentation +- Class name: `CM_Vec3ScalarOperation` +- Category: `math/vec3` +- Output node: `False` + +Performs a scalar operation on a 3-dimensional vector (Vec3) using a specified operation and a scalar value. The operation modifies the vector in a way defined by the operation type, such as multiplication or division by the scalar. +## Input types +### Required +- **`op`** + - Specifies the type of scalar operation to perform on the vector, such as multiplication or division. The choice of operation affects how the vector's components are modified. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 3-dimensional vector (Vec3) to be modified by the scalar operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +- **`b`** + - The scalar value to be used in the operation. It modifies the vector's components according to the specified operation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`vec3`** + - The modified 3-dimensional vector (Vec3) after applying the specified scalar operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and graphical transformations, the CM_Vec3ScalarOperation node performs a specified scalar operation (such as multiplication or division) on a 3-dimensional vector (Vec3) using a scalar value, modifying the vector's components accordingly. This node is essential for adjusting vector magnitudes or directions in applications involving 3D modeling, physics simulations, or data normalization. +## Source code +```python +class Vec3ScalarOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_SCALAR_OPERATION.keys()),), + "a": DEFAULT_VEC3, + "b": ("FLOAT",), + } + } + + RETURN_TYPES = ("VEC3",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3, b: float) -> tuple[Vec3]: + return (_vec3_from_numpy(VEC_SCALAR_OPERATION[op](numpy.array(a), b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3ToScalarBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec3ToScalarBinaryOperation.md new file mode 100644 index 0000000000..b169f7ed66 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3ToScalarBinaryOperation.md @@ -0,0 +1,52 @@ +# Vec3ToScalarBinaryOperation +## Documentation +- Class name: `CM_Vec3ToScalarBinaryOperation` +- Category: `math/vec3` +- Output node: `False` + +This node performs a binary operation between two Vec3 vectors, resulting in a scalar value. The operation is defined by a set of predefined operations such as Dot product or Distance calculation. +## Input types +### Required +- **`op`** + - Specifies the binary operation to be performed on the two Vec3 vectors. It determines the nature of the computation, such as calculating the dot product or the distance between the vectors. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first Vec3 vector involved in the binary operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +- **`b`** + - The second Vec3 vector involved in the binary operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Output types +- **`float`** + - The scalar result of the binary operation between the two Vec3 vectors. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric computations within AI pipelines, the CM_Vec3ToScalarBinaryOperation node performs a binary operation, such as calculating the dot product or the distance, between two Vec3 vectors, resulting in a scalar value. This node is crucial for operations requiring the comparison or combination of vector data, such as spatial calculations in 3D environments or physics simulations. +## Source code +```python +class Vec3ToScalarBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_TO_SCALAR_BINARY_OPERATION.keys()),), + "a": DEFAULT_VEC3, + "b": DEFAULT_VEC3, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3, b: Vec3) -> tuple[float]: + return (VEC_TO_SCALAR_BINARY_OPERATION[op](numpy.array(a), numpy.array(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3ToScalarUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec3ToScalarUnaryOperation.md new file mode 100644 index 0000000000..6e3737efd0 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3ToScalarUnaryOperation.md @@ -0,0 +1,47 @@ +# Vec3ToScalarUnaryOperation +## Documentation +- Class name: `CM_Vec3ToScalarUnaryOperation` +- Category: `math/vec3` +- Output node: `False` + +Performs a unary operation that transforms a 3-dimensional vector (Vec3) into a scalar value based on the specified operation. This operation is useful for extracting specific scalar properties or measurements from a Vec3. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the Vec3. The choice of operation determines how the Vec3 is transformed into a scalar value, affecting the outcome significantly. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The Vec3 on which the unary operation is performed. This vector is the input from which the scalar value is derived. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Output types +- **`float`** + - The scalar result of the unary operation performed on the Vec3. This output provides a specific scalar measurement or property derived from the input Vec3. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric computations to convert a 3-dimensional vector (Vec3) into a meaningful scalar value, such as length, by applying a specified unary operation. This node is crucial for extracting specific measurements or properties from a Vec3 for further analysis or use in the pipeline. +## Source code +```python +class Vec3ToScalarUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_TO_SCALAR_UNARY_OPERATION.keys()),), + "a": DEFAULT_VEC3, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3) -> tuple[float]: + return (VEC_TO_SCALAR_UNARY_OPERATION[op](numpy.array(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3UnaryCondition.md b/docs/md/ComfyMath/Nodes/CM_Vec3UnaryCondition.md new file mode 100644 index 0000000000..94da54f9fc --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3UnaryCondition.md @@ -0,0 +1,47 @@ +# Vec3UnaryCondition +## Documentation +- Class name: `CM_Vec3UnaryCondition` +- Category: `math/vec3` +- Output node: `False` + +Performs a unary operation on a 3-dimensional vector based on a specified condition. This operation evaluates a condition on the vector and returns a boolean result. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the vector. The operation is selected from a predefined list of conditions. The choice of operation directly influences the evaluation process and the resulting boolean value, making it crucial for determining the condition's applicability to the vector. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 3-dimensional vector on which the unary operation is performed. It represents the input vector to be evaluated against the specified condition. The vector's dimensions and values play a significant role in the outcome of the condition check. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Output types +- **`bool`** + - The result of the unary operation as a boolean value, indicating whether the specified condition is met by the input vector. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used in scenarios requiring the evaluation of a 3-dimensional vector against a specific condition, such as determining if a vector falls within a certain range or meets a geometric constraint. The CM_Vec3UnaryCondition node takes a vector (`a`) and a condition operation (`op`) as input, and outputs a boolean value indicating whether the vector satisfies the specified condition. +## Source code +```python +class Vec3UnaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_UNARY_CONDITIONS.keys()),), + "a": DEFAULT_VEC3, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3) -> tuple[bool]: + return (VEC_UNARY_CONDITIONS[op](numpy.array(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec3UnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec3UnaryOperation.md new file mode 100644 index 0000000000..395dfa239a --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec3UnaryOperation.md @@ -0,0 +1,47 @@ +# Vec3UnaryOperation +## Documentation +- Class name: `CM_Vec3UnaryOperation` +- Category: `math/vec3` +- Output node: `False` + +Performs a unary operation on a 3-dimensional vector (Vec3) based on a specified operation. This operation transforms the input Vec3 according to the mathematical function associated with the 'op' parameter. +## Input types +### Required +- **`op`** + - Specifies the unary operation to perform on the Vec3. The operation is chosen from a predefined list of mathematical functions, allowing for a variety of transformations. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 3-dimensional vector (Vec3) to be transformed by the unary operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Output types +- **`vec3`** + - The transformed 3-dimensional vector (Vec3) resulting from the specified unary operation. + - Python dtype: `Vec3` + - Comfy dtype: `VEC3` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric transformations, the CM_Vec3UnaryOperation node performs a specified unary operation on a 3-dimensional vector (Vec3), transforming it based on the chosen mathematical function. This node takes a Vec3 and an operation type as input and outputs the transformed Vec3. +## Source code +```python +class Vec3UnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_UNARY_OPERATIONS.keys()),), + "a": DEFAULT_VEC3, + } + } + + RETURN_TYPES = ("VEC3",) + FUNCTION = "op" + CATEGORY = "math/vec3" + + def op(self, op: str, a: Vec3) -> tuple[Vec3]: + return (_vec3_from_numpy(VEC_UNARY_OPERATIONS[op](numpy.array(a))),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4BinaryCondition.md b/docs/md/ComfyMath/Nodes/CM_Vec4BinaryCondition.md new file mode 100644 index 0000000000..466edebf82 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4BinaryCondition.md @@ -0,0 +1,52 @@ +# Vec4BinaryCondition +## Documentation +- Class name: `CM_Vec4BinaryCondition` +- Category: `math/vec4` +- Output node: `False` + +This node performs a binary condition operation on two Vec4 vectors, evaluating a specified condition between them. The operation is determined by the 'op' parameter, which selects the condition from a predefined set of vector binary conditions. +## Input types +### Required +- **`op`** + - Specifies the binary condition operation to be performed on the vectors. It determines how the two Vec4 vectors will be compared. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first Vec4 vector to be compared. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +- **`b`** + - The second Vec4 vector to be compared. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Output types +- **`bool`** + - The result of the binary condition operation, indicating whether the specified condition is true or false between the two Vec4 vectors. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The CM_Vec4BinaryCondition node is utilized for comparing two Vec4 vectors based on a specified binary condition, such as equality or greater than, to output a boolean result indicating whether the condition holds true. This node is often used in mathematical and logical operations within AI pipelines to facilitate decision-making processes based on vector comparisons. +## Source code +```python +class Vec4BinaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_BINARY_CONDITIONS.keys()),), + "a": DEFAULT_VEC4, + "b": DEFAULT_VEC4, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4, b: Vec4) -> tuple[bool]: + return (VEC_BINARY_CONDITIONS[op](numpy.array(a), numpy.array(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4BinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec4BinaryOperation.md new file mode 100644 index 0000000000..d79266cfff --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4BinaryOperation.md @@ -0,0 +1,54 @@ +# Vec4BinaryOperation +## Documentation +- Class name: `CM_Vec4BinaryOperation` +- Category: `math/vec4` +- Output node: `False` + +Performs a binary operation on two Vec4 vectors, resulting in a new Vec4 vector. The operation is determined by the 'op' parameter, which selects the specific binary operation to apply. +## Input types +### Required +- **`op`** + - Specifies the binary operation to apply on the Vec4 vectors. It affects the nature of the computation and the resulting vector. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first Vec4 vector to be operated on. It serves as one of the inputs to the binary operation. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +- **`b`** + - The second Vec4 vector to be operated on. It serves as the other input to the binary operation. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Output types +- **`vec4`** + - The resulting Vec4 vector after applying the specified binary operation on the input vectors. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and graphical computations requiring the manipulation of vector data, the CM_Vec4BinaryOperation node performs a specified binary operation on two Vec4 vectors, resulting in a new Vec4 vector as output. This node is essential for operations such as addition, subtraction, or other mathematical transformations on 4-dimensional vectors, making it invaluable in scenarios involving 3D graphics or spatial data analysis. +## Source code +```python +class Vec4BinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_BINARY_OPERATIONS.keys()),), + "a": DEFAULT_VEC4, + "b": DEFAULT_VEC4, + } + } + + RETURN_TYPES = ("VEC4",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4, b: Vec4) -> tuple[Vec4]: + return ( + _vec4_from_numpy(VEC_BINARY_OPERATIONS[op](numpy.array(a), numpy.array(b))), + ) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4ScalarOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec4ScalarOperation.md new file mode 100644 index 0000000000..a8c9cca07b --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4ScalarOperation.md @@ -0,0 +1,52 @@ +# Vec4ScalarOperation +## Documentation +- Class name: `CM_Vec4ScalarOperation` +- Category: `math/vec4` +- Output node: `False` + +Performs a scalar operation on a Vec4 vector, applying a specified operation (e.g., multiplication or division) between the vector and a scalar value, resulting in a transformed Vec4 vector. +## Input types +### Required +- **`op`** + - Specifies the scalar operation to perform on the Vec4 vector. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The Vec4 vector to be transformed. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +- **`b`** + - The scalar value to be used in the operation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`vec4`** + - The resulting Vec4 vector after the operation. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The CM_Vec4ScalarOperation node is primarily utilized for performing mathematical operations on a Vec4 vector by applying a scalar operation (e.g., multiplication or division) between the vector and a scalar value, resulting in a modified Vec4 vector. This node is often used in conjunction with nodes that require vector transformations or adjustments in their processing pipeline, enabling precise control over vector attributes without the need for complex coding. +## Source code +```python +class Vec4ScalarOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_SCALAR_OPERATION.keys()),), + "a": DEFAULT_VEC4, + "b": ("FLOAT",), + } + } + + RETURN_TYPES = ("VEC4",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4, b: float) -> tuple[Vec4]: + return (_vec4_from_numpy(VEC_SCALAR_OPERATION[op](numpy.array(a), b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4ToScalarBinaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec4ToScalarBinaryOperation.md new file mode 100644 index 0000000000..9b83487a27 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4ToScalarBinaryOperation.md @@ -0,0 +1,52 @@ +# Vec4ToScalarBinaryOperation +## Documentation +- Class name: `CM_Vec4ToScalarBinaryOperation` +- Category: `math/vec4` +- Output node: `False` + +Performs a binary operation between two Vec4 vectors and returns a scalar value. The operation is defined by a set of predefined operations such as dot product or distance calculation. +## Input types +### Required +- **`op`** + - Specifies the binary operation to perform on the two Vec4 vectors. It determines how the vectors are processed and the nature of the scalar result. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The first Vec4 vector to be operated on. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +- **`b`** + - The second Vec4 vector to be operated on. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Output types +- **`float`** + - The scalar result of the binary operation performed on the two Vec4 vectors. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric operations requiring the comparison or combination of two Vec4 vectors, such as calculating the dot product or the distance between them, to produce a single scalar value as output. +## Source code +```python +class Vec4ToScalarBinaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_TO_SCALAR_BINARY_OPERATION.keys()),), + "a": DEFAULT_VEC4, + "b": DEFAULT_VEC4, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4, b: Vec4) -> tuple[float]: + return (VEC_TO_SCALAR_BINARY_OPERATION[op](numpy.array(a), numpy.array(b)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4ToScalarUnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec4ToScalarUnaryOperation.md new file mode 100644 index 0000000000..2520271135 --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4ToScalarUnaryOperation.md @@ -0,0 +1,47 @@ +# Vec4ToScalarUnaryOperation +## Documentation +- Class name: `CM_Vec4ToScalarUnaryOperation` +- Category: `math/vec4` +- Output node: `False` + +Performs a unary operation on a 4-dimensional vector (Vec4) to produce a scalar value. The operation is defined by a set of predefined operations and applied to the vector. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be applied to the vector. It determines the type of transformation that the vector undergoes to produce a scalar output. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 4-dimensional vector (Vec4) to which the unary operation is applied. This vector's values are crucial as they determine the operation's effect and the resulting scalar value, emphasizing the vector's role in shaping the outcome of the operation. + - Python dtype: `tuple[float, float, float, float]` + - Comfy dtype: `VEC4` +## Output types +- **`float`** + - The scalar result of applying the specified unary operation to the input Vec4 vector. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in mathematical and geometric computations where a 4-dimensional vector needs to be simplified to a scalar value, such as calculating the magnitude or applying a custom operation that reduces Vec4 to a single, meaningful figure. +## Source code +```python +class Vec4ToScalarUnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_TO_SCALAR_UNARY_OPERATION.keys()),), + "a": DEFAULT_VEC4, + } + } + + RETURN_TYPES = ("FLOAT",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4) -> tuple[float]: + return (VEC_TO_SCALAR_UNARY_OPERATION[op](numpy.array(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4UnaryCondition.md b/docs/md/ComfyMath/Nodes/CM_Vec4UnaryCondition.md new file mode 100644 index 0000000000..765e154add --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4UnaryCondition.md @@ -0,0 +1,47 @@ +# Vec4UnaryCondition +## Documentation +- Class name: `CM_Vec4UnaryCondition` +- Category: `math/vec4` +- Output node: `False` + +Performs a unary condition operation on a 4-dimensional vector based on a specified condition. This operation evaluates the vector against the condition and returns a boolean result indicating whether the condition is met. +## Input types +### Required +- **`op`** + - Specifies the unary condition to be applied to the vector. The choice of operation determines how the vector is evaluated against the condition, directly affecting the outcome by defining the criteria for the condition check. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`a`** + - The 4-dimensional vector to be evaluated against the specified condition. Its values are crucial as they are directly compared against the condition defined by 'op', influencing whether the condition is met. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Output types +- **`bool`** + - A boolean value indicating whether the specified condition is met by the vector. + - Python dtype: `bool` + - Comfy dtype: `BOOL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in scenarios requiring logical operations or condition checks on 4-dimensional vectors, such as filtering or decision-making processes based on vector attributes. The CM_Vec4UnaryCondition node takes a 4-dimensional vector and a specified unary condition as input, and outputs a boolean value indicating whether the vector meets the condition. +## Source code +```python +class Vec4UnaryCondition: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_UNARY_CONDITIONS.keys()),), + "a": DEFAULT_VEC4, + } + } + + RETURN_TYPES = ("BOOL",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4) -> tuple[bool]: + return (VEC_UNARY_CONDITIONS[op](numpy.array(a)),) + +``` diff --git a/docs/md/ComfyMath/Nodes/CM_Vec4UnaryOperation.md b/docs/md/ComfyMath/Nodes/CM_Vec4UnaryOperation.md new file mode 100644 index 0000000000..46bed1ac3a --- /dev/null +++ b/docs/md/ComfyMath/Nodes/CM_Vec4UnaryOperation.md @@ -0,0 +1,47 @@ +# Vec4UnaryOperation +## Documentation +- Class name: `CM_Vec4UnaryOperation` +- Category: `math/vec4` +- Output node: `False` + +Performs a unary operation on a 4-dimensional vector (Vec4) based on a specified operation. This operation transforms the input Vec4 according to the mathematical function associated with the 'op' parameter. +## Input types +### Required +- **`op`** + - Specifies the unary operation to be performed on the Vec4. The choice of operation affects how the input vector is transformed, directly influencing the node's execution and results by applying a specific mathematical function to the vector. + - Python dtype: `Enum['Neg', 'Normalize',...]` + - Comfy dtype: `['Neg', 'Normalize',...]` +- **`a`** + - The 4-dimensional vector (Vec4) to be transformed by the unary operation. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Output types +- **`vec4`** + - The transformed 4-dimensional vector (Vec4) after applying the specified unary operation. + - Python dtype: `Vec4` + - Comfy dtype: `VEC4` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The CM_Vec4UnaryOperation node is primarily utilized for applying a specified unary operation to a 4-dimensional vector (Vec4), transforming it according to the chosen mathematical function. This node is often used in mathematical and geometric transformations within AI pipelines, taking a Vec4 as input and outputting the transformed Vec4. +## Source code +```python +class Vec4UnaryOperation: + @classmethod + def INPUT_TYPES(cls) -> Mapping[str, Any]: + return { + "required": { + "op": (list(VEC_UNARY_OPERATIONS.keys()),), + "a": DEFAULT_VEC4, + } + } + + RETURN_TYPES = ("VEC4",) + FUNCTION = "op" + CATEGORY = "math/vec4" + + def op(self, op: str, a: Vec4) -> tuple[Vec4]: + return (_vec4_from_numpy(VEC_UNARY_OPERATIONS[op](numpy.array(a))),) + +``` diff --git a/docs/md/ComfyMath/index.md b/docs/md/ComfyMath/index.md new file mode 100644 index 0000000000..8d145a0d18 --- /dev/null +++ b/docs/md/ComfyMath/index.md @@ -0,0 +1,3 @@ +# ComfyMath +- repo url: `https://github.com/evanspearman/ComfyMath` +- commit hash: `be9beab9923ccf5c5e4132dc1653bcdfa773ed70` \ No newline at end of file diff --git a/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLParams.md b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLParams.md new file mode 100644 index 0000000000..8fa8cade41 --- /dev/null +++ b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLParams.md @@ -0,0 +1,81 @@ +# Add CLIP SDXL Params +## Documentation +- Class name: `BNK_AddCLIPSDXLParams` +- Category: `conditioning/advanced` +- Output node: `False` + +The `BNK_AddCLIPSDXLParams` node is designed to augment the conditioning data with additional parameters related to image dimensions and cropping. This process is essential for tailoring the conditioning to specific requirements of image generation tasks, ensuring that the generated images match the desired dimensions and cropping specifications. +## Input types +### Required +- **`conditioning`** + - Represents the initial conditioning data that will be augmented with additional parameters. It is crucial for the generation process, serving as the base upon which further specifications are added. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`width`** + - Specifies the desired width of the generated image. This parameter directly influences the dimensions of the output, ensuring the generated image meets the specified width. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Determines the height of the generated image, directly affecting its dimensions to match the specified height. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`crop_w`** + - Defines the width of the cropping area, allowing for precise control over the portion of the image to be included or excluded. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`crop_h`** + - Specifies the height of the cropping area, enabling fine-tuning of the image's visible section according to the desired cropping. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`target_width`** + - Sets the target width for the image after processing, ensuring the final image aligns with the specified dimensions. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`target_height`** + - Indicates the target height for the image post-processing, guaranteeing the output matches the desired height. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`conditioning`** + - The augmented conditioning data, now including the specified image dimensions and cropping parameters, ready for use in further image generation processes. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class AddCLIPSDXLParams: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "conditioning": ("CONDITIONING", ), + "width": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "height": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "crop_w": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION}), + "crop_h": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION}), + "target_width": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "target_height": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + }} + + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "conditioning/advanced" + + def encode(self, conditioning, width, height, crop_w, crop_h, target_width, target_height): + c = [] + for t in conditioning: + n = [t[0], t[1].copy()] + n[1]['width'] = width + n[1]['height'] = height + n[1]['crop_w'] = crop_w + n[1]['crop_h'] = crop_h + n[1]['target_width'] = target_width + n[1]['target_height'] = target_height + c.append(n) + return (c,) + +``` diff --git a/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLRParams.md b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLRParams.md new file mode 100644 index 0000000000..01329c2160 --- /dev/null +++ b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLRParams.md @@ -0,0 +1,63 @@ +# Add CLIP SDXL Refiner Params +## Documentation +- Class name: `BNK_AddCLIPSDXLRParams` +- Category: `conditioning/advanced` +- Output node: `False` + +The `BNK_AddCLIPSDXLRParams` node enriches the conditioning data with additional parameters such as width, height, and an aesthetic score. This process is essential for tailoring the conditioning to specific requirements, thereby enhancing the generation process. +## Input types +### Required +- **`conditioning`** + - The base conditioning data to be enriched with additional parameters. It is crucial for defining the context or requirements for the generation process. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +- **`width`** + - Specifies the desired width for the generation. It directly influences the dimensions of the generated output. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`height`** + - Specifies the desired height for the generation. It directly influences the dimensions of the generated output. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`ascore`** + - An aesthetic score that influences the visual appeal of the generated output. It allows for a subjective quality assessment to be factored into the generation. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The enriched conditioning data, now including width, height, and aesthetic score, ready for further processing. + - Python dtype: `List[Tuple[Any, Dict[str, Any]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + + +## Source code +```python +class AddCLIPSDXLRParams: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "conditioning": ("CONDITIONING", ), + "width": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "height": ("INT", {"default": 1024.0, "min": 0, "max": MAX_RESOLUTION}), + "ascore": ("FLOAT", {"default": 6.0, "min": 0.0, "max": 1000.0, "step": 0.01}), + }} + + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "conditioning/advanced" + + def encode(self, conditioning, width, height, ascore): + c = [] + for t in conditioning: + n = [t[0], t[1].copy()] + n[1]['width'] = width + n[1]['height'] = height + n[1]['aesthetic_score'] = ascore + c.append(n) + return (c,) + +``` diff --git a/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeAdvanced.md b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeAdvanced.md new file mode 100644 index 0000000000..cd1ef09c19 --- /dev/null +++ b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeAdvanced.md @@ -0,0 +1,58 @@ +# CLIP Text Encode (Advanced) +## Documentation +- Class name: `BNK_CLIPTextEncodeAdvanced` +- Category: `conditioning/advanced` +- Output node: `False` + +This node performs advanced text encoding using the CLIP model. It processes input text to generate embeddings and pooled output, considering token normalization and weight interpretation strategies. +## Input types +### Required +- **`text`** + - The text input to be encoded. Supports multiline input, allowing for more complex and detailed text data to be processed. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`clip`** + - The CLIP model used for encoding the text. It plays a crucial role in generating the embeddings and pooled output. + - Python dtype: `CLIP` + - Comfy dtype: `CLIP` +- **`token_normalization`** + - Defines the method for normalizing the tokens extracted from the text. It affects the final embeddings by adjusting their scale or distribution. + - Python dtype: `List[str]` + - Comfy dtype: `['none', 'mean', 'length', 'length+mean']` +- **`weight_interpretation`** + - Specifies how the weights are interpreted during the encoding process, influencing the emphasis on certain aspects of the text. + - Python dtype: `List[str]` + - Comfy dtype: `['comfy', 'A1111', 'compel', 'comfy++', 'down_weight']` +## Output types +- **`conditioning`** + - The output includes embeddings and pooled output, structured for further processing or conditioning tasks. + - Python dtype: `Tuple[List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,SamplerCustom,ControlNetApplyAdvanced` + + +## Source code +```python +class AdvancedCLIPTextEncode: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "text": ("STRING", {"multiline": True}), + "clip": ("CLIP", ), + "token_normalization": (["none", "mean", "length", "length+mean"],), + "weight_interpretation": (["comfy", "A1111", "compel", "comfy++" ,"down_weight"],), + #"affect_pooled": (["disable", "enable"],), + }} + + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "conditioning/advanced" + + def encode(self, clip, text, token_normalization, weight_interpretation, affect_pooled='disable'): + embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0, apply_to_pooled=affect_pooled=='enable') + return ([[embeddings_final, {"pooled_output": pooled}]], ) + +``` diff --git a/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeSDXLAdvanced.md b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeSDXLAdvanced.md new file mode 100644 index 0000000000..263c82e923 --- /dev/null +++ b/docs/md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeSDXLAdvanced.md @@ -0,0 +1,68 @@ +# CLIP Text Encode SDXL (Advanced) +## Documentation +- Class name: `BNK_CLIPTextEncodeSDXLAdvanced` +- Category: `conditioning/advanced` +- Output node: `False` + +This node performs advanced text encoding using the CLIP model, specifically designed for SDXL models. It encodes two types of text inputs (local and global) with customizable token normalization and weight interpretation strategies, and allows for balancing the influence of these texts. The output is a conditioning format suitable for further processing or generation tasks. +## Input types +### Required +- **`text_l`** + - The local text input, which is expected to be more specific or detailed. It plays a crucial role in guiding the encoding process by providing fine-grained context. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`text_g`** + - The global text input, providing a broader context or theme for the encoding. It complements the local text by offering a wider perspective that influences the overall encoding. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`clip`** + - The CLIP model used for encoding the text inputs. It's a crucial component that transforms text into a format that can be utilized for further processing or generation. + - Python dtype: `CLIP` + - Comfy dtype: `CLIP` +- **`token_normalization`** + - Defines the strategy for normalizing the tokens, affecting how the text is processed and encoded. It's essential for tailoring the encoding to specific needs or preferences. + - Python dtype: `List[str]` + - Comfy dtype: `['none', 'mean', 'length', 'length+mean']` +- **`weight_interpretation`** + - Specifies how the weights are interpreted during encoding, influencing the emphasis on certain aspects of the text. It allows for fine-tuning the encoding process. + - Python dtype: `List[str]` + - Comfy dtype: `['comfy', 'A1111', 'compel', 'comfy++', 'down_weight']` +- **`balance`** + - Adjusts the balance between the local and global text inputs, determining their relative influence on the final encoding. It's key for achieving the desired emphasis between detailed and broad contexts. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`conditioning`** + - The output is a conditioning format that includes the encoded texts and additional metadata, ready for further processing or generation tasks. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + + +## Source code +```python +class AdvancedCLIPTextEncodeSDXL: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "text_l": ("STRING", {"multiline": True}), + "text_g": ("STRING", {"multiline": True}), + "clip": ("CLIP", ), + "token_normalization": (["none", "mean", "length", "length+mean"],), + "weight_interpretation": (["comfy", "A1111", "compel", "comfy++", "down_weight"],), + #"affect_pooled": (["disable", "enable"],), + "balance": ("FLOAT", {"default": .5, "min": 0.0, "max": 1.0, "step": 0.01}), + }} + + RETURN_TYPES = ("CONDITIONING",) + FUNCTION = "encode" + + CATEGORY = "conditioning/advanced" + + def encode(self, clip, text_l, text_g, token_normalization, weight_interpretation, balance, affect_pooled='disable'): + embeddings_final, pooled = advanced_encode_XL(clip, text_l, text_g, token_normalization, weight_interpretation, w_max=1.0, clip_balance=balance, apply_to_pooled=affect_pooled == "enable") + return ([[embeddings_final, {"pooled_output": pooled}]], ) + +``` diff --git a/docs/md/ComfyUI_ADV_CLIP_emb/index.md b/docs/md/ComfyUI_ADV_CLIP_emb/index.md new file mode 100644 index 0000000000..6543ad6819 --- /dev/null +++ b/docs/md/ComfyUI_ADV_CLIP_emb/index.md @@ -0,0 +1,3 @@ +# ComfyUI_ADV_CLIP_emb +- repo url: `https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb.git` +- commit hash: `c8b14087afab8efae22fce098b742f01cce22947` \ No newline at end of file diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApply.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApply.md new file mode 100644 index 0000000000..fddfd9b114 --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApply.md @@ -0,0 +1,242 @@ +# Apply IPAdapter +## Documentation +- Class name: `IPAdapterApply` +- Category: `ipadapter` +- Output node: `False` + +This node applies an IPAdapter to modify the embeddings or features of a model based on the provided inputs. It supports different weight types to adjust the influence of the IPAdapter, and can handle optional inputs like attention masks for more specialized adjustments. +## Input types +### Required +- **`ipadapter`** + - The IPAdapter to be applied. It is crucial for modifying the embeddings or features of the model. + - Python dtype: `IPAdapter` + - Comfy dtype: `IPADAPTER` +- **`clip_vision`** + - Optional. If provided, it is used to enhance the IPAdapter's application with vision-related features. + - Python dtype: `ClipVisionModel` + - Comfy dtype: `CLIP_VISION` +- **`image`** + - Optional. An image that can be used in conjunction with the IPAdapter for vision-related tasks. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`model`** + - The model whose embeddings or features are to be modified by the IPAdapter. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`weight`** + - A scalar weight that adjusts the influence of the IPAdapter on the model's embeddings or features. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`noise`** + - Optional. Noise level to be applied during the IPAdapter's application, providing a way to introduce variability. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`weight_type`** + - Specifies the method to apply the weight, offering different strategies for adjusting the IPAdapter's influence. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`start_at`** + - Defines the starting point of the IPAdapter's application, allowing for gradual application over the input. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`end_at`** + - Defines the ending point of the IPAdapter's application, allowing for precise control over its influence. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`unfold_batch`** + - Determines whether the batch should be unfolded during the IPAdapter's application, affecting how inputs are processed. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +### Optional +- **`attn_mask`** + - Optional. An attention mask that can be used to focus the IPAdapter's application on specific parts of the input. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`model`** + - The modified model after applying the IPAdapter, reflecting the adjustments made to its embeddings or features. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `Reroute,ADE_AnimateDiffLoaderWithContext,KSampler,ModelMergeSimple,IPAdapterApplyFaceID,KSampler (Efficient),IPAdapterApply,SelfAttentionGuidance,KSamplerAdvanced,ReroutePrimitive|pysssss` + +The IPAdapterApply node applies an IPAdapter to modify the embeddings or features of a model based on the provided inputs, including an IPAdapter model, CLIP vision features, an image, and optionally an attention mask. It outputs a modified model that incorporates the adjustments made by the IPAdapter, enabling enhanced or tailored image processing and generation tasks. +## Source code +```python +class IPAdapterApply: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "ipadapter": ("IPADAPTER", ), + "clip_vision": ("CLIP_VISION",), + "image": ("IMAGE",), + "model": ("MODEL", ), + "weight": ("FLOAT", { "default": 1.0, "min": -1, "max": 3, "step": 0.05 }), + "noise": ("FLOAT", { "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01 }), + "weight_type": (["original", "linear", "channel penalty"], ), + "start_at": ("FLOAT", { "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001 }), + "end_at": ("FLOAT", { "default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001 }), + "unfold_batch": ("BOOLEAN", { "default": False }), + }, + "optional": { + "attn_mask": ("MASK",), + } + } + + RETURN_TYPES = ("MODEL", ) + FUNCTION = "apply_ipadapter" + CATEGORY = "ipadapter" + + def apply_ipadapter(self, ipadapter, model, weight, clip_vision=None, image=None, weight_type="original", noise=None, embeds=None, attn_mask=None, start_at=0.0, end_at=1.0, unfold_batch=False, insightface=None, faceid_v2=False, weight_v2=False): + self.dtype = torch.float16 if comfy.model_management.should_use_fp16() else torch.float32 + self.device = comfy.model_management.get_torch_device() + self.weight = weight + self.is_full = "proj.3.weight" in ipadapter["image_proj"] + self.is_portrait = "proj.2.weight" in ipadapter["image_proj"] and not "proj.3.weight" in ipadapter["image_proj"] and not "0.to_q_lora.down.weight" in ipadapter["ip_adapter"] + self.is_faceid = self.is_portrait or "0.to_q_lora.down.weight" in ipadapter["ip_adapter"] + self.is_plus = (self.is_full or "latents" in ipadapter["image_proj"] or "perceiver_resampler.proj_in.weight" in ipadapter["image_proj"]) + + if self.is_faceid and not insightface: + raise Exception('InsightFace must be provided for FaceID models.') + + output_cross_attention_dim = ipadapter["ip_adapter"]["1.to_k_ip.weight"].shape[1] + self.is_sdxl = output_cross_attention_dim == 2048 + cross_attention_dim = 1280 if self.is_plus and self.is_sdxl and not self.is_faceid else output_cross_attention_dim + clip_extra_context_tokens = 16 if self.is_plus or self.is_portrait else 4 + + if embeds is not None: + embeds = torch.unbind(embeds) + clip_embed = embeds[0].cpu() + clip_embed_zeroed = embeds[1].cpu() + else: + if self.is_faceid: + insightface.det_model.input_size = (640,640) # reset the detection size + face_img = tensorToNP(image) + face_embed = [] + face_clipvision = [] + + for i in range(face_img.shape[0]): + for size in [(size, size) for size in range(640, 128, -64)]: + insightface.det_model.input_size = size # TODO: hacky but seems to be working + face = insightface.get(face_img[i]) + if face: + face_embed.append(torch.from_numpy(face[0].normed_embedding).unsqueeze(0)) + face_clipvision.append(NPToTensor(insightface_face_align.norm_crop(face_img[i], landmark=face[0].kps, image_size=224))) + + if 640 not in size: + print(f"\033[33mINFO: InsightFace detection resolution lowered to {size}.\033[0m") + break + else: + raise Exception('InsightFace: No face detected.') + + face_embed = torch.stack(face_embed, dim=0) + image = torch.stack(face_clipvision, dim=0) + + neg_image = image_add_noise(image, noise) if noise > 0 else None + + if self.is_plus: + clip_embed = clip_vision.encode_image(image).penultimate_hidden_states + if noise > 0: + clip_embed_zeroed = clip_vision.encode_image(neg_image).penultimate_hidden_states + else: + clip_embed_zeroed = zeroed_hidden_states(clip_vision, image.shape[0]) + + # TODO: check noise to the uncods too + face_embed_zeroed = torch.zeros_like(face_embed) + else: + clip_embed = face_embed + clip_embed_zeroed = torch.zeros_like(clip_embed) + else: + if image.shape[1] != image.shape[2]: + print("\033[33mINFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting.\033[0m") + + clip_embed = clip_vision.encode_image(image) + neg_image = image_add_noise(image, noise) if noise > 0 else None + + if self.is_plus: + clip_embed = clip_embed.penultimate_hidden_states + if noise > 0: + clip_embed_zeroed = clip_vision.encode_image(neg_image).penultimate_hidden_states + else: + clip_embed_zeroed = zeroed_hidden_states(clip_vision, image.shape[0]) + else: + clip_embed = clip_embed.image_embeds + if noise > 0: + clip_embed_zeroed = clip_vision.encode_image(neg_image).image_embeds + else: + clip_embed_zeroed = torch.zeros_like(clip_embed) + + clip_embeddings_dim = clip_embed.shape[-1] + + self.ipadapter = IPAdapter( + ipadapter, + cross_attention_dim=cross_attention_dim, + output_cross_attention_dim=output_cross_attention_dim, + clip_embeddings_dim=clip_embeddings_dim, + clip_extra_context_tokens=clip_extra_context_tokens, + is_sdxl=self.is_sdxl, + is_plus=self.is_plus, + is_full=self.is_full, + is_faceid=self.is_faceid, + ) + + self.ipadapter.to(self.device, dtype=self.dtype) + + if self.is_faceid and self.is_plus: + image_prompt_embeds = self.ipadapter.get_image_embeds_faceid_plus(face_embed.to(self.device, dtype=self.dtype), clip_embed.to(self.device, dtype=self.dtype), weight_v2, faceid_v2) + uncond_image_prompt_embeds = self.ipadapter.get_image_embeds_faceid_plus(face_embed_zeroed.to(self.device, dtype=self.dtype), clip_embed_zeroed.to(self.device, dtype=self.dtype), weight_v2, faceid_v2) + else: + image_prompt_embeds, uncond_image_prompt_embeds = self.ipadapter.get_image_embeds(clip_embed.to(self.device, dtype=self.dtype), clip_embed_zeroed.to(self.device, dtype=self.dtype)) + + image_prompt_embeds = image_prompt_embeds.to(self.device, dtype=self.dtype) + uncond_image_prompt_embeds = uncond_image_prompt_embeds.to(self.device, dtype=self.dtype) + + work_model = model.clone() + + if attn_mask is not None: + attn_mask = attn_mask.to(self.device) + + sigma_start = model.model.model_sampling.percent_to_sigma(start_at) + sigma_end = model.model.model_sampling.percent_to_sigma(end_at) + + patch_kwargs = { + "number": 0, + "weight": self.weight, + "ipadapter": self.ipadapter, + "cond": image_prompt_embeds, + "uncond": uncond_image_prompt_embeds, + "weight_type": weight_type, + "mask": attn_mask, + "sigma_start": sigma_start, + "sigma_end": sigma_end, + "unfold_batch": unfold_batch, + } + + if not self.is_sdxl: + for id in [1,2,4,5,7,8]: # id of input_blocks that have cross attention + set_model_patch_replace(work_model, patch_kwargs, ("input", id)) + patch_kwargs["number"] += 1 + for id in [3,4,5,6,7,8,9,10,11]: # id of output_blocks that have cross attention + set_model_patch_replace(work_model, patch_kwargs, ("output", id)) + patch_kwargs["number"] += 1 + set_model_patch_replace(work_model, patch_kwargs, ("middle", 0)) + else: + for id in [4,5,7,8]: # id of input_blocks that have cross attention + block_indices = range(2) if id in [4, 5] else range(10) # transformer_depth + for index in block_indices: + set_model_patch_replace(work_model, patch_kwargs, ("input", id, index)) + patch_kwargs["number"] += 1 + for id in range(6): # id of output_blocks that have cross attention + block_indices = range(2) if id in [3, 4, 5] else range(10) # transformer_depth + for index in block_indices: + set_model_patch_replace(work_model, patch_kwargs, ("output", id, index)) + patch_kwargs["number"] += 1 + for index in range(10): + set_model_patch_replace(work_model, patch_kwargs, ("middle", 0, index)) + patch_kwargs["number"] += 1 + + return (work_model, ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyEncoded.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyEncoded.md new file mode 100644 index 0000000000..ee1d1e9c94 --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyEncoded.md @@ -0,0 +1,78 @@ +# Apply IPAdapter from Encoded +## Documentation +- Class name: `IPAdapterApplyEncoded` +- Category: `ipadapter` +- Output node: `False` + +This node applies an IPAdapter from encoded embeddings to a model, adjusting the model's behavior based on the embeddings and various parameters. It allows for the dynamic modification of a model's internal representations using external data, with options to control the intensity and manner of this modification. +## Input types +### Required +- **`ipadapter`** + - The IPAdapter to be applied. It serves as a bridge between the external embeddings and the model, enabling the modification of the model's internal representations. + - Python dtype: `IPAdapter` + - Comfy dtype: `IPADAPTER` +- **`embeds`** + - The embeddings that contain the encoded information to be applied to the model. These embeddings are crucial for the adaptation process, providing the specific adjustments to be made. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +- **`model`** + - The model to which the IPAdapter and embeddings are applied. This parameter specifies the target of the adaptation, allowing for its behavior to be modified. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`weight`** + - A scalar that adjusts the intensity of the adaptation applied to the model. It allows for fine-tuning the impact of the embeddings on the model's behavior. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`weight_type`** + - Specifies the method of applying the weight to the adaptation process. It influences how the embeddings modify the model, offering different strategies for integration. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`start_at`** + - Defines the starting point of the adaptation in the model's processing. It allows for control over when the modifications begin to take effect. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`end_at`** + - Determines the endpoint of the adaptation within the model's processing. This parameter controls the duration and extent of the modifications. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`unfold_batch`** + - A boolean flag that indicates whether to process each item in a batch separately. It affects the adaptation's application across multiple inputs. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +### Optional +- **`attn_mask`** + - An optional mask that can be applied to the adaptation process. It allows for selective modification of the model's behavior, focusing the adjustments on specific areas. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`model`** + - The model after the application of the IPAdapter and embeddings. It represents the modified version of the original model, with altered behavior based on the provided parameters. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler,CR Apply LoRA Stack,Reroute,ADE_AnimateDiffLoaderWithContext` + +The IPAdapterApplyEncoded node dynamically adjusts a model's behavior by applying encoded embeddings, allowing for the customization of image features or styles during AI-driven image transformation processes. It takes an IPAdapter model, embeddings, and optionally an attention mask as input, producing a model with modified output, ideal for creating highly detailed and stylized images. +## Source code +```python +class IPAdapterApplyEncoded(IPAdapterApply): + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "ipadapter": ("IPADAPTER", ), + "embeds": ("EMBEDS",), + "model": ("MODEL", ), + "weight": ("FLOAT", { "default": 1.0, "min": -1, "max": 3, "step": 0.05 }), + "weight_type": (["original", "linear", "channel penalty"], ), + "start_at": ("FLOAT", { "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001 }), + "end_at": ("FLOAT", { "default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001 }), + "unfold_batch": ("BOOLEAN", { "default": False }), + }, + "optional": { + "attn_mask": ("MASK",), + } + } + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyFaceID.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyFaceID.md new file mode 100644 index 0000000000..d301e19c8c --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyFaceID.md @@ -0,0 +1,103 @@ +# Apply IPAdapter FaceID +## Documentation +- Class name: `IPAdapterApplyFaceID` +- Category: `ipadapter` +- Output node: `False` + +IPAdapterApplyFaceID extends the functionality of IPAdapterApply to incorporate face identification features into the image processing pipeline. It leverages InsightFace for face detection and embedding, adjusting the image based on detected faces to enhance or modify facial features according to specified parameters. +## Input types +### Required +- **`ipadapter`** + - The IPAdapter model to be applied. It's crucial for adapting the input image in accordance with the model's capabilities and configurations. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `IPADAPTER` +- **`clip_vision`** + - A CLIP vision model used for generating embeddings from images. This model plays a key role in aligning the image content with textual descriptions or other images. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP_VISION` +- **`insightface`** + - InsightFace model for face detection and embedding. It's essential for identifying and processing faces within the image, especially for FaceID models. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `INSIGHTFACE` +- **`image`** + - The input image to be processed. This is the primary data on which the IPAdapter and InsightFace models operate. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`model`** + - The generative model to be used in conjunction with the IPAdapter. It defines the overall structure and capabilities of the image generation process. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`weight`** + - A floating-point value that influences the blending of conditions in the generation process. It affects how strongly the specified conditions are applied to the generated image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`noise`** + - A parameter to control the addition of noise to the image. This can be used to introduce variability or to simulate certain effects. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`weight_type`** + - Specifies the method for applying weights to the generation process. Options include 'original', 'linear', and 'channel penalty', each affecting the blending and emphasis of conditions differently. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`start_at`** + - Defines the starting point of the generation process in terms of model sampling. It can be used to control the level of abstraction or detail in the generated image. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`end_at`** + - Defines the ending point of the generation process in terms of model sampling. It affects the final appearance of the generated image by controlling the sampling resolution. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`faceid_v2`** + - A boolean flag indicating whether to use an updated version of the FaceID feature. This can affect the face detection and processing algorithms. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`weight_v2`** + - A floating-point value for the updated version of the weight parameter. It allows for finer control over the blending and application of conditions in the generation process. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`unfold_batch`** + - A boolean flag that controls the batching behavior during processing. Enabling this can improve performance by processing multiple items in parallel. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +### Optional +- **`attn_mask`** + - An optional mask that can be applied to the attention mechanism of the model. This allows for selective focus or exclusion of certain parts of the image during processing. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`model`** + - The modified generative model after applying the IPAdapter with FaceID features. It reflects the adjustments made to incorporate facial identification and processing. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `KSampler //Inspire,FreeU_V2,KSampler,Anything Everywhere,IPAdapterApply,ToIPAdapterPipe //Inspire,ToDetailerPipe,CR Apply LoRA Stack,IPAdapterApplyFaceID,FaceDetailer` + +The IPAdapterApplyFaceID node is designed for enhancing or modifying facial features in images by leveraging InsightFace for face detection and embedding. It inputs an IPAdapter model, CLIP vision features, InsightFace features, an image, and optionally an attention mask, and outputs a model adapted for precise facial identification and processing, ideal for applications requiring detailed face adjustments. +## Source code +```python +class IPAdapterApplyFaceID(IPAdapterApply): + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "ipadapter": ("IPADAPTER", ), + "clip_vision": ("CLIP_VISION",), + "insightface": ("INSIGHTFACE",), + "image": ("IMAGE",), + "model": ("MODEL", ), + "weight": ("FLOAT", { "default": 1.0, "min": -1, "max": 3, "step": 0.05 }), + "noise": ("FLOAT", { "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01 }), + "weight_type": (["original", "linear", "channel penalty"], ), + "start_at": ("FLOAT", { "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001 }), + "end_at": ("FLOAT", { "default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001 }), + "faceid_v2": ("BOOLEAN", { "default": False }), + "weight_v2": ("FLOAT", { "default": 1.0, "min": -1, "max": 3, "step": 0.05 }), + "unfold_batch": ("BOOLEAN", { "default": False }), + }, + "optional": { + "attn_mask": ("MASK",), + } + } + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterBatchEmbeds.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterBatchEmbeds.md new file mode 100644 index 0000000000..31945696a8 --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterBatchEmbeds.md @@ -0,0 +1,45 @@ +# IPAdapter Batch Embeds +## Documentation +- Class name: `IPAdapterBatchEmbeds` +- Category: `ipadapter` +- Output node: `False` + +This node is responsible for batching embeddings by concatenating two sets of embeddings along a specified dimension. It is useful for combining embeddings from different sources or models before further processing. +## Input types +### Required +- **`embed1`** + - The first set of embeddings to be concatenated. It plays a crucial role in the batching process as it forms the initial part of the combined embeddings. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +- **`embed2`** + - The second set of embeddings to be concatenated with the first set. It is equally important as it completes the combined embeddings, allowing for more comprehensive data representation. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +## Output types +- **`embeds`** + - The concatenated embeddings resulting from combining the two input sets. This output is ready for further processing or analysis. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used for combining embeddings from different AI models or sources, the IPAdapterBatchEmbeds node concatenates two sets of embeddings along a specified dimension, preparing them for further processing or analysis. +## Source code +```python +class IPAdapterBatchEmbeds: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "embed1": ("EMBEDS",), + "embed2": ("EMBEDS",), + }} + + RETURN_TYPES = ("EMBEDS",) + FUNCTION = "batch" + CATEGORY = "ipadapter" + + def batch(self, embed1, embed2): + return (torch.cat((embed1, embed2), dim=1), ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterEncoder.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterEncoder.md new file mode 100644 index 0000000000..826575459d --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterEncoder.md @@ -0,0 +1,140 @@ +# Encode IPAdapter Image +## Documentation +- Class name: `IPAdapterEncoder` +- Category: `ipadapter` +- Output node: `False` + +The `IPAdapterEncoder` node is designed for encoding images using the IPAdapter mechanism. It encapsulates the process of transforming images into a format suitable for further processing or integration with other models. +## Input types +### Required +- **`clip_vision`** + - This input represents the CLIP vision model used for image understanding and feature extraction. It's crucial for interpreting the content of the images to be encoded. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP_VISION` +- **`image_1`** + - The primary image to be processed. It serves as the main subject for the encoding operation, influencing the outcome significantly. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`ipadapter_plus`** + - A boolean flag indicating whether to apply an enhanced IPAdapter process. This option can alter the encoding strategy, affecting the complexity and detail of the output. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`noise`** + - A float value representing the amount of noise to introduce during the encoding process. This can affect the robustness and variability of the encoded embeddings. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`weight_1`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +### Optional +- **`image_2`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `IMAGE` +- **`image_3`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `IMAGE` +- **`image_4`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `IMAGE` +- **`weight_2`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`weight_3`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`weight_4`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +## Output types +- **`embeds`** + - The result of the encoding process, which combines the input embeddings into a new set of embeddings suitable for further processing. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IPAdapterApplyEncoded` + +The IPAdapterEncoder node is crucial for encoding images using the IPAdapter mechanism, taking in primary and optional additional images along with a CLIP vision input to produce enhanced embeddings suitable for further AI-driven image processing and manipulation. This node is particularly useful in pipelines like SMOOSH v2 for integrating advanced image processing techniques, thereby enhancing the AI's understanding and manipulation of images based on the provided inputs. +## Source code +```python +class IPAdapterEncoder: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "clip_vision": ("CLIP_VISION",), + "image_1": ("IMAGE",), + "ipadapter_plus": ("BOOLEAN", { "default": False }), + "noise": ("FLOAT", { "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01 }), + "weight_1": ("FLOAT", { "default": 1.0, "min": 0, "max": 1.0, "step": 0.01 }), + }, + "optional": { + "image_2": ("IMAGE",), + "image_3": ("IMAGE",), + "image_4": ("IMAGE",), + "weight_2": ("FLOAT", { "default": 1.0, "min": 0, "max": 1.0, "step": 0.01 }), + "weight_3": ("FLOAT", { "default": 1.0, "min": 0, "max": 1.0, "step": 0.01 }), + "weight_4": ("FLOAT", { "default": 1.0, "min": 0, "max": 1.0, "step": 0.01 }), + } + } + + RETURN_TYPES = ("EMBEDS",) + FUNCTION = "preprocess" + CATEGORY = "ipadapter" + + def preprocess(self, clip_vision, image_1, ipadapter_plus, noise, weight_1, image_2=None, image_3=None, image_4=None, weight_2=1.0, weight_3=1.0, weight_4=1.0): + weight_1 *= (0.1 + (weight_1 - 0.1)) + weight_2 *= (0.1 + (weight_2 - 0.1)) + weight_3 *= (0.1 + (weight_3 - 0.1)) + weight_4 *= (0.1 + (weight_4 - 0.1)) + + image = image_1 + weight = [weight_1]*image_1.shape[0] + + if image_2 is not None: + if image_1.shape[1:] != image_2.shape[1:]: + image_2 = comfy.utils.common_upscale(image_2.movedim(-1,1), image.shape[2], image.shape[1], "bilinear", "center").movedim(1,-1) + image = torch.cat((image, image_2), dim=0) + weight += [weight_2]*image_2.shape[0] + if image_3 is not None: + if image.shape[1:] != image_3.shape[1:]: + image_3 = comfy.utils.common_upscale(image_3.movedim(-1,1), image.shape[2], image.shape[1], "bilinear", "center").movedim(1,-1) + image = torch.cat((image, image_3), dim=0) + weight += [weight_3]*image_3.shape[0] + if image_4 is not None: + if image.shape[1:] != image_4.shape[1:]: + image_4 = comfy.utils.common_upscale(image_4.movedim(-1,1), image.shape[2], image.shape[1], "bilinear", "center").movedim(1,-1) + image = torch.cat((image, image_4), dim=0) + weight += [weight_4]*image_4.shape[0] + + clip_embed = clip_vision.encode_image(image) + neg_image = image_add_noise(image, noise) if noise > 0 else None + + if ipadapter_plus: + clip_embed = clip_embed.penultimate_hidden_states + if noise > 0: + clip_embed_zeroed = clip_vision.encode_image(neg_image).penultimate_hidden_states + else: + clip_embed_zeroed = zeroed_hidden_states(clip_vision, image.shape[0]) + else: + clip_embed = clip_embed.image_embeds + if noise > 0: + clip_embed_zeroed = clip_vision.encode_image(neg_image).image_embeds + else: + clip_embed_zeroed = torch.zeros_like(clip_embed) + + if any(e != 1.0 for e in weight): + weight = torch.tensor(weight).unsqueeze(-1) if not ipadapter_plus else torch.tensor(weight).unsqueeze(-1).unsqueeze(-1) + clip_embed = clip_embed * weight + + output = torch.stack((clip_embed, clip_embed_zeroed)) + + return( output, ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterLoadEmbeds.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterLoadEmbeds.md new file mode 100644 index 0000000000..e16b1b4338 --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterLoadEmbeds.md @@ -0,0 +1,43 @@ +# Load IPAdapter Embeds +## Documentation +- Class name: `IPAdapterLoadEmbeds` +- Category: `ipadapter` +- Output node: `False` + +The `IPAdapterLoadEmbeds` node is designed to load embedding vectors saved in a specific format (.ipadpt) from a designated directory. It facilitates the retrieval of pre-processed embeddings, which can be utilized in subsequent operations or models requiring such embeddings. +## Input types +### Required +- **`embeds`** + - Specifies the file name of the embedding to be loaded. The selection is limited to files with the '.ipadpt' extension within the input directory, ensuring that only compatible embeddings are processed. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`embeds`** + - Returns the loaded embedding tensor, making it available for further processing or analysis. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The `IPAdapterLoadEmbeds` node is essential for loading pre-processed embedding vectors (.ipadpt format) from a specified directory, making these embeddings available for further processing or analysis in subsequent nodes. It accepts a directory containing `.ipadpt` files as input and outputs the corresponding embedding tensors. +## Source code +```python +class IPAdapterLoadEmbeds: + @classmethod + def INPUT_TYPES(s): + input_dir = folder_paths.get_input_directory() + files = [os.path.relpath(os.path.join(root, file), input_dir) for root, dirs, files in os.walk(input_dir) for file in files if file.endswith('.ipadpt')] + return {"required": {"embeds": [sorted(files), ]}, } + + RETURN_TYPES = ("EMBEDS", ) + FUNCTION = "load" + CATEGORY = "ipadapter" + + def load(self, embeds): + path = folder_paths.get_annotated_filepath(embeds) + output = torch.load(path).cpu() + + return (output, ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterModelLoader.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterModelLoader.md new file mode 100644 index 0000000000..ce79c16e9d --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterModelLoader.md @@ -0,0 +1,54 @@ +# Load IPAdapter Model +## Documentation +- Class name: `IPAdapterModelLoader` +- Category: `ipadapter` +- Output node: `False` + +This node is responsible for loading an IPAdapter model from a specified file. It supports loading models saved in a custom format, including handling specific model components like 'image_proj' and 'ip_adapter'. If the model file is in a '.safetensors' format, it restructures the model accordingly. The node ensures the loaded model contains the 'ip_adapter' component, throwing an exception if it's missing. +## Input types +### Required +- **`ipadapter_file`** + - Specifies the filename of the IPAdapter model to be loaded. This parameter is crucial as it determines which model file is accessed and loaded. + - Python dtype: `List[str]` + - Comfy dtype: `MODEL_PATHS` +## Output types +- **`ipadapter`** + - The loaded IPAdapter model. It's either directly loaded or restructured if in '.safetensors' format, ready for further processing or application. + - Python dtype: `Dict[str, Any]` + - Comfy dtype: `IPADAPTER` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IPAdapterApply,IPAdapterApplyFaceID,IPAdapterApplyEncoded,ToIPAdapterPipe //Inspire,Reroute,AV_IPAdapter` + +The IPAdapterModelLoader node is responsible for loading IPAdapter models, which are specialized for adapting and enhancing images within the pipeline. It takes a model file name as input and outputs the loaded IPAdapter model (`IPADAPTER`), ready for further processing or application by subsequent nodes. +## Source code +```python +class IPAdapterModelLoader: + @classmethod + def INPUT_TYPES(s): + return {"required": { "ipadapter_file": (folder_paths.get_filename_list("ipadapter"), )}} + + RETURN_TYPES = ("IPADAPTER",) + FUNCTION = "load_ipadapter_model" + CATEGORY = "ipadapter" + + def load_ipadapter_model(self, ipadapter_file): + ckpt_path = folder_paths.get_full_path("ipadapter", ipadapter_file) + + model = comfy.utils.load_torch_file(ckpt_path, safe_load=True) + + if ckpt_path.lower().endswith(".safetensors"): + st_model = {"image_proj": {}, "ip_adapter": {}} + for key in model.keys(): + if key.startswith("image_proj."): + st_model["image_proj"][key.replace("image_proj.", "")] = model[key] + elif key.startswith("ip_adapter."): + st_model["ip_adapter"][key.replace("ip_adapter.", "")] = model[key] + model = st_model + + if not "ip_adapter" in model.keys() or not model["ip_adapter"]: + raise Exception("invalid IPAdapter model {}".format(ckpt_path)) + + return (model,) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterSaveEmbeds.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterSaveEmbeds.md new file mode 100644 index 0000000000..f326d3ae3d --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterSaveEmbeds.md @@ -0,0 +1,52 @@ +# Save IPAdapter Embeds +## Documentation +- Class name: `IPAdapterSaveEmbeds` +- Category: `ipadapter` +- Output node: `True` + +This node is responsible for saving IPAdapter embeddings to a file. It generates a unique filename based on a prefix and saves the embeddings in a custom format for later retrieval or use. +## Input types +### Required +- **`embeds`** + - The embeddings to be saved. These are typically generated by an IPAdapter model and represent encoded information from images or other inputs. + - Python dtype: `torch.Tensor` + - Comfy dtype: `EMBEDS` +- **`filename_prefix`** + - A prefix for the filename under which the embeddings will be saved. This allows for organized storage and easy retrieval of embeddings. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used with IPAdapters to save the generated embeddings for later retrieval or use, particularly useful for storing encoded information from images or other inputs in a custom format. +## Source code +```python +class IPAdapterSaveEmbeds: + def __init__(self): + self.output_dir = folder_paths.get_output_directory() + + @classmethod + def INPUT_TYPES(s): + return {"required": { + "embeds": ("EMBEDS",), + "filename_prefix": ("STRING", {"default": "embeds/IPAdapter"}) + }, + } + + RETURN_TYPES = () + FUNCTION = "save" + OUTPUT_NODE = True + CATEGORY = "ipadapter" + + def save(self, embeds, filename_prefix): + full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir) + file = f"{filename}_{counter:05}_.ipadpt" + file = os.path.join(full_output_folder, file) + + torch.save(embeds, file) + return (None, ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/InsightFaceLoader.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/InsightFaceLoader.md new file mode 100644 index 0000000000..892c2886c4 --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/InsightFaceLoader.md @@ -0,0 +1,54 @@ +# Load InsightFace +## Documentation +- Class name: `InsightFaceLoader` +- Category: `ipadapter` +- Output node: `False` + +The `InsightFaceLoader` node is responsible for loading the InsightFace model with a specified provider (e.g., CPU, CUDA). It initializes the model for face analysis tasks, setting up the necessary configurations and preparing it for use. +## Input types +### Required +- **`provider`** + - Specifies the backend provider for the InsightFace model, which can be CPU, CUDA, or ROCM. The choice of provider is crucial as it directly influences the model's execution speed and efficiency, potentially affecting the overall performance and responsiveness of the face analysis tasks. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`insightface`** + - Returns the initialized InsightFace model, ready for performing face analysis tasks. + - Python dtype: `tuple[FaceAnalysis]` + - Comfy dtype: `INSIGHTFACE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IPAdapterApplyFaceID` + +The InsightFaceLoader node is crucial for initializing the InsightFace model for face analysis tasks, particularly in pipelines requiring precise facial recognition and adjustments. It accepts a provider (CPU, CUDA, ROCM) as input and outputs the initialized InsightFace model (`INSIGHTFACE`), ready for performing detailed face analysis in subsequent nodes. +## Source code +```python +class InsightFaceLoader: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "provider": (["CPU", "CUDA", "ROCM"], ), + }, + } + + RETURN_TYPES = ("INSIGHTFACE",) + FUNCTION = "load_insight_face" + CATEGORY = "ipadapter" + + def load_insight_face(self, provider): + try: + from insightface.app import FaceAnalysis + except ImportError as e: + raise Exception(e) + + from insightface.utils import face_align + global insightface_face_align + insightface_face_align = face_align + + model = FaceAnalysis(name="buffalo_l", root=INSIGHTFACE_DIR, providers=[provider + 'ExecutionProvider',]) + model.prepare(ctx_id=0, det_size=(640, 640)) + + return (model,) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForClipVision.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForClipVision.md new file mode 100644 index 0000000000..7cee3854da --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForClipVision.md @@ -0,0 +1,59 @@ +# Prepare Image For Clip Vision +## Documentation +- Class name: `PrepImageForClipVision` +- Category: `ipadapter` +- Output node: `False` + +This node prepares an image for processing by CLIP vision models by resizing, cropping, and optionally sharpening the image. It standardizes the image format to meet the input requirements of CLIP models, ensuring compatibility and optimal performance. +## Input types +### Required +- **`image`** + - The input image to be processed. This image will be resized, cropped, and optionally sharpened according to the specified parameters. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`interpolation`** + - Specifies the interpolation method to be used when resizing the image. Different methods can affect the quality and characteristics of the resized image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`crop_position`** + - Determines the position of the crop applied to the image, which can influence the focus area of the processed image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`sharpening`** + - Applies a sharpening filter to the image, enhancing edge definition and detail. The strength of the sharpening effect can be adjusted. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The processed image, ready for input into CLIP vision models. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IDGenerationNode,IPAdapterApply,AIO_Preprocessor,ImageBatch,CLIPVisionEncode,PrepImageForInsightFace,SetNode,IPAdapterApplyFaceID,PreviewImage,IPAdapterEncoder` + +The PrepImageForClipVision node is essential for preparing images for compatibility with CLIP vision models by resizing, cropping, and optionally sharpening the image to meet the input requirements of CLIP models. It is often used with IPAdapters and CLIPVisionLoader to ensure that the processed images are optimized for analysis or further processing, enhancing the model's ability to accurately interpret and generate images based on textual descriptions or visual inputs. +## Source code +```python +class PrepImageForClipVision: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "image": ("IMAGE",), + "interpolation": (["LANCZOS", "BICUBIC", "HAMMING", "BILINEAR", "BOX", "NEAREST"],), + "crop_position": (["top", "bottom", "left", "right", "center", "pad"],), + "sharpening": ("FLOAT", {"default": 0.0, "min": 0, "max": 1, "step": 0.05}), + }, + } + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "prep_image" + + CATEGORY = "ipadapter" + + def prep_image(self, image, interpolation="LANCZOS", crop_position="center", sharpening=0.0): + size = (224, 224) + output = prepImage(image, interpolation, crop_position, size, sharpening, 0) + return (output, ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForInsightFace.md b/docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForInsightFace.md new file mode 100644 index 0000000000..8a389f9fdb --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForInsightFace.md @@ -0,0 +1,65 @@ +# Prepare Image For InsightFace +## Documentation +- Class name: `PrepImageForInsightFace` +- Category: `ipadapter` +- Output node: `False` + +This node prepares an image for processing with InsightFace by optionally padding it, resizing it to a specific size, and applying sharpening. The process involves determining whether to pad the image and to what dimensions it should be resized, followed by the application of a specified sharpening level. +## Input types +### Required +- **`image`** + - The input image to be processed. It is the primary subject for preparation before being processed by InsightFace. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`crop_position`** + - Specifies the position from which the image should be cropped. This affects the focus area of the image before further processing. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`sharpening`** + - The level of sharpening to apply to the image. This can enhance the image's details for better facial recognition. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`pad_around`** + - A boolean indicating whether padding should be added around the image. This can affect the image's aspect ratio and size. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +## Output types +- **`image`** + - The processed image, ready for InsightFace analysis. It has been optionally padded, resized, and sharpened according to the specified parameters. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: `IPAdapterApplyFaceID` + +The PrepImageForInsightFace node is essential for optimizing images for facial feature enhancement with the InsightFace model, by adjusting parameters like padding, resizing, and sharpening based on the input image's characteristics. It is particularly useful in pipelines focusing on style and character face adaptation, where precise facial feature analysis is crucial. +## Source code +```python +class PrepImageForInsightFace: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "image": ("IMAGE",), + "crop_position": (["center", "top", "bottom", "left", "right"],), + "sharpening": ("FLOAT", {"default": 0.0, "min": 0, "max": 1, "step": 0.05}), + "pad_around": ("BOOLEAN", { "default": True }), + }, + } + + RETURN_TYPES = ("IMAGE",) + FUNCTION = "prep_image" + + CATEGORY = "ipadapter" + + def prep_image(self, image, crop_position, sharpening=0.0, pad_around=True): + if pad_around: + padding = 30 + size = (580, 580) + else: + padding = 0 + size = (640, 640) + output = prepImage(image, "LANCZOS", crop_position, size, sharpening, padding) + + return (output, ) + +``` diff --git a/docs/md/ComfyUI_IPAdapter_plus/index.md b/docs/md/ComfyUI_IPAdapter_plus/index.md new file mode 100644 index 0000000000..8eb41a61fc --- /dev/null +++ b/docs/md/ComfyUI_IPAdapter_plus/index.md @@ -0,0 +1,3 @@ +# ComfyUI_IPAdapter_plus +- repo url: `https://github.com/cubiq/ComfyUI_IPAdapter_plus` +- commit hash: `46241f3ba5401f076f8d90c2aa85f2194910e1a9` \ No newline at end of file diff --git a/docs/md/SavePNG/Nodes/PNG_Save.md b/docs/md/SavePNG/Nodes/PNG_Save.md new file mode 100644 index 0000000000..c194e34303 --- /dev/null +++ b/docs/md/SavePNG/Nodes/PNG_Save.md @@ -0,0 +1,104 @@ +# PNG Save Node +## Documentation +- Class name: `PNG_Save` +- Category: `Ebsynth Nodes` +- Output node: `True` + +The PNG_Save node is designed to take an image or a batch of images and save them as PNG files to a specified location. It supports setting the image quality, deciding whether to overwrite existing files, and automatically handles the conversion of image tensors to the appropriate format for PNG encoding. +## Input types +### Required +- **`save_path`** + - Specifies the directory where the image(s) will be saved. It allows for organizing output images in a structured manner. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`file_name`** + - Determines the base name for the output file(s). This can be used to identify and differentiate between saved images. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`images`** + - The image tensor or a batch of image tensors to be saved. This node handles the conversion and encoding of these tensors into PNG format. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`quality`** + - Sets the quality of the saved PNG image(s). Higher values result in better image quality but larger file sizes, directly influencing the visual fidelity and the compression level of the output images. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`overwrite_mode`** + - Controls whether existing files with the same name should be overwritten. This is useful for avoiding accidental data loss. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used with image processing and transformation nodes to save the final, processed images as PNG files to a specified location, ensuring the output is readily accessible and in a widely supported format. +## Source code +```python +class PNG_Save: + """ + PNG Save Node + + This node takes in an image and saves it to the specified location using the PNG encoding provided by torchvision's encode_png. + """ + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "save_path": ("STRING", {"default": "./ComfyUI/output"}), + "file_name": ("STRING", {"default": "ComfyUI"}), + "images": ("IMAGE",), + "quality": ("INT", {"default": 96, "min": 0, "max": 100, "step": 1}), + "overwrite_mode": ("BOOLEAN", {"default": True}), + } + } + + RETURN_TYPES = () + FUNCTION = "save_image" + OUTPUT_NODE = True + CATEGORY = "Ebsynth Nodes" + + def save_image(self, save_path, file_name, images, quality, overwrite_mode=True): + # If the images tensor has 4 dimensions, it's a batch. Otherwise, make it a batch of 1. + if len(images.shape) == 3: + images = images.unsqueeze(0) + + for idx, image in enumerate(images): + # Construct full path for each image in the batch + base_name, ext = os.path.splitext(file_name) + # Ensure the extension is .png + ext = ".png" + current_file_name = f"{base_name}{ext}" + full_path = os.path.join(save_path, current_file_name) + + # Ensure the directory exists + directory = os.path.dirname(full_path) + os.makedirs(directory, exist_ok=True) + + # Ensure the image tensor has the correct shape + image_tensor = image.permute(2, 0, 1).float() # Change to (C, H, W) format + C, H, W = image_tensor.shape + + if C != 1 and C != 3: + raise ValueError(f"Image tensor has {C} channels. Expected 1 or 3.") + + # Convert and encode the image to PNG + image_tensor = image_tensor.mul(255).clamp(0, 255).type(torch.uint8) + encoded_image = encode_png(image_tensor.cpu()).numpy().tobytes() + + # Check if file exists and overwrite mode is False + if os.path.exists(full_path) and not overwrite_mode: + print(f"File {full_path} already exists and overwrite mode is disabled.") + continue + + # Save the encoded image + with open(full_path, 'wb') as f: + f.write(encoded_image) + + print(f"Image saved to: {full_path}") + + return {"status": "Image(s) saved successfully"} + +``` diff --git a/docs/md/SavePNG/index.md b/docs/md/SavePNG/index.md new file mode 100644 index 0000000000..51f9ebb979 --- /dev/null +++ b/docs/md/SavePNG/index.md @@ -0,0 +1,3 @@ +# SavePNG +- repo url: `` +- commit hash: `` \ No newline at end of file diff --git a/docs/md/WAS_Extras/Nodes/BLVAEEncode.md b/docs/md/WAS_Extras/Nodes/BLVAEEncode.md new file mode 100644 index 0000000000..dbb5eaf9c3 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/BLVAEEncode.md @@ -0,0 +1,155 @@ +# VAEEncode (Bundle Latent) +## Documentation +- Class name: `BLVAEEncode` +- Category: `latent` +- Output node: `False` + +The BLVAEEncode node is designed for encoding images into a latent representation using a Variational Autoencoder (VAE). It supports both standard and tiled encoding modes, allowing for flexibility in handling images of various sizes. Additionally, it provides functionality for storing or loading the encoded latent representations, which can be useful in workflows that require persistent state or caching of intermediate results. +## Input types +### Required +- **`vae`** + - Specifies the Variational Autoencoder (VAE) model to be used for encoding. This parameter is crucial as it determines the encoding mechanism and the quality of the generated latent representation. + - Python dtype: `VAEModel` + - Comfy dtype: `VAE` +- **`tiled`** + - A boolean flag indicating whether tiled encoding should be used. Tiled encoding can be beneficial for large images, as it processes the image in smaller, manageable tiles. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`tile_size`** + - Defines the size of the tiles used in tiled encoding. This parameter is relevant only when tiled encoding is enabled and affects the granularity of the encoding process. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`store_or_load_latent`** + - A boolean flag that controls whether the encoded latent representation should be stored or loaded. This functionality is useful for caching purposes or when working with precomputed latents. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`remove_latent_on_load`** + - Determines whether the latent representation should be removed from storage after loading. This can help manage storage space when caching latents. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`delete_workflow_latent`** + - A boolean flag that indicates whether the latent representation associated with a workflow should be deleted. This is useful for cleaning up resources and managing storage. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +### Optional +- **`image`** + - The input image to be encoded into a latent representation. This parameter is optional and is only required if an image is to be encoded. + - Python dtype: `ImageType` + - Comfy dtype: `IMAGE` +## Output types +- **`latent`** + - The encoded latent representation of the input image. This output is crucial for subsequent processing steps that require a latent representation, such as decoding or manipulation. + - Python dtype: `dict` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The BLVAEEncode node is primarily utilized for transforming images into a compact latent representation using a Variational Autoencoder (VAE), suitable for both standard and tiled encoding to accommodate images of varying sizes. It is often used in scenarios requiring the storage or retrieval of these latent representations for efficient processing or caching in AI workflows. +## Source code +```python +class BLVAEEncode: + def __init__(self): + self.VAEEncode = nodes.VAEEncode() + self.VAEEncodeTiled = nodes.VAEEncodeTiled() + self.last_hash = None + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "vae": ("VAE",), + "tiled": ("BOOLEAN", {"default": False}), + "tile_size": ("INT", {"default": 512, "min": 320, "max": 4096, "step": 64}), + "store_or_load_latent": ("BOOLEAN", {"default": True}), + "remove_latent_on_load": ("BOOLEAN", {"default": True}), + "delete_workflow_latent": ("BOOLEAN", {"default": False}) + }, + "optional": { + "image": ("IMAGE",), + }, + "hidden": { + "extra_pnginfo": "EXTRA_PNGINFO", + "unique_id": "UNIQUE_ID" + } + } + + RETURN_TYPES = ("LATENT", ) + RETURN_NAMES = ("latent", ) + + FUNCTION = "encode" + CATEGORY = "latent" + + def encode(self, vae, tiled, tile_size, store_or_load_latent, remove_latent_on_load, delete_workflow_latent, image=None, extra_pnginfo=None, unique_id=None): + workflow_latent = None + latent_key = f"latent_{unique_id}" + + if self.last_hash and torch.is_tensor(image): + if self.last_hash is not self.sha256(image): + delete_workflow_latent = True + if torch.is_tensor(image): + self.last_hash = self.sha256(image) + + if delete_workflow_latent: + if extra_pnginfo['workflow']['extra'].__contains__(latent_key): + try: + del extra_pnginfo['workflow']['extra'][latent_key] + except Exception: + print(f"Unable to delete latent image from workflow node: {unqiue_id}") + pass + + if store_or_load_latent and unique_id: + if latent_key in extra_pnginfo['workflow']['extra']: + print(f"Loading latent image from workflow node: {unique_id}") + try: + workflow_latent = self.deserialize(extra_pnginfo['workflow']['extra'][latent_key]) + except Exception as e: + print("There was an issue extracting the latent tensor from the workflow. Is it corrupted?") + workflow_latent = None + if not torch.is_tensor(image): + raise ValueError(f"Node {unique_id}: There was no image provided, and workflow latent missing. Unable to proceed.") + + if workflow_latent and remove_latent_on_load: + try: + del extra_pnginfo['workflow']['extra'][latent_key] + except Exception: + pass + + if workflow_latent: + print(f"Loaded workflow latent from node: {unique_id}") + return workflow_latent, { "extra_pnginfo": extra_pnginfo } + + if not torch.is_tensor(image): + raise ValueError(f"Node {unique_id}: No workflow latent was loaded, and no image provided to encode. Unable to proceed. ") + + if tiled: + encoded = self.VAEEncodeTiled.encode(pixels=image, tile_size=tile_size, vae=vae) + else: + encoded = self.VAEEncode.encode(pixels=image, vae=vae) + + if store_or_load_latent and unique_id: + print(f"Saving latent to workflow node {unique_id}") + new_workflow_latent = self.serialize(encoded[0]) + extra_pnginfo['workflow']['extra'][latent_key] = new_workflow_latent + + return encoded[0], { "extra_pnginfo": extra_pnginfo } + + def sha256(self, tensor): + tensor_bytes = tensor.cpu().contiguous().numpy().tobytes() + hash_obj = hashlib.sha256() + hash_obj.update(tensor_bytes) + return hash_obj.hexdigest() + + def serialize(self, obj): + json_str = json.dumps(obj, default=lambda o: {'__tensor__': True, 'value': o.cpu().numpy().tolist()} if torch.is_tensor(o) else o.__dict__) + compressed_data = zlib.compress(json_str.encode('utf-8')) + base64_encoded = base64.b64encode(compressed_data).decode('utf-8') + return base64_encoded + + def deserialize(self, base64_str): + compressed_data = base64.b64decode(base64_str) + json_str = zlib.decompress(compressed_data).decode('utf-8') + obj = json.loads(json_str, object_hook=lambda d: torch.tensor(d['value']) if '__tensor__' in d else d) + return obj + +``` diff --git a/docs/md/WAS_Extras/Nodes/CLIPTextEncodeList.md b/docs/md/WAS_Extras/Nodes/CLIPTextEncodeList.md new file mode 100644 index 0000000000..3dc7647bd2 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/CLIPTextEncodeList.md @@ -0,0 +1,81 @@ +# CLIP Text Encode Sequence (Advanced) +## Documentation +- Class name: `CLIPTextEncodeList` +- Category: `conditioning` +- Output node: `False` + +The node `CLIPTextEncodeList` is not explicitly defined in the provided context. However, based on the naming convention and the context of other CLIPTextEncode nodes, it can be inferred that if a `CLIPTextEncodeList` node existed, it would likely be involved in processing a list of text inputs through a CLIP model to generate encoded representations. These representations could then be used for various applications such as conditioning for generative models, text-based image editing, or enhancing text understanding in multimodal contexts. +## Input types +### Required +- **`clip`** + - The CLIP model used for encoding the text inputs. + - Python dtype: `sd1_clip.SDClipModel` + - Comfy dtype: `CLIP` +- **`token_normalization`** + - Specifies the method for normalizing tokens. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`weight_interpretation`** + - Defines how weights are interpreted in the encoding process. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`text`** + - A list of text inputs to be processed. + - Python dtype: `List[str]` + - Comfy dtype: `STRING` +## Output types +- **`conditioning_seq`** + - The encoded representations of the text inputs. + - Python dtype: `List[EncodedRepresentation]` + - Comfy dtype: `CONDITIONING_SEQ` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The `CLIPTextEncodeList` node is primarily utilized for encoding a list of text inputs using a CLIP model, producing encoded representations suitable for tasks like text-based image editing, enhancing multimodal text understanding, or conditioning for generative models. It is often used with conditioning nodes to provide detailed textual context for image generation processes. +## Source code +```python +class CLIPTextEncodeSequence: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "clip": ("CLIP", ), + "token_normalization": (["none", "mean", "length", "length+mean"],), + "weight_interpretation": (["comfy", "A1111", "compel", "comfy++"],), + "text": ("STRING", {"multiline": True, "default": '''0:A portrait of a rosebud +5:A portrait of a blooming rosebud +10:A portrait of a blooming rose +15:A portrait of a rose'''}), + } + } + + RETURN_TYPES = ("CONDITIONING_SEQ",) + RETURN_NAMES = ("conditioning_sequence",) + IS_LIST_OUTPUT = (True,) + + FUNCTION = "encode" + CATEGORY = "conditioning" + + def encode(self, clip, text, token_normalization, weight_interpretation): + text = text.strip() + conditionings = [] + for l in text.splitlines(): + match = re.match(r'(\d+):', l) + + if match: + idx = int(match.group(1)) + _, line = l.split(":", 1) + line = line.strip() + + if USE_BLK: + encoded = blk_adv.encode(clip=clip, text=line, token_normalization=token_normalization, weight_interpretation=weight_interpretation) + else: + encoded = CLIPTextEncode.encode(clip=clip, text=line) + + conditioning = (idx, [encoded[0][0][0], encoded[0][0][1]]) + conditionings.append(conditioning) + + return (conditionings, ) + +``` diff --git a/docs/md/WAS_Extras/Nodes/CLIPTextEncodeSequence2.md b/docs/md/WAS_Extras/Nodes/CLIPTextEncodeSequence2.md new file mode 100644 index 0000000000..68d858ae14 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/CLIPTextEncodeSequence2.md @@ -0,0 +1,120 @@ +# CLIP Text Encode Sequence (v2) +## Documentation +- Class name: `CLIPTextEncodeSequence2` +- Category: `conditioning` +- Output node: `False` + +This node processes a sequence of text lines for conditioning, using either a basic or an advanced CLIP text encoding method based on configuration. It supports token normalization and weight interpretation for the advanced encoding method, and organizes the encoded data along with their indices for further processing. +## Input types +### Required +- **`clip`** + - The CLIP model used for text encoding. It's crucial for converting text into a format that can be used for conditioning in generative models. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `CLIP` +- **`token_normalization`** + - Indicates whether token normalization is applied during encoding, affecting the representation of the text in the conditioning. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`weight_interpretation`** + - Determines how weights are interpreted in the advanced encoding process, influencing the final conditioning output. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`cond_keyframes_type`** + - Defines the method used to distribute keyframes across the conditioning sequence, influencing the temporal structure of the generated content. The choice of method (linear, sinusoidal, etc.) affects how the narrative or visual elements are paced over time, providing a framework for dynamic storytelling or visual transitions. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`frame_count`** + - Specifies the total number of frames to be generated, directly affecting the length and coverage of the conditioning sequence. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`text`** + - The input text, potentially containing multiple lines, each intended for separate encoding. The structure and content of this text directly influence the generated conditionings. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`conditioning`** + - A list of conditionings prepared for further generative processes. + - Python dtype: `List[Tuple[int, dict]]` + - Comfy dtype: `CONDITIONING` +- **`int`** + - The total number of frames in the conditioning sequence, confirming the sequence length. + - Python dtype: `int` + - Comfy dtype: `INT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The CLIPTextEncodeSequence2 node is primarily utilized for processing a sequence of text lines through either basic or advanced CLIP text encoding methods, tailored for dynamic storytelling or visual transitions in AI-generated content. It accepts multiline text as input and outputs a list of conditionings along with their indices and the total frame count, making it ideal for applications requiring nuanced text interpretation and temporal structuring in generative models. +## Source code +```python +class CLIPTextEncodeSequence2: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "clip": ("CLIP", ), + "token_normalization": (["none", "mean", "length", "length+mean"],), + "weight_interpretation": (["comfy", "A1111", "compel", "comfy++"],), + "cond_keyframes_type": (["linear", "sinus", "sinus_inverted", "half_sinus", "half_sinus_inverted"],), + "frame_count": ("INT", {"default": 100, "min": 1, "max": 1024, "step": 1}), + "text": ("STRING", {"multiline": True, "default": '''A portrait of a rosebud +A portrait of a blooming rosebud +A portrait of a blooming rose +A portrait of a rose'''}), + } + } + + RETURN_TYPES = ("CONDITIONING", "INT", "INT") + RETURN_NAMES = ("conditioning_sequence", "cond_keyframes", "frame_count") + IS_LIST_OUTPUT = (True, True, False) + + FUNCTION = "encode" + CATEGORY = "conditioning" + + def encode(self, clip, text, cond_keyframes_type, frame_count, token_normalization, weight_interpretation): + text = text.strip() + conditionings = [] + for line in text.splitlines(): + if USE_BLK: + encoded = blk_adv.encode(clip=clip, text=line, token_normalization=token_normalization, weight_interpretation=weight_interpretation) + else: + encoded = CLIPTextEncode.encode(clip=clip, text=line) + + conditionings.append([encoded[0][0][0], encoded[0][0][1]]) + + conditioning_count = len(conditionings) + cond_keyframes = self.calculate_cond_keyframes(cond_keyframes_type, frame_count, conditioning_count) + + return (conditionings, cond_keyframes, frame_count) + + def calculate_cond_keyframes(self, type, frame_count, conditioning_count): + if type == "linear": + return np.linspace(frame_count // conditioning_count, frame_count, conditioning_count, dtype=int).tolist() + + elif type == "sinus": + # Create a sinusoidal distribution + t = np.linspace(0, np.pi, conditioning_count) + sinus_values = np.sin(t) + # Normalize the sinusoidal values to 0-1 range + normalized_values = (sinus_values - sinus_values.min()) / (sinus_values.max() - sinus_values.min()) + # Scale to frame count and shift to avoid starting at frame 0 + scaled_values = normalized_values * (frame_count - 1) + 1 + # Ensure unique keyframes by rounding and converting to integer + unique_keyframes = np.round(scaled_values).astype(int) + # Deduplicate while preserving order + unique_keyframes = np.unique(unique_keyframes, return_index=True)[1] + return sorted(unique_keyframes.tolist()) + + elif type == "sinus_inverted": + return (np.cos(np.linspace(0, np.pi, conditioning_count)) * (frame_count - 1) + 1).astype(int).tolist() + + elif type == "half_sinus": + return (np.sin(np.linspace(0, np.pi / 2, conditioning_count)) * (frame_count - 1) + 1).astype(int).tolist() + + elif type == "half_sinus_inverted": + return (np.cos(np.linspace(0, np.pi / 2, conditioning_count)) * (frame_count - 1) + 1).astype(int).tolist() + + else: + raise ValueError("Unsupported cond_keyframes_type: " + type) + +``` diff --git a/docs/md/WAS_Extras/Nodes/ConditioningBlend.md b/docs/md/WAS_Extras/Nodes/ConditioningBlend.md new file mode 100644 index 0000000000..96b89fb3ec --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/ConditioningBlend.md @@ -0,0 +1,79 @@ +# Conditioning (Blend) +## Documentation +- Class name: `ConditioningBlend` +- Category: `conditioning` +- Output node: `False` + +The ConditioningBlend node blends two conditioning inputs using specified blending modes and strength, potentially influenced by a seed for deterministic results. It supports operations like spherical linear interpolation (SLERP) and subtraction, allowing for flexible manipulation of conditioning data. +## Input types +### Required +- **`conditioning_a`** + - The first conditioning input to blend. It plays a crucial role in determining the base of the blend operation. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +- **`conditioning_b`** + - The second conditioning input to blend. It contributes to the blend operation by providing an additional set of conditioning data to mix with the first input. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +- **`blending_mode`** + - Specifies the method used for blending the two conditioning inputs. Different modes can produce varied effects on the resulting conditioning. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`blending_strength`** + - Determines the intensity of the blend between the two conditioning inputs. A higher value results in a stronger influence of the second input. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`seed`** + - An optional seed for random number generation, ensuring reproducible results when blending. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`conditioning`** + - The result of blending the two conditioning inputs according to the specified mode and strength. + - Python dtype: `List[Tuple[torch.Tensor, Dict[str, torch.Tensor]]]` + - Comfy dtype: `CONDITIONING` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The ConditioningBlend node is utilized for merging two sets of conditioning data, such as those from different AI models or inputs, by applying a specified blending mode and strength, optionally influenced by a seed for consistent outcomes. It is particularly useful for adjusting the influence of one set of conditioning data over another, resulting in a customized conditioning output that can be used for more nuanced AI model interactions. +## Source code +```python +class WAS_ConditioningBlend: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "conditioning_a": ("CONDITIONING", ), + "conditioning_b": ("CONDITIONING", ), + "blending_mode": (list(blending_modes.keys()), ), + "blending_strength": ("FLOAT", {"default": 0.5, "min": -10.0, "max": 10.0, "step": 0.001}), + "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}), + } + } + + RETURN_TYPES = ("CONDITIONING",) + RETURN_NAMES = ("conditioning",) + FUNCTION = "combine" + + CATEGORY = "conditioning" + + def combine(self, conditioning_a, conditioning_b, blending_mode, blending_strength, seed): + + if seed > 0: + torch.manual_seed(seed) + + a = conditioning_a[0][0].clone() + b = conditioning_b[0][0].clone() + + pa = conditioning_a[0][1]["pooled_output"].clone() + pb = conditioning_b[0][1]["pooled_output"].clone() + + cond = normalize(blending_modes[blending_mode](a, b, 1 - blending_strength)) + pooled = normalize(blending_modes[blending_mode](pa, pb, 1 - blending_strength)) + + conditioning = [[cond, {"pooled_output": pooled}]] + + return (conditioning, ) + +``` diff --git a/docs/md/WAS_Extras/Nodes/DebugInput.md b/docs/md/WAS_Extras/Nodes/DebugInput.md new file mode 100644 index 0000000000..dff89825d9 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/DebugInput.md @@ -0,0 +1,50 @@ +# Debug Input +## Documentation +- Class name: `DebugInput` +- Category: `debug` +- Output node: `True` + +The `DebugInput` node is designed for debugging purposes. It prints the input it receives and, if the input is an object that is not a basic data type (str, int, float, bool, list, dict, tuple), it also prints a directory listing of the object. This functionality can be particularly useful for inspecting the structure and properties of complex objects during the development and debugging of workflows. +## Input types +### Required +- **`input`** + - The `input` parameter can accept any type of data, making this node highly versatile for debugging purposes. It is used to print the input data and, if applicable, the directory listing of the input object. + - Python dtype: `AnyType` + - Comfy dtype: `ANY` +## Output types +The node doesn't have output types +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used for inspecting the structure and properties of complex objects during the development and debugging of workflows, the `DebugInput` node prints the input data it receives and, if applicable, provides a directory listing of the input object. +## Source code +```python +class WAS_DebugThis: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(cls): + return { + "required": {"input": (wildcard, {})}, + } + + RETURN_TYPES = () + OUTPUT_NODE = True + + FUNCTION = "debug" + + CATEGORY = "debug" + + def debug(self, input): + + print("Debug:") + print(input) + if isinstance(input, object) and not isinstance(input, (str, int, float, bool, list, dict, tuple)): + print("Objects directory listing:") + pprint(dir(input), indent=4) + + return () + +``` diff --git a/docs/md/WAS_Extras/Nodes/KSamplerSeq.md b/docs/md/WAS_Extras/Nodes/KSamplerSeq.md new file mode 100644 index 0000000000..0868e7f276 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/KSamplerSeq.md @@ -0,0 +1,260 @@ +# KSampler Sequence +## Documentation +- Class name: `KSamplerSeq` +- Category: `sampling` +- Output node: `False` + +The `KSamplerSeq` node is designed for advanced image sampling in sequences, incorporating features like latent interpolation, conditioning slerp, and optional unsampling of latents. It allows for dynamic adjustment of denoising levels and supports various seed modes and alternate value toggling for enhanced control over the sampling process. +## Input types +### Required +- **`model`** + - Specifies the model to be used for sampling, central to the operation of the node. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`seed`** + - Determines the initial seed for randomness, affecting the sampling outcome. It can be dynamically adjusted based on the seed mode. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`seed_mode_seq`** + - Controls how the seed is updated across sequence iterations, allowing for incremental, decremental, random, or fixed seed values. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`alternate_values`** + - Enables or disables the alternation of certain parameters between sequence iterations, adding variability to the sampling process. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`steps`** + - Defines the number of steps to be used in the sampling process, impacting the detail and quality of the generated images. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`cfg`** + - Sets the classifier-free guidance scale, influencing the adherence to the provided conditioning. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sampler_name`** + - Selects the specific sampling algorithm to be used, affecting the characteristics of the generated images. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`scheduler`** + - Chooses the scheduler for controlling the sampling process, further customizing the generation. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`sequence_loop_count`** + - Determines the number of iterations for the sequence, allowing for extended or shortened sampling sequences. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`positive_seq`** + - Provides the positive conditioning sequence, guiding the sampling towards desired attributes. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `CONDITIONING_SEQ` +- **`negative_seq`** + - Supplies the negative conditioning sequence, steering the sampling away from undesired attributes. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `CONDITIONING_SEQ` +- **`use_conditioning_slerp`** + - Enables spherical linear interpolation (slerp) for conditioning, blending positive and negative conditioning smoothly. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`cond_slerp_strength`** + - Adjusts the strength of the conditioning slerp, controlling the blend between positive and negative conditioning. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`latent_image`** + - Provides the initial latent image for the sampling process, serving as a starting point. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +- **`use_latent_interpolation`** + - Activates latent interpolation, allowing for smooth transitions between latent images. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`latent_interpolation_mode`** + - Determines the method of latent interpolation, offering options like blend, slerp, and cosine interpolation. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`latent_interp_strength`** + - Controls the intensity of the latent interpolation, affecting the smoothness of transitions. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`denoise_start`** + - Sets the initial denoising level, impacting the clarity and detail of the initial images in the sequence. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`denoise_seq`** + - Specifies the denoising level for subsequent images in the sequence, allowing for dynamic adjustment of image clarity. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`unsample_latents`** + - Determines whether to perform unsampling on the latents, potentially enhancing image quality at the cost of additional computation. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +## Output types +- **`latent`** + - Produces a sequence of sampled images based on the provided parameters and conditions, showcasing the node's capability to generate diverse and high-quality images. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The `KSamplerSeq` node is pivotal for advanced image sequence generation, enabling features like latent interpolation and conditioning slerp, and is often used with conditioning sequences to guide the sampling towards or away from specific attributes. It takes a model, seed, and various sampling parameters as input, and outputs a sequence of sampled images, allowing for dynamic adjustments in denoising levels and seed modes for enhanced control over the image generation process. +## Source code +```python +class KSamplerSeq: + + def __init__(self): + self.previous_seed = None + self.current_seed = None + + def initialize_seeds(self, initial_seed): + self.previous_seed = initial_seed + self.current_seed = initial_seed + + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}), + "seed_mode_seq": (["increment", "decrement", "random", "fixed"],), + "alternate_values": ("BOOLEAN", {"default": True}), + "steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.5, "round": 0.01}), + "sampler_name": (comfy.samplers.KSampler.SAMPLERS, ), + "scheduler": (comfy.samplers.KSampler.SCHEDULERS, ), + "sequence_loop_count": ("INT", {"default": 20, "min": 1, "max": 1024, "step": 1}), + "positive_seq": ("CONDITIONING_SEQ", ), + "negative_seq": ("CONDITIONING_SEQ", ), + "use_conditioning_slerp": ("BOOLEAN", {"default": False}), + "cond_slerp_strength": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.001}), + "latent_image": ("LATENT", ), + "use_latent_interpolation": ("BOOLEAN", {"default": False}), + "latent_interpolation_mode": (["Blend", "Slerp", "Cosine Interp"],), + "latent_interp_strength": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.001}), + "denoise_start": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "denoise_seq": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}), + "unsample_latents": ("BOOLEAN", {"default": False}) + } + } + + RETURN_TYPES = ("LATENT",) + FUNCTION = "sample" + + CATEGORY = "sampling" + + def update_seed(self, seed, seed_mode): + if seed_mode == "increment": + return seed + 1 + elif seed_mode == "decrement": + return seed - 1 + elif seed_mode == "random": + return random.randint(0, 0xffffffffffffffff) + elif seed_mode == "fixed": + return seed + + def hash_tensor(self, tensor): + tensor = tensor.cpu().contiguous() + return hashlib.sha256(tensor.numpy().tobytes()).hexdigest() + + def update_conditioning(self, conditioning_seq, loop_count, last_conditioning): + matching_conditioning = None + for idx, conditioning, *_ in conditioning_seq: + if int(idx) == loop_count: + matching_conditioning = conditioning + break + return matching_conditioning if matching_conditioning else (last_conditioning if last_conditioning else None) + + def update_alternate_seed(self, loop_count): + if loop_count % 3 == 0: + if self.previous_seed is None: + self.previous_seed = self.current_seed + else: + self.previous_seed, self.current_seed = self.current_seed, self.previous_seed + 1 if loop_count // 2 % 2 == 0 else self.previous_seed - 1 + return self.current_seed + + def alternate_denoise(self, current_denoise): + return 0.95 if current_denoise == 0.75 else 0.75 + + def sample(self, model, seed, seed_mode_seq, alternate_values, steps, cfg, sampler_name, scheduler, sequence_loop_count, positive_seq, negative_seq, cond_slerp_strength, latent_image, use_latent_interpolation, latent_interpolation_mode, latent_interp_strength, denoise_start=1.0, denoise_seq=0.5, use_conditioning_slerp=False, unsample_latents=False, alternate_mode=False): + positive_seq = positive_seq + negative_seq = negative_seq + results = [] + positive_conditioning = None + negative_conditioning = None + + self.initialize_seeds(seed) + + for loop_count in range(sequence_loop_count): + if alternate_values and loop_count % 2 == 0: + seq_seed = self.update_alternate_seed(seed) if seed_mode_seq != "fixed" else seed + #denoise_seq = self.alternate_denoise(denoise_seq) + else: + seq_seed = seed if loop_count <= 0 else self.update_seed(seq_seed, seed_mode_seq) + + + print(f"Loop count: {loop_count}, Seed: {seq_seed}") + + last_positive_conditioning = positive_conditioning[0] if positive_conditioning else None + last_negative_conditioning = negative_conditioning[0] if negative_conditioning else None + + positive_conditioning = self.update_conditioning(positive_seq, loop_count, last_positive_conditioning) + negative_conditioning = self.update_conditioning(negative_seq, loop_count, last_negative_conditioning) + + if use_conditioning_slerp and (last_positive_conditioning and last_negative_conditioning): + a = last_positive_conditioning[0].clone() + b = positive_conditioning[0].clone() + na = last_negative_conditioning[0].clone() + nb = negative_conditioning[0].clone() + + pa = last_positive_conditioning[1]["pooled_output"].clone() + pb = positive_conditioning[1]["pooled_output"].clone() + npa = last_negative_conditioning[1]["pooled_output"].clone() + npb = negative_conditioning[1]["pooled_output"].clone() + + pos_cond = slerp(cond_slerp_strength, a, b) + pos_pooled = slerp(cond_slerp_strength, pa, pb) + neg_cond = slerp(cond_slerp_strength, na, nb) + neg_pooled = slerp(cond_slerp_strength, npa, npb) + + positive_conditioning = [pos_cond, {"pooled_output": pos_pooled}] + negative_conditioning = [neg_cond, {"pooled_output": neg_pooled}] + + positive_conditioning = [positive_conditioning] + negative_conditioning = [negative_conditioning] + + if positive_conditioning is not None or negative_conditioning is not None: + + end_at_step = steps + if results is not None and len(results) > 0: + latent_input = {'samples': results[-1]} + denoise = denoise_seq + start_at_step = round((1 - denoise) * steps) + end_at_step = steps + else: + latent_input = latent_image + denoise = denoise_start + + if unsample_latents and loop_count > 0: + force_full_denoise = False if loop_count > 0 or loop_count <= steps - 1 else True + disable_noise = False + unsampled_latent = unsample(model=model, seed=seq_seed, cfg=cfg, sampler_name=sampler_name, steps=steps, end_at_step=end_at_step, scheduler=scheduler, normalize=False, positive=positive_conditioning, negative=negative_conditioning, latent_image=latent_input)[0] + sample = nodes.common_ksampler(model, seq_seed, steps, cfg, sampler_name, scheduler, positive_conditioning, negative_conditioning, unsampled_latent, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise)[0]['samples'] + else: + sample = nodes.common_ksampler(model, seq_seed, steps, cfg, sampler_name, scheduler, positive_conditioning, negative_conditioning, latent_input, denoise=denoise)[0]['samples'] + + if use_latent_interpolation and results and loop_count > 0: + if latent_interpolation_mode == "Blend": + sample = blend_latents(latent_interp_strength, results[-1], sample) + elif latent_interpolation_mode == "Slerp": + sample = slerp_latents(latent_interp_strength, results[-1], sample) + elif latent_interpolation_mode == "Cosine Interp": + sample = cosine_interp_latents(latent_interp_strength, results[-1], sample) + else: + sample = sample + + results.append(sample) + + results = torch.cat(results, dim=0) + results = {'samples': results} + + return (results,) + +``` diff --git a/docs/md/WAS_Extras/Nodes/KSamplerSeq2.md b/docs/md/WAS_Extras/Nodes/KSamplerSeq2.md new file mode 100644 index 0000000000..d1d72668dc --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/KSamplerSeq2.md @@ -0,0 +1,323 @@ +# KSampler Sequence (v2) +## Documentation +- Class name: `KSamplerSeq2` +- Category: `sampling` +- Output node: `False` + +KSamplerSeq2 is an advanced version of the KSampler sequence node designed for generating sequences of images or latents by iteratively sampling and optionally modifying the sampling process based on various parameters. It supports features like unsampling latents, injecting noise, and using different seed modes for each iteration, providing a flexible framework for creative image generation. +## Input types +### Required +- **`model`** + - The model parameter specifies the generative model to be used for sampling. It is crucial for defining the architecture and parameters of the model that will generate the images or latents. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `MODEL` +- **`seed`** + - The seed parameter is used to initialize the random number generator, ensuring reproducibility of the samples. It can be modified per iteration based on the seed mode. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`seed_mode_seq`** + - This parameter allows for controlling how the seed is updated across iterations, offering options to diversify or stabilize the generated sequences. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`alternate_values`** + - The alternate_values boolean controls whether to alternate certain parameters between iterations, adding variability to the sampling process. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`steps`** + - Specifies the number of steps to be used in the sampling process. A higher number of steps generally leads to more detailed and coherent samples. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`cfg`** + - The cfg parameter adjusts the conditioning strength, influencing the adherence of the generated samples to the provided conditioning. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`sampler_name`** + - Determines the specific sampling algorithm to be used, which can affect the characteristics of the generated samples. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`scheduler`** + - The scheduler parameter selects the scheduling algorithm for controlling the denoising process during sampling, impacting the quality and diversity of the output. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`frame_count`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`cond_keyframes`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +- **`positive_seq`** + - A sequence of positive conditioning inputs to guide the generation towards desired attributes or content, enhancing the relevance and appeal of the generated sequence. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `CONDITIONING` +- **`negative_seq`** + - A sequence of negative conditioning inputs to steer the generation away from undesired attributes or content, ensuring the generated sequence aligns with the intended direction. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `CONDITIONING` +- **`use_conditioning_slerp`** + - Controls whether spherical linear interpolation (slerp) is applied to the conditioning inputs, blending them smoothly across iterations. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`cond_slerp_strength`** + - Adjusts the strength of the conditioning slerp, allowing for fine-tuning the blend between conditioning inputs. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`latent_image`** + - The initial latent image to start the sequence generation from. It can be modified or unsampled in subsequent iterations. + - Python dtype: `torch.Tensor` + - Comfy dtype: `LATENT` +- **`use_latent_interpolation`** + - Determines whether to apply interpolation to the latent images between iterations, which can create smoother transitions in the generated sequence. + - Python dtype: `bool` + - Comfy dtype: `BOOLEAN` +- **`latent_interpolation_mode`** + - Selects the mode of interpolation for the latent images, such as blending, spherical linear interpolation, or cosine interpolation. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`latent_interp_strength`** + - Controls the strength of the latent interpolation, affecting the degree of blending between latent images. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`denoise_start`** + - Sets the initial denoising strength for the sampling process, influencing the clarity and detail of the first generated sample. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`denoise_seq`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`unsample_latents`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `BOOLEAN` +- **`inject_noise`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `BOOLEAN` +- **`noise_strength`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`denoise_sine`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `BOOLEAN` +- **`denoise_max`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `FLOAT` +- **`seed_keying`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `BOOLEAN` +- **`seed_keying_mode`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `['sine', 'modulo'...]` +- **`seed_divisor`** + - unknown + - Python dtype: `unknown` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The output is a sequence of generated samples or images, produced by iteratively applying the specified sampling and modification parameters. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The KSamplerSeq2 node is designed for generating sequences of images or latents by iteratively sampling and modifying the sampling process based on various parameters such as seed mode, noise injection, and conditioning strength. It is often used for creating dynamic and creative image sequences with the ability to fine-tune each step of the generation process, making it ideal for applications requiring sequential image generation or animation. +## Source code +```python +class KSamplerSeq2: + + def __init__(self): + self.previous_seed = None + self.current_seed = None + + def initialize_seeds(self, initial_seed): + self.previous_seed = initial_seed + self.current_seed = initial_seed + + @classmethod + def INPUT_TYPES(s): + return {"required": + {"model": ("MODEL",), + "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}), + "seed_mode_seq": (["increment", "decrement", "random", "fixed"],), + "alternate_values": ("BOOLEAN", {"default": True}), + "steps": ("INT", {"default": 20, "min": 1, "max": 10000}), + "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.5, "round": 0.01}), + "sampler_name": (comfy.samplers.KSampler.SAMPLERS, ), + "scheduler": (comfy.samplers.KSampler.SCHEDULERS, ), + "frame_count": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 1}), + "cond_keyframes": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 1}), + "positive_seq": ("CONDITIONING", ), + "negative_seq": ("CONDITIONING", ), + "use_conditioning_slerp": ("BOOLEAN", {"default": False}), + "cond_slerp_strength": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.001}), + "latent_image": ("LATENT", ), + "use_latent_interpolation": ("BOOLEAN", {"default": False}), + "latent_interpolation_mode": (["Blend", "Slerp", "Cosine Interp"],), + "latent_interp_strength": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.001}), + "denoise_start": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "denoise_seq": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}), + "unsample_latents": ("BOOLEAN", {"default": False}), + "inject_noise": ("BOOLEAN", {"default": True}), + "noise_strength": ("FLOAT", {"default": 0.1, "max": 1.0, "min": 0.001, "step": 0.001}), + "denoise_sine": ("BOOLEAN", {"default": True}), + "denoise_max": ("FLOAT", {"default": 0.9, "max": 1.0, "min": 0.0, "step": 0.001}), + "seed_keying": ("BOOLEAN", {"default": True}), + "seed_keying_mode": (["sine", "modulo"],), + "seed_divisor": ("INT", {"default": 4, "max": 1024, "min": 2, "step": 1}), + } + } + + RETURN_TYPES = ("LATENT",) + FUNCTION = "sample" + + CATEGORY = "sampling" + + def update_seed(self, seed, seed_mode): + if seed_mode == "increment": + return seed + 1 + elif seed_mode == "decrement": + return seed - 1 + elif seed_mode == "random": + return random.randint(0, 0xffffffffffffffff) + elif seed_mode == "fixed": + return seed + + def alternate_seed_modulo(self, current, seed, divisor): + if current % divisor == 0: + new_seed = (seed + current) % 0xffffffffffffffff + else: + new_seed = seed + return new_seed + + def alternate_seed_sine(self, current, start_seed, divisor): + seed = 1000 * np.sin(2 * math.pi * current / divisor) + start_seed + return seed + + def alternate_denoise(self, curent, total, start_denoise=0.5, max_denoise=0.95): + amplitude = (max_denoise - start_denoise) / 2 + mid_point = (max_denoise + start_denoise) / 2 + cycle_position = (math.pi * 2 * curent) / total + current_denoise = amplitude * math.sin(cycle_position) + mid_point + return current_denoise + + def inject_noise(self, latent_image, noise_strength): + noise = torch.randn_like(latent_image) * noise_strength + return latent_image + noise + + def sample(self, model, seed, seed_mode_seq, alternate_values, steps, cfg, sampler_name, scheduler, + frame_count, cond_keyframes, positive_seq, negative_seq, cond_slerp_strength, latent_image, + use_latent_interpolation, latent_interpolation_mode, latent_interp_strength, denoise_start=1.0, + denoise_seq=0.5, use_conditioning_slerp=False, unsample_latents=False, alternate_mode=False, + inject_noise=True, noise_strength=0.1, denoise_sine=True, denoise_max=0.9, seed_keying=True, + seed_keying_mode="sine", seed_divisor=4): + + if not isinstance(positive_seq, list): + positive_seq = [positive_seq] + if not isinstance(negative_seq, list): + negative_seq = [negative_seq] + if not isinstance(cond_keyframes, list): + cond_keyframes = [cond_keyframes] + cond_keyframes.sort() + + positive_cond_idx = 0 + negative_cond_idx = 0 + results = [] + + self.initialize_seeds(seed) + sequence_loop_count = max(frame_count, len(positive_seq)) if cond_keyframes else len(positive_seq) + + print(f"Starting loop sequence with {sequence_loop_count} frames.") + print(f"Using {len(positive_seq)} positive conditionings and {len(negative_seq)} negative conditionings") + print(f"Conditioning keyframe schedule is: {', '.join(map(str, cond_keyframes))}") + + + for loop_count in range(sequence_loop_count): + if loop_count in cond_keyframes: + positive_cond_idx = min(positive_cond_idx + 1, len(positive_seq) - 1) + negative_cond_idx = min(negative_cond_idx + 1, len(negative_seq) - 1) + + positive_conditioning = positive_seq[positive_cond_idx] + negative_conditioning = negative_seq[negative_cond_idx] + + if seed_keying: + if seed_keying_mode == "sine": + seq_seed = seed if loop_count <= 0 else self.alternate_seed_sine(loop_count, seed, seed_divisor) + else: + seq_seed = seed if loop_count <= 0 else self.alternate_seed_modulo(loop_count, seed, seed_divisor) + else: + seq_seed = seed if loop_count <= 0 else self.update_seed(seq_seed, seed_mode_seq) + + seq_seed = seed if loop_count <= 0 else self.update_seed(seq_seed, seed_mode_seq) + print(f"Loop count: {loop_count}, Seed: {seq_seed}") + + last_positive_conditioning = positive_conditioning if positive_conditioning else None + last_negative_conditioning = negative_conditioning if negative_conditioning else None + + if use_conditioning_slerp and (last_positive_conditioning and last_negative_conditioning): + a, b = last_positive_conditioning[0].clone(), positive_conditioning[0].clone() + na, nb = last_negative_conditioning[0].clone(), negative_conditioning[0].clone() + pa, pb = last_positive_conditioning[1]["pooled_output"].clone(), positive_conditioning[1]["pooled_output"].clone() + npa, npb = last_negative_conditioning[1]["pooled_output"].clone(), negative_conditioning[1]["pooled_output"].clone() + pos_cond = slerp(cond_slerp_strength, a, b) + pos_pooled = slerp(cond_slerp_strength, pa, pb) + neg_cond = slerp(cond_slerp_strength, na, nb) + neg_pooled = slerp(cond_slerp_strength, npa, npb) + positive_conditioning = [pos_cond, {"pooled_output": pos_pooled}] + negative_conditioning = [neg_cond, {"pooled_output": neg_pooled}] + + positive_conditioning = [positive_conditioning] + negative_conditioning = [negative_conditioning] + + end_at_step = steps + if results and len(results) > 0: + latent_input = {'samples': results[-1]} + denoise = self.alternate_denoise(loop_count, sequence_loop_count, denoise_seq, denoise_max) if denoise_sine else denoise_seq + start_at_step = round((1 - denoise) * steps) + end_at_step = steps + else: + latent_input = latent_image + denoise = denoise_start + + if unsample_latents and loop_count > 0: + force_full_denoise = not (loop_count > 0 or loop_count <= steps - 1) + disable_noise = False + if seed_keying: + if seed_keying_mode == "modulo" and loop_count % seed_divisor == 0: + unsampled_latent = latent_input + else: + unsampled_latent = unsample(model=model, seed=seq_seed, cfg=cfg, sampler_name=sampler_name, steps=steps, end_at_step=end_at_step, scheduler=scheduler, normalize=False, positive=positive_conditioning, negative=negative_conditioning, latent_image=latent_input)[0] + else: + unsampled_latent = unsample(model=model, seed=seq_seed, cfg=cfg, sampler_name=sampler_name, steps=steps, end_at_step=end_at_step, scheduler=scheduler, normalize=False, positive=positive_conditioning, negative=negative_conditioning, latent_image=latent_input)[0] + if inject_noise and loop_count > 0: + print(f"Injecting noise at {noise_strength} strength.") + unsampled_latent['samples'] = self.inject_noise(unsampled_latent['samples'], noise_strength) + sample = nodes.common_ksampler(model, seq_seed, steps, cfg, sampler_name, scheduler, positive_conditioning, negative_conditioning, unsampled_latent, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise)[0]['samples'] + else: + if inject_noise and loop_count > 0: + print(f"Injecting noise at {noise_strength} strength.") + latent_input['samples'] = self.inject_noise(latent_input['samples'], noise_strength) + sample = nodes.common_ksampler(model, seq_seed, steps, cfg, sampler_name, scheduler, positive_conditioning, negative_conditioning, latent_input, denoise=denoise)[0]['samples'] + + if use_latent_interpolation and results and loop_count > 0: + if latent_interpolation_mode == "Blend": + sample = blend_latents(latent_interp_strength, results[-1], sample) + elif latent_interpolation_mode == "Slerp": + sample = slerp_latents(latent_interp_strength, results[-1], sample) + elif latent_interpolation_mode == "Cosine Interp": + sample = cosine_interp_latents(latent_interp_strength, results[-1], sample) + + results.append(sample) + + results = torch.cat(results, dim=0) + results = {'samples': results} + return (results,) + +``` diff --git a/docs/md/WAS_Extras/Nodes/VAEEncodeForInpaint (WAS).md b/docs/md/WAS_Extras/Nodes/VAEEncodeForInpaint (WAS).md new file mode 100644 index 0000000000..8875bbb999 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/VAEEncodeForInpaint (WAS).md @@ -0,0 +1,86 @@ +# Inpainting VAE Encode (WAS) +## Documentation +- Class name: `VAEEncodeForInpaint (WAS)` +- Category: `latent/inpaint` +- Output node: `False` + +This node is designed for encoding images for inpainting tasks using a Variational Autoencoder (VAE). It preprocesses the input image and mask to ensure compatibility with the VAE's requirements, applies mask modifications based on the provided offset, and then encodes the modified image into a latent representation suitable for inpainting. +## Input types +### Required +- **`pixels`** + - The input image to be encoded for inpainting. It undergoes preprocessing to match the VAE's input dimensions and is modified based on the mask before encoding. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`vae`** + - The Variational Autoencoder (VAE) model used for encoding the preprocessed image into a latent representation. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `VAE` +- **`mask`** + - A mask indicating the regions to inpaint. It is used to modify the input image before encoding. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +- **`mask_offset`** + - Determines how much to erode or dilate the mask. This affects the area of the image that will be inpainted. + - Python dtype: `int` + - Comfy dtype: `INT` +## Output types +- **`latent`** + - The encoded latent representation of the input image, suitable for inpainting tasks. + - Python dtype: `Dict[str, torch.Tensor]` + - Comfy dtype: `LATENT` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The VAEEncodeForInpaint (WAS) node is designed for encoding images for inpainting tasks, where it preprocesses the input image and mask to match the VAE's requirements, modifies the mask based on the provided offset, and encodes the image into a latent representation suitable for inpainting. It is often used with inpainting models to prepare the image and mask for generating the inpainted output. +## Source code +```python +class WAS_VAEEncodeForInpaint: + @classmethod + def INPUT_TYPES(s): + return {"required": { "pixels": ("IMAGE", ), "vae": ("VAE", ), "mask": ("MASK", ), "mask_offset": ("INT", {"default": 6, "min": -128, "max": 128, "step": 1}),}} + RETURN_TYPES = ("LATENT",) + FUNCTION = "encode" + + CATEGORY = "latent/inpaint" + + def encode(self, vae, pixels, mask, mask_offset=6): + x = (pixels.shape[1] // 8) * 8 + y = (pixels.shape[2] // 8) * 8 + mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(pixels.shape[1], pixels.shape[2]), mode="bilinear") + + pixels = pixels.clone() + if pixels.shape[1] != x or pixels.shape[2] != y: + x_offset = (pixels.shape[1] % 8) // 2 + y_offset = (pixels.shape[2] % 8) // 2 + pixels = pixels[:,x_offset:x + x_offset, y_offset:y + y_offset,:] + mask = mask[:,:,x_offset:x + x_offset, y_offset:y + y_offset] + + mask_erosion = self.modify_mask(mask, mask_offset) + + m = (1.0 - mask_erosion.round()).squeeze(1) + for i in range(3): + pixels[:,:,:,i] -= 0.5 + pixels[:,:,:,i] *= m + pixels[:,:,:,i] += 0.5 + t = vae.encode(pixels) + + return ({"samples":t, "noise_mask": (mask_erosion[:,:,:x,:y].round())}, ) + + def modify_mask(self, mask, modify_by): + if modify_by == 0: + return mask + if modify_by > 0: + kernel_size = 2 * modify_by + 1 + kernel_tensor = torch.ones((1, 1, kernel_size, kernel_size)) + padding = modify_by + modified_mask = torch.clamp(torch.nn.functional.conv2d(mask.round(), kernel_tensor, padding=padding), 0, 1) + else: + kernel_size = 2 * abs(modify_by) + 1 + kernel_tensor = torch.ones((1, 1, kernel_size, kernel_size)) + padding = abs(modify_by) + eroded_mask = torch.nn.functional.conv2d(1 - mask.round(), kernel_tensor, padding=padding) + modified_mask = torch.clamp(1 - eroded_mask, 0, 1) + return modified_mask + +``` diff --git a/docs/md/WAS_Extras/Nodes/VividSharpen.md b/docs/md/WAS_Extras/Nodes/VividSharpen.md new file mode 100644 index 0000000000..3a01865e43 --- /dev/null +++ b/docs/md/WAS_Extras/Nodes/VividSharpen.md @@ -0,0 +1,68 @@ +# VividSharpen +## Documentation +- Class name: `VividSharpen` +- Category: `image/postprocessing` +- Output node: `False` + +The VividSharpen node applies a vivid sharpening effect to images. It enhances image details by applying a vivid light and overlay blend based on the specified radius and strength of the sharpening effect. +## Input types +### Required +- **`images`** + - The images to be sharpened. This input is crucial for determining the visual quality and detail enhancement of the output images. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`radius`** + - Defines the radius of the Gaussian blur applied during the sharpening process. A larger radius results in a more pronounced sharpening effect. The radius directly influences the extent of the blur effect, which is a critical factor in achieving the desired vivid sharpening outcome. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`strength`** + - Controls the intensity of the sharpening effect. A higher strength value leads to a more vivid and pronounced sharpening. The strength parameter adjusts the blending level of the sharpened layer with the original image, affecting the overall sharpness and clarity. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The sharpened images, enhanced with vivid light and overlay effects for improved detail and visual quality. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The VividSharpen node is primarily utilized for enhancing the visual quality of images by applying a vivid sharpening effect, which is achieved through adjusting the radius and strength parameters. It takes images as input and outputs images with improved detail and clarity, making it ideal for post-processing tasks in image enhancement pipelines. +## Source code +```python +class VividSharpen: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "images": ("IMAGE",), + "radius": ("FLOAT", {"default": 1.5, "min": 0.01, "max": 64.0, "step": 0.01}), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + }, + } + + RETURN_TYPES = ("IMAGE",) + RETURN_NAMES = ("images",) + + FUNCTION = "sharpen" + + CATEGORY = "image/postprocessing" + + def sharpen(self, images, radius, strength): + + results = [] + if images.size(0) > 1: + for image in images: + image = tensor2pil(image) + results.append(pil2tensor(vivid_sharpen(image, radius=radius, strength=strength))) + results = torch.cat(results, dim=0) + else: + results = pil2tensor(vivid_sharpen(tensor2pil(images), radius=radius, strength=strength)) + + return (results,) + +``` diff --git a/docs/md/WAS_Extras/index.md b/docs/md/WAS_Extras/index.md new file mode 100644 index 0000000000..1d918a8ffc --- /dev/null +++ b/docs/md/WAS_Extras/index.md @@ -0,0 +1,3 @@ +# WAS_Extras +- repo url: `https://github.com/WASasquatch/WAS_Extras.git` +- commit hash: `755e5597c787d4ef1d1bc1e099da11827c326e6f` \ No newline at end of file diff --git a/docs/md/comfyui_segment_anything/Nodes/GroundingDinoModelLoader (segment anything).md b/docs/md/comfyui_segment_anything/Nodes/GroundingDinoModelLoader (segment anything).md new file mode 100644 index 0000000000..77b0d0265e --- /dev/null +++ b/docs/md/comfyui_segment_anything/Nodes/GroundingDinoModelLoader (segment anything).md @@ -0,0 +1,42 @@ +# GroundingDinoModelLoader (segment anything) +## Documentation +- Class name: `GroundingDinoModelLoader` +- Category: `segment_anything` +- Output node: `False` + +This node is responsible for loading a specific GroundingDINO model based on the provided model name. GroundingDINO is a model architecture designed for visual grounding tasks, which involves identifying and localizing objects within images based on textual descriptions. The node dynamically loads the model configuration and weights, adjusts the text encoder if necessary, and prepares the model for inference. +## Input types +### Required +- **`model_name`** + - Specifies the name of the GroundingDINO model to be loaded. This name is used to retrieve the model's configuration and weights for initialization. The choice of model directly influences the performance and capabilities of the visual grounding task. The model name must be one of the keys from the groundingdino_model_list, which includes a predefined list of available models. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`grounding_dino_model`** + - The loaded GroundingDINO model, ready for performing visual grounding tasks. This model is capable of understanding textual descriptions and localizing the described objects within images. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `GROUNDING_DINO_MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `GroundingDinoSAMSegment (segment anything)` + +The GroundingDinoModelLoader node is responsible for loading the GroundingDINO model, which excels in visual grounding tasks by identifying and localizing objects within images based on textual descriptions. It is often used with segmentation nodes like GroundingDinoSAMSegment to precisely segment specific parts of the image, enhancing tasks such as image editing, enhancement, and object isolation. +## Source code +```python +class GroundingDinoModelLoader: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model_name": (list_groundingdino_model(), ), + } + } + CATEGORY = "segment_anything" + FUNCTION = "main" + RETURN_TYPES = ("GROUNDING_DINO_MODEL", ) + + def main(self, model_name): + dino_model = load_groundingdino_model(model_name) + return (dino_model, ) + +``` diff --git a/docs/md/comfyui_segment_anything/Nodes/GroundingDinoSAMSegment (segment anything).md b/docs/md/comfyui_segment_anything/Nodes/GroundingDinoSAMSegment (segment anything).md new file mode 100644 index 0000000000..28173243d9 --- /dev/null +++ b/docs/md/comfyui_segment_anything/Nodes/GroundingDinoSAMSegment (segment anything).md @@ -0,0 +1,94 @@ +# GroundingDinoSAMSegment (segment anything) +## Documentation +- Class name: `GroundingDinoSAMSegment` +- Category: `segment_anything` +- Output node: `False` + +The GroundingDinoSAMSegment node is designed for segmenting various objects within an image using the GroundingDINO model. It leverages advanced deep learning techniques, specifically a transformer-based architecture, to accurately identify and segment objects by understanding the context and details within the image. This node is part of a larger system that integrates with the GroundingDINO framework, which is known for its effectiveness in conditional object detection and segmentation tasks. +## Input types +### Required +- **`sam_model`** + - Specifies the SAM model used for the segmentation of detected objects. This model refines the segmentation results provided by the GroundingDINO model. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `SAM_MODEL` +- **`grounding_dino_model`** + - Specifies the GroundingDINO model to be used for object detection. This model is essential for identifying objects within the image based on the given prompt. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `GROUNDING_DINO_MODEL` +- **`image`** + - The input image to be processed. The GroundingDINO model uses this image to detect objects based on the provided prompt. + - Python dtype: `torch.Tensor` + - Comfy dtype: `IMAGE` +- **`prompt`** + - A text prompt that guides the GroundingDINO model in detecting specific objects within the image. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`threshold`** + - A threshold value that determines the sensitivity of object detection. Higher values result in fewer detections. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +## Output types +- **`image`** + - The images resulting from the segmentation process, containing the detected and segmented objects. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `IMAGE` +- **`mask`** + - The masks corresponding to each segmented object in the images. These masks outline the boundaries of detected objects. + - Python dtype: `List[torch.Tensor]` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `GPU` +- Common nodes: `PreviewImage,Reroute,VAEEncodeForInpaint,MaskToImage,Mask Gaussian Region,ArithmeticBlend,InvertMask (segment anything),InvertMask,GrowMask` + +The GroundingDinoSAMSegment node is designed for advanced image segmentation, leveraging the GroundingDINO and SAM models to accurately identify and segment objects within an image based on textual prompts. It takes as input an image, a SAM model, a Grounding Dino model, and a textual prompt, and outputs segmented parts of the image along with corresponding masks, enabling precise editing and manipulation tasks in pipelines focused on image enhancement, editing, or conditional manipulation. +## Source code +```python +class GroundingDinoSAMSegment: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "sam_model": ('SAM_MODEL', {}), + "grounding_dino_model": ('GROUNDING_DINO_MODEL', {}), + "image": ('IMAGE', {}), + "prompt": ("STRING", {}), + "threshold": ("FLOAT", { + "default": 0.3, + "min": 0, + "max": 1.0, + "step": 0.01 + }), + } + } + CATEGORY = "segment_anything" + FUNCTION = "main" + RETURN_TYPES = ("IMAGE", "MASK") + + def main(self, grounding_dino_model, sam_model, image, prompt, threshold): + res_images = [] + res_masks = [] + for item in image: + item = Image.fromarray( + np.clip(255. * item.cpu().numpy(), 0, 255).astype(np.uint8)).convert('RGBA') + boxes = groundingdino_predict( + grounding_dino_model, + item, + prompt, + threshold + ) + if boxes.shape[0] == 0: + break + (images, masks) = sam_segment( + sam_model, + item, + boxes + ) + res_images.extend(images) + res_masks.extend(masks) + if len(res_images) == 0: + _, height, width, _ = image.size() + empty_mask = torch.zeros((1, height, width), dtype=torch.uint8, device="cpu") + return (empty_mask, empty_mask) + return (torch.cat(res_images, dim=0), torch.cat(res_masks, dim=0)) + +``` diff --git a/docs/md/comfyui_segment_anything/Nodes/InvertMask (segment anything).md b/docs/md/comfyui_segment_anything/Nodes/InvertMask (segment anything).md new file mode 100644 index 0000000000..913fe713a3 --- /dev/null +++ b/docs/md/comfyui_segment_anything/Nodes/InvertMask (segment anything).md @@ -0,0 +1,42 @@ +# InvertMask (segment anything) +## Documentation +- Class name: `InvertMask` +- Category: `segment_anything` +- Output node: `False` + +The InvertMask node inverts the input mask by subtracting each element from 1.0, effectively flipping the mask's binary values. This operation is useful for switching the focus between the masked and unmasked regions in various image processing tasks. +## Input types +### Required +- **`mask`** + - The input mask to be inverted. This mask defines areas of interest in an image or another graphical representation. Inverting the mask switches the focus between the foreground and background. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`mask`** + - The inverted mask, with each original value subtracted from 1.0. This output is useful for operations that require the opposite areas of the original mask to be highlighted. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Usage tips +- Infra type: `CPU` +- Common nodes: `MaskToImage,PorterDuffImageComposite,ImageCompositeMasked,VAEEncodeForInpaint,FeatherMask` + +The InvertMask node inverts the input mask by subtracting each element from 1.0, effectively flipping the mask's binary values. This operation is useful for switching the focus between the masked and unmasked regions in various image processing tasks, often used with segmentation to invert selected areas for operations like inpainting or selective application of effects. +## Source code +```python +class InvertMask: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mask": ("MASK",), + } + } + CATEGORY = "segment_anything" + FUNCTION = "main" + RETURN_TYPES = ("MASK",) + + def main(self, mask): + out = 1.0 - mask + return (out,) + +``` diff --git a/docs/md/comfyui_segment_anything/Nodes/IsMaskEmpty.md b/docs/md/comfyui_segment_anything/Nodes/IsMaskEmpty.md new file mode 100644 index 0000000000..7d0bb0ed6c --- /dev/null +++ b/docs/md/comfyui_segment_anything/Nodes/IsMaskEmpty.md @@ -0,0 +1,43 @@ +# IsMaskEmpty +## Documentation +- Class name: `IsMaskEmptyNode` +- Category: `segment_anything` +- Output node: `False` + +This node checks if a given mask is entirely empty (i.e., all its elements are zero) and returns a number indicating the result. It's useful for determining whether any part of an image or area is covered by a mask. +## Input types +### Required +- **`mask`** + - The mask to be checked for emptiness. This input is crucial for determining whether the mask covers any area or not. + - Python dtype: `torch.Tensor` + - Comfy dtype: `MASK` +## Output types +- **`number`** + - A number indicating whether the mask is empty (1) or not (0). This numeric representation is useful for conditional operations based on mask coverage. + - Python dtype: `int` + - Comfy dtype: `NUMBER` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used in pipelines to determine if a selected area of an image is devoid of any features or objects, by checking if the mask applied to that area is entirely empty. This node takes a mask as input and outputs a number indicating whether the mask is empty (1) or not (0), which is crucial for conditional operations in image processing and segmentation tasks. +## Source code +```python +class IsMaskEmptyNode: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "mask": ("MASK",), + }, + } + RETURN_TYPES = ["NUMBER"] + RETURN_NAMES = ["boolean_number"] + + FUNCTION = "main" + CATEGORY = "segment_anything" + + def main(self, mask): + return (torch.all(mask == 0).int().item(), ) + +``` diff --git a/docs/md/comfyui_segment_anything/Nodes/SAMModelLoader (segment anything).md b/docs/md/comfyui_segment_anything/Nodes/SAMModelLoader (segment anything).md new file mode 100644 index 0000000000..0625d41a1a --- /dev/null +++ b/docs/md/comfyui_segment_anything/Nodes/SAMModelLoader (segment anything).md @@ -0,0 +1,42 @@ +# SAMModelLoader (segment anything) +## Documentation +- Class name: `SAMModelLoader (segment anything)` +- Category: `segment_anything` +- Output node: `False` + +The SAMModelLoader node is responsible for loading a specific SAM (Segment Anything Model) based on the provided model name. It retrieves the model from a predefined list of available models, downloads it if necessary, and prepares it for inference by setting it to evaluation mode and moving it to the appropriate device (CPU or GPU). +## Input types +### Required +- **`model_name`** + - Specifies the name of the SAM model to be loaded. This parameter is crucial as it determines which specific model is retrieved and prepared for use, impacting the node's execution and the results of the segmentation process. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`sam_model`** + - The loaded SAM model, ready for inference. This includes setting the model to evaluation mode and moving it to the appropriate device. + - Python dtype: `torch.nn.Module` + - Comfy dtype: `SAM_MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: `GroundingDinoSAMSegment (segment anything)` + +The SAMModelLoader (segment anything) node is responsible for loading a specific SAM (Segment Anything Model) for use in image or video segmentation tasks. It takes a model name as input and outputs a SAM_MODEL, ready for inference, which is crucial for precisely segmenting various objects within the input data without requiring specific training on those objects. +## Source code +```python +class SAMModelLoader: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model_name": (list_sam_model(), ), + } + } + CATEGORY = "segment_anything" + FUNCTION = "main" + RETURN_TYPES = ("SAM_MODEL", ) + + def main(self, model_name): + sam_model = load_sam_model(model_name) + return (sam_model, ) + +``` diff --git a/docs/md/comfyui_segment_anything/index.md b/docs/md/comfyui_segment_anything/index.md new file mode 100644 index 0000000000..c73d74d9be --- /dev/null +++ b/docs/md/comfyui_segment_anything/index.md @@ -0,0 +1,3 @@ +# comfyui_segment_anything +- repo url: `https://github.com/storyicon/comfyui_segment_anything.git` +- commit hash: `681d977c2cc244de729fd9e08145ff365c609262` \ No newline at end of file diff --git a/docs/md/index.md b/docs/md/index.md new file mode 100644 index 0000000000..9c942834ee --- /dev/null +++ b/docs/md/index.md @@ -0,0 +1,4 @@ +# Welcome to SALT nodes documentation! + +!!! note + This documentation is part of an ongoing research project and is continuously updated to reflect our latest findings and improvements. Content may change, and new information will be added over time. We recommend revisiting periodically for the most current insights. diff --git a/docs/md/llmnodes/Nodes/DocumentNode.md b/docs/md/llmnodes/Nodes/DocumentNode.md new file mode 100644 index 0000000000..177b630f46 --- /dev/null +++ b/docs/md/llmnodes/Nodes/DocumentNode.md @@ -0,0 +1,74 @@ +# Document Node +## Documentation +- Class name: `DocumentNode` +- Category: `LLM Nodes` +- Output node: `True` + +This node is responsible for generating documentation for a specific node within a pack. It identifies the node based on its display name and pack name, then retrieves and formats its documentation. +## Input types +### Required +- **`name`** + - The unique identifier for the node, composed of the pack name and node display name. It is used to locate the specific node within a pack for documentation. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`chunk_lines`** + - Specifies the number of lines each documentation chunk should contain. This helps in managing the size of the documentation generated. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`chunk_lines_overlap`** + - Determines the number of lines that can overlap between two consecutive chunks of documentation. This ensures continuity and context preservation in the documentation. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`max_chars`** + - Sets the maximum number of characters allowed in the documentation. This limit helps in keeping the documentation concise. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`num_retries`** + - Defines the number of attempts to regenerate the documentation in case of failure. This ensures reliability in documentation generation. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`top_k`** + - Determines the top K models to consider for generating the documentation. This parameter helps in selecting the best model based on performance. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`regenerate`** + - Controls whether to regenerate the documentation, offering flexibility based on the requirement. This choice influences the documentation generation process, potentially leading to multiple attempts for optimal results. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`model`** + - Specifies the model to be used for generating the documentation. This parameter allows for flexibility in choosing the appropriate model, directly impacting the quality and style of the generated documentation. + - Python dtype: `str` + - Comfy dtype: `MODEL` +## Output types +- **`string`** + - The generated documentation for the specified node. This output is a string that contains the formatted documentation. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The Document Node is primarily utilized for generating and formatting documentation for a specific node within a pack, based on its display name and pack name. It takes the unique identifier of the node, along with parameters like chunk lines, overlap, and model choice, to produce a concise, formatted documentation string as output. +## Source code +```python +class DocumentNode(DocumentPack): + @classmethod + def get_all_names(cls): + cls.all_packs = get_all_nodes_packs() + return [ + f"{pack_name}/{node['display_name']}" + for pack_name, pack in cls.all_packs.items() + for node in pack["nodes"].values() + ] + + def document(self, *args, **kwargs) -> Tuple: + name = kwargs.pop("name") + pack_name = name.split("/")[0] + node_display_name = "/".join(name.split("/")[1:]) + for node_name, node_info in self.all_packs[pack_name]["nodes"].items(): + if node_info["display_name"] == node_display_name: + break + nodes_list = [node_name] + return self._document(pack_name=pack_name, nodes_list=nodes_list, *args, **kwargs) + +``` diff --git a/docs/md/llmnodes/Nodes/DocumentPack.md b/docs/md/llmnodes/Nodes/DocumentPack.md new file mode 100644 index 0000000000..a06848667f --- /dev/null +++ b/docs/md/llmnodes/Nodes/DocumentPack.md @@ -0,0 +1,225 @@ +# Document Pack +## Documentation +- Class name: `DocumentPack` +- Category: `LLM Nodes` +- Output node: `True` + +The `DocumentPack` class serves as a foundational component in a system designed to manage and document nodes within a larger framework. It primarily acts as a base class, providing essential functionalities and structures for node documentation and management. This includes methods for retrieving all node names and packs, which are crucial for organizing and accessing the various nodes available within the system. +## Input types +### Required +- **`name`** + - The `name` parameter specifies the unique identifier of a node within a pack. It is essential for retrieving specific node documentation, as it determines which node's information is to be documented based on its unique name. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`chunk_lines`** + - Specifies the number of lines each code chunk should contain. This is important for controlling the granularity of documentation and affects how code is segmented and documented. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`chunk_lines_overlap`** + - Determines the number of lines that consecutive code chunks will overlap. This parameter is crucial for ensuring continuity and context between chunks, enhancing the quality of generated documentation. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`max_chars`** + - Sets the maximum number of characters allowed in a single code chunk. This parameter helps in managing the size of documentation chunks, ensuring they are of a manageable size for processing and understanding. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`num_retries`** + - Indicates the number of times the documentation generation process should retry in case of failures. This parameter is important for robustness, ensuring that temporary issues do not prevent documentation from being generated. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`top_k`** + - Specifies the number of top results to consider when generating documentation. This parameter influences the quality and relevance of the generated documentation by focusing on the most promising results. + - Python dtype: `int` + - Comfy dtype: `INT` +- **`regenerate`** + - Controls whether existing documentation should be regenerated. This parameter allows for flexibility in managing documentation updates and ensures that the most current information is always available. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`model`** + - Refers to the model used for generating documentation. This parameter is critical as it determines the quality and style of the generated documentation, based on the capabilities of the specified model. + - Python dtype: `Dict[str, Any]` + - Comfy dtype: `MODEL` +## Output types +- **`string`** + - The output is a tuple containing the generated documentation for the specified nodes. This provides a structured and accessible way to retrieve documentation for nodes within the system. + - Python dtype: `Tuple[str,]` + - Comfy dtype: `STRING` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +The DocumentPack node is primarily used for generating and managing documentation for other nodes within a pipeline, taking as input parameters like node names, code chunk specifications, and model details, and outputs structured documentation in the form of strings. This node is essential for organizing and accessing comprehensive documentation on the various nodes available within the system, ensuring users can easily understand and manage their AI pipelines. +## Source code +```python +class DocumentPack: + @classmethod + def get_all_names(cls): + cls.all_packs = get_all_nodes_packs() + return [f"{pack_name}/{len(pack['nodes'])} nodes" for pack_name, pack in cls.all_packs.items()] + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "name": [ + sorted(cls.get_all_names()), + ], + "chunk_lines": ("INT", {"default": 40}), + "chunk_lines_overlap": ("INT", {"default": 15}), + "max_chars": ("INT", {"default": 1500}), + "num_retries": ("INT", {"default": 5}), + "top_k": ("INT", {"default": 10}), + "regenerate": ([e.value for e in RegenerateOptions], {"default": RegenerateOptions.no.value}), + "model": ("MODEL",), + } + } + + RETURN_TYPES = ("STRING",) + OUTPUT_NODE = True + FUNCTION = "document" + CATEGORY = NAME + + def document(self, *args, **kwargs) -> Tuple: + pack_name = kwargs.pop("name").split("/")[0] + nodes_list = self.all_packs[pack_name]["nodes"] + return self._document(pack_name=pack_name, nodes_list=nodes_list, *args, **kwargs) + + def _document( + self, + pack_name: str, + nodes_list: Dict[str, Any], + chunk_lines: int, + chunk_lines_overlap: int, + max_chars: int, + num_retries: int, + top_k: int, + regenerate: str, + model: Dict[str, Any], + ) -> Tuple[str,]: + self._init_phoenix() + emb_name = model["name"].split("_")[1] + f"_{chunk_lines}_{chunk_lines_overlap}_{max_chars}" + query_name = model["name"] + f"_{chunk_lines}_{chunk_lines_overlap}_{max_chars}_{num_retries}_{top_k}" + Settings.llm = model["llm"] + Settings.embed_model = model["embed_model"] + load_index_kwargs = dict( + emb_name=emb_name, + chunk_lines=chunk_lines, + chunk_lines_overlap=chunk_lines_overlap, + max_chars=max_chars, + regenerate=regenerate == "index & doc", + ) + # adding comfy files because many packs import from there + global comfy_nodes_index + if comfy_nodes_index is None: + comfy_nodes_index = self._load_index(pack_name="Comfy", **load_index_kwargs) + indexes = [comfy_nodes_index] + if pack_name != "Comfy": + indexes.append(self._load_index(pack_name=pack_name, **load_index_kwargs)) + node_query_engine = NodeQueryEngine(indexes=indexes, num_retries=num_retries, top_k=top_k) + is_max, no_skip, no_error = [], [], [] + for node_name in nodes_list: + debug_info = self._document_node(node_query_engine, query_name, pack_name, node_name, regenerate) + score, max_score = debug_info["_eval_score"].split("/") + is_max.append(int(score) == int(max_score)) + no_skip.append(int(not debug_info["_has_unknown"])) + no_error.append(debug_info["_error"] is None) + Settings.llm = "default" + Settings.embed_model = "default" + log_str = f"""Nodes with max score: {sum(is_max)}/{len(is_max)}\n +Nodes without skipped fields: {sum(no_skip)}/{len(no_skip)}\n +Nodes without errors: {sum(no_error)}/{len(no_error)}""" + self._log_phoenix() + return (log_str,) + + def _load_index( + self, + emb_name: str, + pack_name: str, + chunk_lines: int, + chunk_lines_overlap: int, + max_chars: int, + regenerate: bool, + ) -> VectorStoreIndex: + persist_dir = MAIN_CACHE / "indices" / emb_name / pack_name + if not regenerate and persist_dir.exists() and any(persist_dir.iterdir()): + logger.debug(f"Loading index for {pack_name} from {persist_dir}...") + return load_index_from_storage(StorageContext.from_defaults(persist_dir=persist_dir)) + persist_dir.mkdir(parents=True, exist_ok=True) + code_path = self.all_packs[pack_name]["code_path"] + code_files = [code_path] if str(code_path).endswith(".py") else list(code_path.rglob("*.py")) + documents = SimpleDirectoryReader(input_files=code_files).load_data() + splitter = CodeSplitter( + language="python", chunk_lines=chunk_lines, chunk_lines_overlap=chunk_lines_overlap, max_chars=max_chars + ) + index = VectorStoreIndex(splitter.get_nodes_from_documents(documents), use_async=True) + index.storage_context.persist(persist_dir=persist_dir) + return index + + def _document_node( + self, + node_query_engine: NodeQueryEngine, + query_name: str, + pack_name: str, + node_name: str, + regenerate: str, + ) -> Dict[str, str]: + json_file = MAIN_CACHE / "documented_nodes" / query_name / pack_name / f"{node_name}.json" + old_score, old_unknown = None, None + if regenerate in ["no", "failed"] and json_file.exists(): + with open(json_file, "r") as file: + old_debug_info = json.load(file)["_debug_info"] + if regenerate == "no": + logger.debug(f"Loading existing response for {node_name} from {json_file}...") + return old_debug_info + if regenerate == "failed": + if old_debug_info["_error"] is None: + score, max_score = old_debug_info["_eval_score"].split("/") + old_score, old_unknown = int(score) / int(max_score), old_debug_info["_has_unknown"] + if not old_unknown and old_score == 1: + logger.debug(f"Loading existing response for {node_name} from {json_file}...") + return old_debug_info + node_info = self.all_packs[pack_name]["nodes"][node_name] + response = node_query_engine.query(node_name, node_info) + debug_info = response["_debug_info"] + if old_score is not None: + # don't update the result if we made it even worse + score, max_score = debug_info["_eval_score"].split("/") + new_score, new_unknown = int(score) / int(max_score), debug_info["_has_unknown"] + if old_unknown: + if new_unknown and new_score < old_score: + return old_debug_info + else: + if new_unknown or not new_unknown and new_score < old_score: + return old_debug_info + + json_file.parent.mkdir(parents=True, exist_ok=True) + with open(json_file, "w") as file: + json.dump(response, file, indent=4) + return debug_info + + def _init_phoenix(self) -> None: + if type(sys.stdout).__name__ == "ComfyUIManagerLogger": + sys.stdout.close_log() + config = { + "handlers": [ + {"sink": sys.stdout, "format": "{time} - {message}"}, + {"sink": sys.stderr, "format": "{time} - {message}"}, + ], + } + logger.configure(**config) + sess = px.active_session() + # clear before next run, instead of at the end, to allow debugging of outputs + if sess is not None: + sess.end() + px.launch_app() + + def _log_phoenix(self) -> None: + from phoenix.trace.dsl import SpanQuery + + query = SpanQuery().select(tokens_in="llm.token_count.prompt", tokens_out="llm.token_count.completion") + # The Phoenix Client can take this query and return the dataframe + info_df = px.Client().query_spans(query) + if info_df is not None: + logger.info(f"Total tokens in: {info_df.tokens_in.sum()}. Total tokens out: {info_df.tokens_out.sum()}") + +``` diff --git a/docs/md/llmnodes/Nodes/LoadOpenAIModel.md b/docs/md/llmnodes/Nodes/LoadOpenAIModel.md new file mode 100644 index 0000000000..85b75ea226 --- /dev/null +++ b/docs/md/llmnodes/Nodes/LoadOpenAIModel.md @@ -0,0 +1,58 @@ +# Load OpenAI Model +## Documentation +- Class name: `LoadOpenAIModel` +- Category: `LLM Nodes` +- Output node: `False` + +This node is responsible for loading an OpenAI model along with its embedding model based on the specified parameters. It creates a unique identifier for the loaded model combination by concatenating the model names and temperature setting. This allows for the flexible use of different OpenAI models and their embeddings in downstream tasks. +## Input types +### Required +- **`model`** + - Specifies the OpenAI model to be loaded. The choice of model can significantly impact the behavior and capabilities of the generated outputs. It is selected from a predefined list of available models, allowing for customization based on the task at hand. + - Python dtype: `str` + - Comfy dtype: `MODEL` +- **`temperature`** + - Controls the randomness of the output generation. A higher temperature results in more random outputs, while a lower temperature produces more deterministic outputs. + - Python dtype: `float` + - Comfy dtype: `FLOAT` +- **`embed_model`** + - Determines the embedding model to be used alongside the main OpenAI model. This affects how inputs are processed and represented internally. The embedding model is chosen from a predefined list, enabling tailored input processing. + - Python dtype: `str` + - Comfy dtype: `MODEL` +## Output types +- **`model`** + - The combined OpenAI model and embedding model loaded based on the specified parameters. + - Python dtype: `Dict[str, Any]` + - Comfy dtype: `MODEL` +## Usage tips +- Infra type: `GPU` +- Common nodes: unknown + +Often used with downstream tasks such as text generation, summarization, or embedding extraction, the LoadOpenAIModel node loads a specified OpenAI model and its embedding counterpart based on user-defined parameters, enabling tailored AI functionalities. +## Source code +```python +class LoadOpenAIModel: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model": (sorted(ALL_AVAILABLE_MODELS.keys()), {"default": "gpt-4-turbo-preview"}), + "temperature": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0}), + "embed_model": ( + sorted([x.value for x in OpenAIEmbeddingModelType]), + {"default": "text-embedding-3-small"}, + ), + } + } + + RETURN_TYPES = ("MODEL",) + FUNCTION = "load_openai_model" + CATEGORY = NAME + + def load_openai_model(self, model: str, temperature: int, embed_model: str) -> Dict[str, Any]: + name = f"{model}_{embed_model}_{temperature}" + llm = OpenAI(model=model, temperature=temperature) + embed_model = OpenAIEmbedding(model=embed_model) + return ({"llm": llm, "embed_model": embed_model, "name": name},) + +``` diff --git a/docs/md/llmnodes/index.md b/docs/md/llmnodes/index.md new file mode 100644 index 0000000000..0935a4a6fe --- /dev/null +++ b/docs/md/llmnodes/index.md @@ -0,0 +1,3 @@ +# llmnodes +- repo url: `https://gitlab.com/plai-app/ai-studio/orchestra-nodes/llmnodes.git` +- commit hash: `98cb2de96916d7ad4227903fe3bc76a158864ce7` \ No newline at end of file diff --git a/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppFooter.md b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppFooter.md new file mode 100644 index 0000000000..befcf7b8d2 --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppFooter.md @@ -0,0 +1,50 @@ +# App Footer (PlaiFlow Player) +## Documentation +- Class name: `PlaiFlowPlayerAppFooter` +- Category: `PlaiFlow Player` +- Output node: `False` + +This node is responsible for generating the footer content of the PlaiFlow Player application. It takes a string input representing the footer content and outputs a UI component that displays this content at the bottom of the application. +## Input types +### Required +- **`footer_content`** + - The footer content to be displayed below the app. This string can be multiline, allowing for more detailed footer information or acknowledgments. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`ui`** + - The UI component that represents the footer content to be displayed at the bottom of the PlaiFlow Player application. +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used to customize the user interface by adding a personalized footer to the PlaiFlow Player application, taking a multiline string as input to display detailed footer information or acknowledgments at the bottom of the app. +## Source code +```python +class PlaiFlowPlayerAppFooter: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "footer_content": ("STRING", { + "multiline": True, + "dynamicPrompts": False, + "placeholder": "The footer content displayed below the app" + }), + } + } + + RETURN_TYPES = () + + FUNCTION = "app_info" + CATEGORY = NAME + + def app_info(self, **kwargs): + ui = { "ui": { "PlaiFlowPlayerAppOutput": { "footer_content": kwargs['footer_content'] } } } + + from pprint import pprint + pprint(ui, indent=4) + + return ui + +``` diff --git a/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppInfo.md b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppInfo.md new file mode 100644 index 0000000000..c7ebf28c85 --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppInfo.md @@ -0,0 +1,51 @@ +# App Info (PlaiFlow Player) +## Documentation +- Class name: `PlaiFlowPlayerAppInfo` +- Category: `PlaiFlow Player` +- Output node: `False` + +This node is responsible for generating the application information UI, including the title and description of the application. It takes the title and description as inputs and constructs a UI element that displays this information. +## Input types +### Required +- **`title`** + - The title of the application. It is a crucial element as it represents the application's identity and is prominently displayed in the app info UI. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`description`** + - The description of the application. It provides users with more context about the application's purpose and functionality, enhancing the app info UI. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`ui`** + - The constructed UI element that displays the application's title and description. +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used at the beginning of a pipeline to provide users with an overview of the application, taking the application's title and description as inputs and generating a UI element that displays this information. +## Source code +```python +class PlaiFlowPlayerAppInfo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "title": ("STRING", {}), + "description": ("STRING", {}), + } + } + + RETURN_TYPES = () + + FUNCTION = "app_info" + CATEGORY = NAME + + def app_info(self, **kwargs): + ui = { "ui": { "PlaiFlowPlayerAppInfo": { "title": kwargs['title'], "description": kwargs['description'] } } } + + from pprint import pprint + pprint(ui, indent=4) + + return ui + +``` diff --git a/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerInput.md b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerInput.md new file mode 100644 index 0000000000..eac0377219 --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerInput.md @@ -0,0 +1,191 @@ +# Input Node (PlaiFlow Player) +## Documentation +- Class name: `PlaiFlowPlayerInput` +- Category: `PlaiFlow Player` +- Output node: `True` + +The `PlaiFlowPlayerInput` node is designed to handle various types of input data for the PlaiFlow Player. It supports a wide range of data types including strings, numbers, images, and more, allowing for flexible data input handling within the PlaiFlow Player environment. +## Input types +### Required +- **`field_name`** + - Specifies the name of the input field. This is crucial for identifying and processing the input data correctly within the node. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`field_desc`** + - Provides a description for the input field, enhancing the understanding and context of the data being processed. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`data_type`** + - Determines the type of the input data, supporting a variety of formats such as strings, floats, integers, images, and more. This flexibility allows the node to handle a wide range of input data types. + - Python dtype: `Union[str, float, int, Any]` + - Comfy dtype: `['STRING', 'FLOAT', 'INT', 'IMAGE', 'IMAGES', 'MASK', 'SAM', 'DOCUMENT', 'SELECTION']` +- **`field_value`** + - Contains the actual value of the input data, which is processed based on the specified data type. + - Python dtype: `str` + - Comfy dtype: `STRING` +### Optional +- **`image`** + - Optional parameter for providing an image as input. This is used when the input data type is specifically an image. + - Python dtype: `Any` + - Comfy dtype: `IMAGE` +- **`base64_image`** + - Optional parameter for providing an image in base64 encoded string format. This allows for image data to be transmitted and processed as a string. + - Python dtype: `str` + - Comfy dtype: `STRING` +## Output types +- **`*`** + - The processed output value of the input data, which can be of any type depending on the input data type specified. + - Python dtype: `Any` + - Comfy dtype: `*` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used to provide a variety of input data types, including strings, numbers, images, and more, to the PlaiFlow Player, allowing for flexible data input handling within AI pipelines. +## Source code +```python +class PlaiFlowPlayerInput: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "field_name": ("STRING", {}), + "field_desc": ("STRING", {}), + "data_type": (["STRING", "FLOAT", "INT", "IMAGE", "IMAGES", "MASK", "SAM", "DOCUMENT", "SELECTION"],), + "field_value": ("STRING", {"multiline": True, "dynamicPrompts": False}), + }, + "optional": { + "image": ("IMAGE",), + "base64_image": ("STRING", {"multiline": True, "dynamicPrompts": False}), + } + } + + OUTPUT_NODE = True + RETURN_TYPES = (WILDCARD, ) + RETURN_NAMES = ("value", ) + + FUNCTION = "input" + CATEGORY = NAME + + def input(self, field_name, field_desc, field_value, data_type, image=None, base64_image=None, file_list=[]): + + if data_type == 'IMAGE': + + src_image = None + + if not image and not base64_image: + raise ValueError("A image, or base64_image input is required when using data_type `IMAGE`") + + if isinstance(image, torch.Tensor): + src_image = image + + if base64_image: + try: + src_image = pil2tensor(Image.open(BytesIO(base64.b64decode(base64_image))).convert("RGB")) + except Exception as e: + print("There was an error processing the base64 string, is it malformed?") + raise e + + return (src_image, ) + + if data_type == 'MASK': + + src_image = None + + if isinstance(image, torch.Tensor): + src_image = image + + if base64_image: + try: + src_image = pil2mask(Image.open(BytesIO(base64.b64decode(base64_image))).convert("L")) + except Exception as e: + print("There was an error processing the base64 string, is it malformed?") + raise e + + return (src_image, ) + + elif data_type == "IMAGES": + + tensors = [] + + if not image and field_value.strip() == "": + raise ValueError("No image batch provided, or field_value consisting of a UUID") + + if isinstance(image, torch.Tensor): + return (image, ) + + try: + uuid_obj = uuid.UUID(field_value, version=4) + except ValueError as e: + print("There was an error processing `field_value` as a valid UUID4 hash pointing to a cache location!") + raise e + + uuid = str(uuid_obj) + + return (load_images(uuid_obj, ['.jpeg', '.jpg', '.png', '.gif', '.webp']), ) + + + out = "" + match data_type: + + case 'STRING': + + out = field_value + + case 'INT': + + out = int(field_value) + + case 'FLOAT': + + out = float(field_value) + + case 'SELECTION': + + fields = field_value.splitlines() + if fields: + out = str(fields[0]) + else: + raise ValueError("Unable to determine any list values!") + + case 'SAM': + + src_image = None + + if base64_image: + try: + src_image = pil2tensor(Image.open(BytesIO(base64.b64decode(base64_image))).convert("RGB")) + except Exception as e: + print("There was an error processing the base64 string, is it malformed?") + raise e + + # Parse JSON + sam_json = json.loads(field_value) + + points_raw = sam_json.get('points', '[128, 128]; [0, 0]') + points_list = [list(map(int, point.strip('[] ').split(','))) for point in points_raw.split(';')] + points_array = np.array(points_list) + + labels_list = sam_json.get('labels', [1, 0]) + labels_array = np.array(labels_list) + + sam_params = { + "points": points_array, + "labels": labels_array + } + + out = (sam_params, src_image) + + + + case _: + + out = field_value + + + print(f"PlaiFlow Player Input `{field_name}` ({data_type}) Value:") + print(out) + + return (out, ) + +``` diff --git a/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerModelList.md b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerModelList.md new file mode 100644 index 0000000000..b21d510382 --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerModelList.md @@ -0,0 +1,48 @@ +# Model List (PlaiFlow Player) +## Documentation +- Class name: `PlaiFlowPlayerModelList` +- Category: `PlaiFlow Player` +- Output node: `False` + +This node generates a list of files based on a given directory. It is useful for dynamically populating a dropdown or selection list with available files within a specified directory. +## Input types +### Required +- **`combo_list`** + - The directory from which to generate the file list. This selection is crucial as it determines the scope of files to be included in the output list. + - Python dtype: `List[str]` + - Comfy dtype: `['STRING']` +## Output types +- **`*`** + - A list of files from the specified directory. This output can be used to populate UI elements with file options. + - Python dtype: `List[str]` + - Comfy dtype: `*` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used with UI elements to dynamically populate a dropdown or selection list with available files from a specified directory, based on the user's selection in the `combo_list`. +## Source code +```python +class PlaiFlowPlayerModelList: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "combo_list": (list(folder_paths.folder_names_and_paths.keys()),) + }, + } + + RETURN_TYPES = (WILDCARD,) + RETURN_NAMES = ("file_list",) + OUTPUT_IS_LIST = (True,) + + FUNCTION = "file_list" + CATEGORY = NAME + + def file_list(self, combo_list): + if combo_list.strip() == "": + raise ValueError("The selection is empty or invalid!") + files = folder_paths.get_filename_list(combo_list) + return (files, ) + +``` diff --git a/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerOutput.md b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerOutput.md new file mode 100644 index 0000000000..0e5c8bc03c --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerOutput.md @@ -0,0 +1,94 @@ +# Output Node (PlaiFlow Player) +## Documentation +- Class name: `PlaiFlowPlayerOutput` +- Category: `PlaiFlow Player` +- Output node: `True` + +This node is responsible for processing and outputting data in various formats such as images, video paths, strings, or document responses based on the input data type. It dynamically handles different types of data, converting and saving them appropriately, and constructs a UI response that encapsulates the processed data along with its type. +## Input types +### Required +- **`file_name_and_ext`** + - The name and extension of the file to be processed. This parameter is crucial as it determines the naming convention for saved files or the extraction of file names from paths. + - Python dtype: `str` + - Comfy dtype: `STRING` +- **`data_type`** + - Specifies the type of data being processed (e.g., IMAGE, VIDEO_PATH, STRING, DOCUMENT_RESPONSE), which dictates how the data is handled and formatted for output. + - Python dtype: `str` + - Comfy dtype: `['IMAGE', 'VIDEO_PATH', 'STRING', 'DOCUMENT_RESPONSE']` +- **`data`** + - The actual data to be processed. Depending on the data type, this could be image tensors, video paths, strings, or document responses. + - Python dtype: `Union[List[Any], Tuple[Any, ...], Any]` + - Comfy dtype: `*` +## Output types +- **`ui`** + - A UI response that includes the processed data and its type, formatted for display within the PlaiFlow Player environment. +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +Often used with nodes that process various data types like images, video paths, strings, or document responses, to dynamically handle and convert these data into a format suitable for UI display within the PlaiFlow Player environment. +## Source code +```python +class PlaiFlowPlayerOutput: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "file_name_and_ext": ("STRING", {}), + "data_type": (["IMAGE", "VIDEO_PATH", "STRING", "DOCUMENT_RESPONSE", "OBJECT"],), + "data": (WILDCARD,), + } + } + + OUTPUT_NODE = True + RETURN_TYPES = () + + FUNCTION = "output" + CATEGORY = NAME + + @classmethod + def IS_CHANGED(s, image): + return float("NaN") + + def output(self, file_name_and_ext, data_type, data): + + response = [] + match(data_type): + + case 'IMAGE': + try: + for index, img in enumerate(data): + pil_image = tensor2pil(img) + file_name, file_ext = os.path.splitext(file_name_and_ext) + formatted_file_name = f"{file_name}_{index:04d}{file_ext}" + image_path = os.path.join(folder_paths.get_output_directory(), formatted_file_name) + pil_image.save(image_path) + response.append(formatted_file_name) + except Exception as e: + raise e + + case 'VIDEO_PATH': + video_path = data[1][1] + response = os.path.basename(video_path) + + case 'STRING': + if isinstance(data, list): + response = data + else: + response.append(str(data)) + + case 'DOCUMENT_RESPONSE': + if isinstance(data, list): + response = data + else: + response.append(str(data)) + + + ui = { "ui": { "PlaiFlowPlayerOutput": response, "data_type": str(data_type) } } + + from pprint import pprint + pprint(ui, indent=4) + + return ui + +``` diff --git a/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerSAMSplit.md b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerSAMSplit.md new file mode 100644 index 0000000000..50a86d8069 --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerSAMSplit.md @@ -0,0 +1,52 @@ +# Split SAM Data (PlaiFlow Player) +## Documentation +- Class name: `PlaiFlowPlayerSAMSplit` +- Category: `PlaiFlow Player` +- Output node: `False` + +The `PlaiFlowPlayerSAMSplit` node is designed to process and split SAM (Spatial Analysis Map) data into its constituent parameters and an image. It validates the input to ensure it is in the expected tuple format, suitable for further analysis or visualization. +## Input types +### Required +- **`input_node_sam_value`** + - This parameter represents the SAM data to be processed. It is crucial for the operation as it contains the data that will be split into parameters and an image for further use. + - Python dtype: `Tuple[Dict[str, np.ndarray], Optional[Image.Image]]` + - Comfy dtype: `*` +## Output types +- **`*`** + - Contains the parameters extracted from the SAM data, essential for understanding or analyzing the spatial aspects. + - Python dtype: `Dict[str, np.ndarray]` + - Comfy dtype: `*` +- **`image`** + - Represents the image part of the SAM data, useful for visualization purposes. + - Python dtype: `Optional[Image.Image]` + - Comfy dtype: `IMAGE` +## Usage tips +- Infra type: `CPU` +- Common nodes: unknown + +The `PlaiFlowPlayerSAMSplit` node is essential for processing and splitting SAM (Spatial Analysis Map) data into parameters and an image, making it crucial for tasks that require detailed spatial analysis or visualization. It is often used with nodes designed for visualizing spatial data or further analyzing the extracted parameters. +## Source code +```python +class PlaiFlowPlayerSAMSplit: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "input_node_sam_value": (WILDCARD,) + }, + } + + RETURN_TYPES = (WILDCARD, "IMAGE") + RETURN_NAMES = ("sam_params", "image") + + FUNCTION = "extract_sam_data" + CATEGORY = NAME + + def extract_sam_data(self, input_node_sam_value): + if not isinstance(input_node_sam_value, tuple): + raise ValueError("Invalid SAM data deteceted! Split SAM Data expects output from a PlaiFlow Player Iput Node in SAM mode.") + if len(input_node_sam_value) > 2: + raise ValueError("Too many values to unpack for SAM data! Split SAM Data expects output from a PlaiFlow Player Iput Node in SAM mode.") + return input_node_sam_value[0], input_node_sam_value[1] + +``` diff --git a/docs/md/plaiflowplayer_blocks/index.md b/docs/md/plaiflowplayer_blocks/index.md new file mode 100644 index 0000000000..aefec7e5c1 --- /dev/null +++ b/docs/md/plaiflowplayer_blocks/index.md @@ -0,0 +1,3 @@ +# plaiflowplayer_blocks +- repo url: `https://gitlab.com/plai-app/ai-studio/orchestra-nodes/plaiflowplayer_blocks` +- commit hash: `ce26a8d5c7aeae5f698ecf02b3010482f1dd191b` \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 48eebb93c1..2ab09971a2 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -3,254 +3,275 @@ nav: - Getting Started: index.md - Platform Resources: resources.md - Nodes packs documentation: - - Comfy: - - BasicScheduler: md/Comfy/BasicScheduler.md - - CLIPLoader: md/Comfy/CLIPLoader.md - - CLIPMergeSimple: md/Comfy/CLIPMergeSimple.md - - CLIPSave: md/Comfy/CLIPSave.md - - CLIPSetLastLayer: md/Comfy/CLIPSetLastLayer.md - - CLIPTextEncode: md/Comfy/CLIPTextEncode.md - - CLIPTextEncodeControlnet: md/Comfy/CLIPTextEncodeControlnet.md - - CLIPTextEncodeSDXL: md/Comfy/CLIPTextEncodeSDXL.md - - CLIPTextEncodeSDXLRefiner: md/Comfy/CLIPTextEncodeSDXLRefiner.md - - CLIPVisionEncode: md/Comfy/CLIPVisionEncode.md - - CLIPVisionLoader: md/Comfy/CLIPVisionLoader.md - - Canny: md/Comfy/Canny.md - - CheckpointLoader: md/Comfy/CheckpointLoader.md - - CheckpointLoaderSimple: md/Comfy/CheckpointLoaderSimple.md - - CheckpointSave: md/Comfy/CheckpointSave.md - - ConditioningAverage: md/Comfy/ConditioningAverage.md - - ConditioningCombine: md/Comfy/ConditioningCombine.md - - ConditioningConcat: md/Comfy/ConditioningConcat.md - - ConditioningSetArea: md/Comfy/ConditioningSetArea.md - - ConditioningSetAreaPercentage: md/Comfy/ConditioningSetAreaPercentage.md - - ConditioningSetAreaStrength: md/Comfy/ConditioningSetAreaStrength.md - - ConditioningSetMask: md/Comfy/ConditioningSetMask.md - - ConditioningSetTimestepRange: md/Comfy/ConditioningSetTimestepRange.md - - ConditioningZeroOut: md/Comfy/ConditioningZeroOut.md - - ControlNetApply: md/Comfy/ControlNetApply.md - - ControlNetApplyAdvanced: md/Comfy/ControlNetApplyAdvanced.md - - ControlNetLoader: md/Comfy/ControlNetLoader.md - - CropMask: md/Comfy/CropMask.md - - DiffControlNetLoader: md/Comfy/DiffControlNetLoader.md - - DiffusersLoader: md/Comfy/DiffusersLoader.md - - DualCLIPLoader: md/Comfy/DualCLIPLoader.md - - EmptyImage: md/Comfy/EmptyImage.md - - EmptyLatentImage: md/Comfy/EmptyLatentImage.md - - ExponentialScheduler: md/Comfy/ExponentialScheduler.md - - FeatherMask: md/Comfy/FeatherMask.md - - FlipSigmas: md/Comfy/FlipSigmas.md - - FreeU: md/Comfy/FreeU.md - - FreeU_V2: md/Comfy/FreeU_V2.md - - GLIGENLoader: md/Comfy/GLIGENLoader.md - - GLIGENTextBoxApply: md/Comfy/GLIGENTextBoxApply.md - - GrowMask: md/Comfy/GrowMask.md - - HyperTile: md/Comfy/HyperTile.md - - HypernetworkLoader: md/Comfy/HypernetworkLoader.md - - ImageBatch: md/Comfy/ImageBatch.md - - ImageBlend: md/Comfy/ImageBlend.md - - ImageBlur: md/Comfy/ImageBlur.md - - ImageColorToMask: md/Comfy/ImageColorToMask.md - - ImageCompositeMasked: md/Comfy/ImageCompositeMasked.md - - ImageCrop: md/Comfy/ImageCrop.md - - ImageFromBatch: md/Comfy/ImageFromBatch.md - - ImageInvert: md/Comfy/ImageInvert.md - - ImageOnlyCheckpointLoader: md/Comfy/ImageOnlyCheckpointLoader.md - - ImageOnlyCheckpointSave: md/Comfy/ImageOnlyCheckpointSave.md - - ImagePadForOutpaint: md/Comfy/ImagePadForOutpaint.md - - ImageQuantize: md/Comfy/ImageQuantize.md - - ImageScale: md/Comfy/ImageScale.md - - ImageScaleBy: md/Comfy/ImageScaleBy.md - - ImageScaleToTotalPixels: md/Comfy/ImageScaleToTotalPixels.md - - ImageSharpen: md/Comfy/ImageSharpen.md - - ImageToMask: md/Comfy/ImageToMask.md - - ImageUpscaleWithModel: md/Comfy/ImageUpscaleWithModel.md - - InpaintModelConditioning: md/Comfy/InpaintModelConditioning.md - - InvertMask: md/Comfy/InvertMask.md - - JoinImageWithAlpha: md/Comfy/JoinImageWithAlpha.md - - KSampler: md/Comfy/KSampler.md - - KSamplerAdvanced: md/Comfy/KSamplerAdvanced.md - - KSamplerSelect: md/Comfy/KSamplerSelect.md - - KarrasScheduler: md/Comfy/KarrasScheduler.md - - LatentAdd: md/Comfy/LatentAdd.md - - LatentBatch: md/Comfy/LatentBatch.md - - LatentBatchSeedBehavior: md/Comfy/LatentBatchSeedBehavior.md - - LatentBlend: md/Comfy/LatentBlend.md - - LatentComposite: md/Comfy/LatentComposite.md - - LatentCompositeMasked: md/Comfy/LatentCompositeMasked.md - - LatentCrop: md/Comfy/LatentCrop.md - - LatentFlip: md/Comfy/LatentFlip.md - - LatentFromBatch: md/Comfy/LatentFromBatch.md - - LatentInterpolate: md/Comfy/LatentInterpolate.md - - LatentMultiply: md/Comfy/LatentMultiply.md - - LatentRotate: md/Comfy/LatentRotate.md - - LatentSubtract: md/Comfy/LatentSubtract.md - - LatentUpscale: md/Comfy/LatentUpscale.md - - LatentUpscaleBy: md/Comfy/LatentUpscaleBy.md - - LoadImage: md/Comfy/LoadImage.md - - LoadImageMask: md/Comfy/LoadImageMask.md - - LoadLatent: md/Comfy/LoadLatent.md - - LoraLoader: md/Comfy/LoraLoader.md - - LoraLoaderModelOnly: md/Comfy/LoraLoaderModelOnly.md - - MaskComposite: md/Comfy/MaskComposite.md - - MaskToImage: md/Comfy/MaskToImage.md - - ModelMergeAdd: md/Comfy/ModelMergeAdd.md - - ModelMergeBlocks: md/Comfy/ModelMergeBlocks.md - - ModelMergeSimple: md/Comfy/ModelMergeSimple.md - - ModelMergeSubtract: md/Comfy/ModelMergeSubtract.md - - ModelSamplingContinuousEDM: md/Comfy/ModelSamplingContinuousEDM.md - - ModelSamplingDiscrete: md/Comfy/ModelSamplingDiscrete.md - - ModelSamplingStableCascade: md/Comfy/ModelSamplingStableCascade.md - - PatchModelAddDownscale: md/Comfy/PatchModelAddDownscale.md - - PerpNeg: md/Comfy/PerpNeg.md - - PhotoMakerEncode: md/Comfy/PhotoMakerEncode.md - - PhotoMakerLoader: md/Comfy/PhotoMakerLoader.md - - PolyexponentialScheduler: md/Comfy/PolyexponentialScheduler.md - - PorterDuffImageComposite: md/Comfy/PorterDuffImageComposite.md - - PreviewImage: md/Comfy/PreviewImage.md - - RebatchImages: md/Comfy/RebatchImages.md - - RebatchLatents: md/Comfy/RebatchLatents.md - - RepeatImageBatch: md/Comfy/RepeatImageBatch.md - - RepeatLatentBatch: md/Comfy/RepeatLatentBatch.md - - RescaleCFG: md/Comfy/RescaleCFG.md - - SDTurboScheduler: md/Comfy/SDTurboScheduler.md - - SD_4XUpscale_Conditioning: md/Comfy/SD_4XUpscale_Conditioning.md - - SVD_img2vid_Conditioning: md/Comfy/SVD_img2vid_Conditioning.md - - SamplerCustom: md/Comfy/SamplerCustom.md - - SamplerDPMPP_2M_SDE: md/Comfy/SamplerDPMPP_2M_SDE.md - - SamplerDPMPP_SDE: md/Comfy/SamplerDPMPP_SDE.md - - SaveAnimatedPNG: md/Comfy/SaveAnimatedPNG.md - - SaveAnimatedWEBP: md/Comfy/SaveAnimatedWEBP.md - - SaveImage: md/Comfy/SaveImage.md - - SaveLatent: md/Comfy/SaveLatent.md - - SelfAttentionGuidance: md/Comfy/SelfAttentionGuidance.md - - SetLatentNoiseMask: md/Comfy/SetLatentNoiseMask.md - - SolidMask: md/Comfy/SolidMask.md - - SplitImageWithAlpha: md/Comfy/SplitImageWithAlpha.md - - SplitSigmas: md/Comfy/SplitSigmas.md - - StableCascade_EmptyLatentImage: md/Comfy/StableCascade_EmptyLatentImage.md - - StableCascade_StageB_Conditioning: md/Comfy/StableCascade_StageB_Conditioning.md - - StableCascade_StageC_VAEEncode: md/Comfy/StableCascade_StageC_VAEEncode.md - - StableZero123_Conditioning: md/Comfy/StableZero123_Conditioning.md - - StableZero123_Conditioning_Batched: md/Comfy/StableZero123_Conditioning_Batched.md - - StyleModelApply: md/Comfy/StyleModelApply.md - - StyleModelLoader: md/Comfy/StyleModelLoader.md - - TomePatchModel: md/Comfy/TomePatchModel.md - - UNETLoader: md/Comfy/UNETLoader.md - - UpscaleModelLoader: md/Comfy/UpscaleModelLoader.md - - VAEDecode: md/Comfy/VAEDecode.md - - VAEDecodeTiled: md/Comfy/VAEDecodeTiled.md - - VAEEncode: md/Comfy/VAEEncode.md - - VAEEncodeForInpaint: md/Comfy/VAEEncodeForInpaint.md - - VAEEncodeTiled: md/Comfy/VAEEncodeTiled.md - - VAELoader: md/Comfy/VAELoader.md - - VAESave: md/Comfy/VAESave.md - - VPScheduler: md/Comfy/VPScheduler.md - - VideoLinearCFGGuidance: md/Comfy/VideoLinearCFGGuidance.md - - unCLIPCheckpointLoader: md/Comfy/unCLIPCheckpointLoader.md - - unCLIPConditioning: md/Comfy/unCLIPConditioning.md - - ComfyMath: - - CM_BoolBinaryOperation: md/ComfyMath/CM_BoolBinaryOperation.md - - CM_BoolToInt: md/ComfyMath/CM_BoolToInt.md - - CM_BoolUnaryOperation: md/ComfyMath/CM_BoolUnaryOperation.md - - CM_BreakoutVec2: md/ComfyMath/CM_BreakoutVec2.md - - CM_BreakoutVec3: md/ComfyMath/CM_BreakoutVec3.md - - CM_BreakoutVec4: md/ComfyMath/CM_BreakoutVec4.md - - CM_ComposeVec2: md/ComfyMath/CM_ComposeVec2.md - - CM_ComposeVec3: md/ComfyMath/CM_ComposeVec3.md - - CM_ComposeVec4: md/ComfyMath/CM_ComposeVec4.md - - CM_FloatBinaryCondition: md/ComfyMath/CM_FloatBinaryCondition.md - - CM_FloatBinaryOperation: md/ComfyMath/CM_FloatBinaryOperation.md - - CM_FloatToInt: md/ComfyMath/CM_FloatToInt.md - - CM_FloatToNumber: md/ComfyMath/CM_FloatToNumber.md - - CM_FloatUnaryCondition: md/ComfyMath/CM_FloatUnaryCondition.md - - CM_FloatUnaryOperation: md/ComfyMath/CM_FloatUnaryOperation.md - - CM_IntBinaryCondition: md/ComfyMath/CM_IntBinaryCondition.md - - CM_IntBinaryOperation: md/ComfyMath/CM_IntBinaryOperation.md - - CM_IntToBool: md/ComfyMath/CM_IntToBool.md - - CM_IntToFloat: md/ComfyMath/CM_IntToFloat.md - - CM_IntToNumber: md/ComfyMath/CM_IntToNumber.md - - CM_IntUnaryCondition: md/ComfyMath/CM_IntUnaryCondition.md - - CM_IntUnaryOperation: md/ComfyMath/CM_IntUnaryOperation.md - - CM_NearestSDXLResolution: md/ComfyMath/CM_NearestSDXLResolution.md - - CM_NumberBinaryCondition: md/ComfyMath/CM_NumberBinaryCondition.md - - CM_NumberBinaryOperation: md/ComfyMath/CM_NumberBinaryOperation.md - - CM_NumberToFloat: md/ComfyMath/CM_NumberToFloat.md - - CM_NumberToInt: md/ComfyMath/CM_NumberToInt.md - - CM_NumberUnaryCondition: md/ComfyMath/CM_NumberUnaryCondition.md - - CM_NumberUnaryOperation: md/ComfyMath/CM_NumberUnaryOperation.md - - CM_SDXLResolution: md/ComfyMath/CM_SDXLResolution.md - - CM_Vec2BinaryCondition: md/ComfyMath/CM_Vec2BinaryCondition.md - - CM_Vec2BinaryOperation: md/ComfyMath/CM_Vec2BinaryOperation.md - - CM_Vec2ScalarOperation: md/ComfyMath/CM_Vec2ScalarOperation.md - - CM_Vec2ToScalarBinaryOperation: md/ComfyMath/CM_Vec2ToScalarBinaryOperation.md - - CM_Vec2ToScalarUnaryOperation: md/ComfyMath/CM_Vec2ToScalarUnaryOperation.md - - CM_Vec2UnaryCondition: md/ComfyMath/CM_Vec2UnaryCondition.md - - CM_Vec2UnaryOperation: md/ComfyMath/CM_Vec2UnaryOperation.md - - CM_Vec3BinaryCondition: md/ComfyMath/CM_Vec3BinaryCondition.md - - CM_Vec3BinaryOperation: md/ComfyMath/CM_Vec3BinaryOperation.md - - CM_Vec3ScalarOperation: md/ComfyMath/CM_Vec3ScalarOperation.md - - CM_Vec3ToScalarBinaryOperation: md/ComfyMath/CM_Vec3ToScalarBinaryOperation.md - - CM_Vec3ToScalarUnaryOperation: md/ComfyMath/CM_Vec3ToScalarUnaryOperation.md - - CM_Vec3UnaryCondition: md/ComfyMath/CM_Vec3UnaryCondition.md - - CM_Vec3UnaryOperation: md/ComfyMath/CM_Vec3UnaryOperation.md - - CM_Vec4BinaryCondition: md/ComfyMath/CM_Vec4BinaryCondition.md - - CM_Vec4BinaryOperation: md/ComfyMath/CM_Vec4BinaryOperation.md - - CM_Vec4ScalarOperation: md/ComfyMath/CM_Vec4ScalarOperation.md - - CM_Vec4ToScalarBinaryOperation: md/ComfyMath/CM_Vec4ToScalarBinaryOperation.md - - CM_Vec4ToScalarUnaryOperation: md/ComfyMath/CM_Vec4ToScalarUnaryOperation.md - - CM_Vec4UnaryCondition: md/ComfyMath/CM_Vec4UnaryCondition.md - - CM_Vec4UnaryOperation: md/ComfyMath/CM_Vec4UnaryOperation.md - - ComfyUI_ADV_CLIP_emb: - - BNK_AddCLIPSDXLParams: md/ComfyUI_ADV_CLIP_emb/BNK_AddCLIPSDXLParams.md - - BNK_AddCLIPSDXLRParams: md/ComfyUI_ADV_CLIP_emb/BNK_AddCLIPSDXLRParams.md - - BNK_CLIPTextEncodeAdvanced: md/ComfyUI_ADV_CLIP_emb/BNK_CLIPTextEncodeAdvanced.md - - BNK_CLIPTextEncodeSDXLAdvanced: md/ComfyUI_ADV_CLIP_emb/BNK_CLIPTextEncodeSDXLAdvanced.md - - ComfyUI_IPAdapter_plus: - - IPAdapterApply: md/ComfyUI_IPAdapter_plus/IPAdapterApply.md - - IPAdapterApplyEncoded: md/ComfyUI_IPAdapter_plus/IPAdapterApplyEncoded.md - - IPAdapterApplyFaceID: md/ComfyUI_IPAdapter_plus/IPAdapterApplyFaceID.md - - IPAdapterBatchEmbeds: md/ComfyUI_IPAdapter_plus/IPAdapterBatchEmbeds.md - - IPAdapterEncoder: md/ComfyUI_IPAdapter_plus/IPAdapterEncoder.md - - IPAdapterLoadEmbeds: md/ComfyUI_IPAdapter_plus/IPAdapterLoadEmbeds.md - - IPAdapterModelLoader: md/ComfyUI_IPAdapter_plus/IPAdapterModelLoader.md - - IPAdapterSaveEmbeds: md/ComfyUI_IPAdapter_plus/IPAdapterSaveEmbeds.md - - InsightFaceLoader: md/ComfyUI_IPAdapter_plus/InsightFaceLoader.md - - PrepImageForClipVision: md/ComfyUI_IPAdapter_plus/PrepImageForClipVision.md - - PrepImageForInsightFace: md/ComfyUI_IPAdapter_plus/PrepImageForInsightFace.md - - SavePNG: - - PNG_Save: md/SavePNG/PNG_Save.md - - WAS_Extras: - - BLVAEEncode: md/WAS_Extras/BLVAEEncode.md - - CLIPTextEncodeList: md/WAS_Extras/CLIPTextEncodeList.md - - CLIPTextEncodeSequence2: md/WAS_Extras/CLIPTextEncodeSequence2.md - - ConditioningBlend: md/WAS_Extras/ConditioningBlend.md - - DebugInput: md/WAS_Extras/DebugInput.md - - KSamplerSeq: md/WAS_Extras/KSamplerSeq.md - - KSamplerSeq2: md/WAS_Extras/KSamplerSeq2.md - - VAEEncodeForInpaint (WAS): md/WAS_Extras/VAEEncodeForInpaint (WAS).md - - VividSharpen: md/WAS_Extras/VividSharpen.md - - comfyui_segment_anything: - - GroundingDinoModelLoader (segment anything): md/comfyui_segment_anything/GroundingDinoModelLoader - (segment anything).md - - GroundingDinoSAMSegment (segment anything): md/comfyui_segment_anything/GroundingDinoSAMSegment - (segment anything).md - - InvertMask (segment anything): md/comfyui_segment_anything/InvertMask - (segment anything).md - - IsMaskEmpty: md/comfyui_segment_anything/IsMaskEmpty.md - - SAMModelLoader (segment anything): md/comfyui_segment_anything/SAMModelLoader - (segment anything).md - - llmnodes: - - DocumentNode: md/llmnodes/DocumentNode.md - - DocumentPack: md/llmnodes/DocumentPack.md - - LoadOpenAIModel: md/llmnodes/LoadOpenAIModel.md - - plaiflowplayer_blocks: - - PlaiFlowPlayerAppFooter: md/plaiflowplayer_blocks/PlaiFlowPlayerAppFooter.md - - PlaiFlowPlayerAppInfo: md/plaiflowplayer_blocks/PlaiFlowPlayerAppInfo.md - - PlaiFlowPlayerInput: md/plaiflowplayer_blocks/PlaiFlowPlayerInput.md - - PlaiFlowPlayerModelList: md/plaiflowplayer_blocks/PlaiFlowPlayerModelList.md - - PlaiFlowPlayerOutput: md/plaiflowplayer_blocks/PlaiFlowPlayerOutput.md - - PlaiFlowPlayerSAMSplit: md/plaiflowplayer_blocks/PlaiFlowPlayerSAMSplit.md + - Nodes packs documentation: + - Home: md/index.md + - Comfy: + - Home: md/Comfy/index.md + - Nodes: + - BasicScheduler: md/Comfy/Nodes/BasicScheduler.md + - CLIPLoader: md/Comfy/Nodes/CLIPLoader.md + - CLIPMergeSimple: md/Comfy/Nodes/CLIPMergeSimple.md + - CLIPSave: md/Comfy/Nodes/CLIPSave.md + - CLIPSetLastLayer: md/Comfy/Nodes/CLIPSetLastLayer.md + - CLIPTextEncode: md/Comfy/Nodes/CLIPTextEncode.md + - CLIPTextEncodeControlnet: md/Comfy/Nodes/CLIPTextEncodeControlnet.md + - CLIPTextEncodeSDXL: md/Comfy/Nodes/CLIPTextEncodeSDXL.md + - CLIPTextEncodeSDXLRefiner: md/Comfy/Nodes/CLIPTextEncodeSDXLRefiner.md + - CLIPVisionEncode: md/Comfy/Nodes/CLIPVisionEncode.md + - CLIPVisionLoader: md/Comfy/Nodes/CLIPVisionLoader.md + - Canny: md/Comfy/Nodes/Canny.md + - CheckpointLoader: md/Comfy/Nodes/CheckpointLoader.md + - CheckpointLoaderSimple: md/Comfy/Nodes/CheckpointLoaderSimple.md + - CheckpointSave: md/Comfy/Nodes/CheckpointSave.md + - ConditioningAverage: md/Comfy/Nodes/ConditioningAverage.md + - ConditioningCombine: md/Comfy/Nodes/ConditioningCombine.md + - ConditioningConcat: md/Comfy/Nodes/ConditioningConcat.md + - ConditioningSetArea: md/Comfy/Nodes/ConditioningSetArea.md + - ConditioningSetAreaPercentage: md/Comfy/Nodes/ConditioningSetAreaPercentage.md + - ConditioningSetAreaStrength: md/Comfy/Nodes/ConditioningSetAreaStrength.md + - ConditioningSetMask: md/Comfy/Nodes/ConditioningSetMask.md + - ConditioningSetTimestepRange: md/Comfy/Nodes/ConditioningSetTimestepRange.md + - ConditioningZeroOut: md/Comfy/Nodes/ConditioningZeroOut.md + - ControlNetApply: md/Comfy/Nodes/ControlNetApply.md + - ControlNetApplyAdvanced: md/Comfy/Nodes/ControlNetApplyAdvanced.md + - ControlNetLoader: md/Comfy/Nodes/ControlNetLoader.md + - CropMask: md/Comfy/Nodes/CropMask.md + - DiffControlNetLoader: md/Comfy/Nodes/DiffControlNetLoader.md + - DiffusersLoader: md/Comfy/Nodes/DiffusersLoader.md + - DualCLIPLoader: md/Comfy/Nodes/DualCLIPLoader.md + - EmptyImage: md/Comfy/Nodes/EmptyImage.md + - EmptyLatentImage: md/Comfy/Nodes/EmptyLatentImage.md + - ExponentialScheduler: md/Comfy/Nodes/ExponentialScheduler.md + - FeatherMask: md/Comfy/Nodes/FeatherMask.md + - FlipSigmas: md/Comfy/Nodes/FlipSigmas.md + - FreeU: md/Comfy/Nodes/FreeU.md + - FreeU_V2: md/Comfy/Nodes/FreeU_V2.md + - GLIGENLoader: md/Comfy/Nodes/GLIGENLoader.md + - GLIGENTextBoxApply: md/Comfy/Nodes/GLIGENTextBoxApply.md + - GrowMask: md/Comfy/Nodes/GrowMask.md + - HyperTile: md/Comfy/Nodes/HyperTile.md + - HypernetworkLoader: md/Comfy/Nodes/HypernetworkLoader.md + - ImageBatch: md/Comfy/Nodes/ImageBatch.md + - ImageBlend: md/Comfy/Nodes/ImageBlend.md + - ImageBlur: md/Comfy/Nodes/ImageBlur.md + - ImageColorToMask: md/Comfy/Nodes/ImageColorToMask.md + - ImageCompositeMasked: md/Comfy/Nodes/ImageCompositeMasked.md + - ImageCrop: md/Comfy/Nodes/ImageCrop.md + - ImageFromBatch: md/Comfy/Nodes/ImageFromBatch.md + - ImageInvert: md/Comfy/Nodes/ImageInvert.md + - ImageOnlyCheckpointLoader: md/Comfy/Nodes/ImageOnlyCheckpointLoader.md + - ImageOnlyCheckpointSave: md/Comfy/Nodes/ImageOnlyCheckpointSave.md + - ImagePadForOutpaint: md/Comfy/Nodes/ImagePadForOutpaint.md + - ImageQuantize: md/Comfy/Nodes/ImageQuantize.md + - ImageScale: md/Comfy/Nodes/ImageScale.md + - ImageScaleBy: md/Comfy/Nodes/ImageScaleBy.md + - ImageScaleToTotalPixels: md/Comfy/Nodes/ImageScaleToTotalPixels.md + - ImageSharpen: md/Comfy/Nodes/ImageSharpen.md + - ImageToMask: md/Comfy/Nodes/ImageToMask.md + - ImageUpscaleWithModel: md/Comfy/Nodes/ImageUpscaleWithModel.md + - InpaintModelConditioning: md/Comfy/Nodes/InpaintModelConditioning.md + - InvertMask: md/Comfy/Nodes/InvertMask.md + - JoinImageWithAlpha: md/Comfy/Nodes/JoinImageWithAlpha.md + - KSampler: md/Comfy/Nodes/KSampler.md + - KSamplerAdvanced: md/Comfy/Nodes/KSamplerAdvanced.md + - KSamplerSelect: md/Comfy/Nodes/KSamplerSelect.md + - KarrasScheduler: md/Comfy/Nodes/KarrasScheduler.md + - LatentAdd: md/Comfy/Nodes/LatentAdd.md + - LatentBatch: md/Comfy/Nodes/LatentBatch.md + - LatentBatchSeedBehavior: md/Comfy/Nodes/LatentBatchSeedBehavior.md + - LatentBlend: md/Comfy/Nodes/LatentBlend.md + - LatentComposite: md/Comfy/Nodes/LatentComposite.md + - LatentCompositeMasked: md/Comfy/Nodes/LatentCompositeMasked.md + - LatentCrop: md/Comfy/Nodes/LatentCrop.md + - LatentFlip: md/Comfy/Nodes/LatentFlip.md + - LatentFromBatch: md/Comfy/Nodes/LatentFromBatch.md + - LatentInterpolate: md/Comfy/Nodes/LatentInterpolate.md + - LatentMultiply: md/Comfy/Nodes/LatentMultiply.md + - LatentRotate: md/Comfy/Nodes/LatentRotate.md + - LatentSubtract: md/Comfy/Nodes/LatentSubtract.md + - LatentUpscale: md/Comfy/Nodes/LatentUpscale.md + - LatentUpscaleBy: md/Comfy/Nodes/LatentUpscaleBy.md + - LoadImage: md/Comfy/Nodes/LoadImage.md + - LoadImageMask: md/Comfy/Nodes/LoadImageMask.md + - LoadLatent: md/Comfy/Nodes/LoadLatent.md + - LoraLoader: md/Comfy/Nodes/LoraLoader.md + - LoraLoaderModelOnly: md/Comfy/Nodes/LoraLoaderModelOnly.md + - MaskComposite: md/Comfy/Nodes/MaskComposite.md + - MaskToImage: md/Comfy/Nodes/MaskToImage.md + - ModelMergeAdd: md/Comfy/Nodes/ModelMergeAdd.md + - ModelMergeBlocks: md/Comfy/Nodes/ModelMergeBlocks.md + - ModelMergeSimple: md/Comfy/Nodes/ModelMergeSimple.md + - ModelMergeSubtract: md/Comfy/Nodes/ModelMergeSubtract.md + - ModelSamplingContinuousEDM: md/Comfy/Nodes/ModelSamplingContinuousEDM.md + - ModelSamplingDiscrete: md/Comfy/Nodes/ModelSamplingDiscrete.md + - ModelSamplingStableCascade: md/Comfy/Nodes/ModelSamplingStableCascade.md + - PatchModelAddDownscale: md/Comfy/Nodes/PatchModelAddDownscale.md + - PerpNeg: md/Comfy/Nodes/PerpNeg.md + - PhotoMakerEncode: md/Comfy/Nodes/PhotoMakerEncode.md + - PhotoMakerLoader: md/Comfy/Nodes/PhotoMakerLoader.md + - PolyexponentialScheduler: md/Comfy/Nodes/PolyexponentialScheduler.md + - PorterDuffImageComposite: md/Comfy/Nodes/PorterDuffImageComposite.md + - PreviewImage: md/Comfy/Nodes/PreviewImage.md + - RebatchImages: md/Comfy/Nodes/RebatchImages.md + - RebatchLatents: md/Comfy/Nodes/RebatchLatents.md + - RepeatImageBatch: md/Comfy/Nodes/RepeatImageBatch.md + - RepeatLatentBatch: md/Comfy/Nodes/RepeatLatentBatch.md + - RescaleCFG: md/Comfy/Nodes/RescaleCFG.md + - SDTurboScheduler: md/Comfy/Nodes/SDTurboScheduler.md + - SD_4XUpscale_Conditioning: md/Comfy/Nodes/SD_4XUpscale_Conditioning.md + - SVD_img2vid_Conditioning: md/Comfy/Nodes/SVD_img2vid_Conditioning.md + - SamplerCustom: md/Comfy/Nodes/SamplerCustom.md + - SamplerDPMPP_2M_SDE: md/Comfy/Nodes/SamplerDPMPP_2M_SDE.md + - SamplerDPMPP_SDE: md/Comfy/Nodes/SamplerDPMPP_SDE.md + - SaveAnimatedPNG: md/Comfy/Nodes/SaveAnimatedPNG.md + - SaveAnimatedWEBP: md/Comfy/Nodes/SaveAnimatedWEBP.md + - SaveImage: md/Comfy/Nodes/SaveImage.md + - SaveLatent: md/Comfy/Nodes/SaveLatent.md + - SelfAttentionGuidance: md/Comfy/Nodes/SelfAttentionGuidance.md + - SetLatentNoiseMask: md/Comfy/Nodes/SetLatentNoiseMask.md + - SolidMask: md/Comfy/Nodes/SolidMask.md + - SplitImageWithAlpha: md/Comfy/Nodes/SplitImageWithAlpha.md + - SplitSigmas: md/Comfy/Nodes/SplitSigmas.md + - StableCascade_EmptyLatentImage: md/Comfy/Nodes/StableCascade_EmptyLatentImage.md + - StableCascade_StageB_Conditioning: md/Comfy/Nodes/StableCascade_StageB_Conditioning.md + - StableCascade_StageC_VAEEncode: md/Comfy/Nodes/StableCascade_StageC_VAEEncode.md + - StableZero123_Conditioning: md/Comfy/Nodes/StableZero123_Conditioning.md + - StableZero123_Conditioning_Batched: md/Comfy/Nodes/StableZero123_Conditioning_Batched.md + - StyleModelApply: md/Comfy/Nodes/StyleModelApply.md + - StyleModelLoader: md/Comfy/Nodes/StyleModelLoader.md + - TomePatchModel: md/Comfy/Nodes/TomePatchModel.md + - UNETLoader: md/Comfy/Nodes/UNETLoader.md + - UpscaleModelLoader: md/Comfy/Nodes/UpscaleModelLoader.md + - VAEDecode: md/Comfy/Nodes/VAEDecode.md + - VAEDecodeTiled: md/Comfy/Nodes/VAEDecodeTiled.md + - VAEEncode: md/Comfy/Nodes/VAEEncode.md + - VAEEncodeForInpaint: md/Comfy/Nodes/VAEEncodeForInpaint.md + - VAEEncodeTiled: md/Comfy/Nodes/VAEEncodeTiled.md + - VAELoader: md/Comfy/Nodes/VAELoader.md + - VAESave: md/Comfy/Nodes/VAESave.md + - VPScheduler: md/Comfy/Nodes/VPScheduler.md + - VideoLinearCFGGuidance: md/Comfy/Nodes/VideoLinearCFGGuidance.md + - unCLIPCheckpointLoader: md/Comfy/Nodes/unCLIPCheckpointLoader.md + - unCLIPConditioning: md/Comfy/Nodes/unCLIPConditioning.md + - ComfyMath: + - Home: md/ComfyMath/index.md + - Nodes: + - CM_BoolBinaryOperation: md/ComfyMath/Nodes/CM_BoolBinaryOperation.md + - CM_BoolToInt: md/ComfyMath/Nodes/CM_BoolToInt.md + - CM_BoolUnaryOperation: md/ComfyMath/Nodes/CM_BoolUnaryOperation.md + - CM_BreakoutVec2: md/ComfyMath/Nodes/CM_BreakoutVec2.md + - CM_BreakoutVec3: md/ComfyMath/Nodes/CM_BreakoutVec3.md + - CM_BreakoutVec4: md/ComfyMath/Nodes/CM_BreakoutVec4.md + - CM_ComposeVec2: md/ComfyMath/Nodes/CM_ComposeVec2.md + - CM_ComposeVec3: md/ComfyMath/Nodes/CM_ComposeVec3.md + - CM_ComposeVec4: md/ComfyMath/Nodes/CM_ComposeVec4.md + - CM_FloatBinaryCondition: md/ComfyMath/Nodes/CM_FloatBinaryCondition.md + - CM_FloatBinaryOperation: md/ComfyMath/Nodes/CM_FloatBinaryOperation.md + - CM_FloatToInt: md/ComfyMath/Nodes/CM_FloatToInt.md + - CM_FloatToNumber: md/ComfyMath/Nodes/CM_FloatToNumber.md + - CM_FloatUnaryCondition: md/ComfyMath/Nodes/CM_FloatUnaryCondition.md + - CM_FloatUnaryOperation: md/ComfyMath/Nodes/CM_FloatUnaryOperation.md + - CM_IntBinaryCondition: md/ComfyMath/Nodes/CM_IntBinaryCondition.md + - CM_IntBinaryOperation: md/ComfyMath/Nodes/CM_IntBinaryOperation.md + - CM_IntToBool: md/ComfyMath/Nodes/CM_IntToBool.md + - CM_IntToFloat: md/ComfyMath/Nodes/CM_IntToFloat.md + - CM_IntToNumber: md/ComfyMath/Nodes/CM_IntToNumber.md + - CM_IntUnaryCondition: md/ComfyMath/Nodes/CM_IntUnaryCondition.md + - CM_IntUnaryOperation: md/ComfyMath/Nodes/CM_IntUnaryOperation.md + - CM_NearestSDXLResolution: md/ComfyMath/Nodes/CM_NearestSDXLResolution.md + - CM_NumberBinaryCondition: md/ComfyMath/Nodes/CM_NumberBinaryCondition.md + - CM_NumberBinaryOperation: md/ComfyMath/Nodes/CM_NumberBinaryOperation.md + - CM_NumberToFloat: md/ComfyMath/Nodes/CM_NumberToFloat.md + - CM_NumberToInt: md/ComfyMath/Nodes/CM_NumberToInt.md + - CM_NumberUnaryCondition: md/ComfyMath/Nodes/CM_NumberUnaryCondition.md + - CM_NumberUnaryOperation: md/ComfyMath/Nodes/CM_NumberUnaryOperation.md + - CM_SDXLResolution: md/ComfyMath/Nodes/CM_SDXLResolution.md + - CM_Vec2BinaryCondition: md/ComfyMath/Nodes/CM_Vec2BinaryCondition.md + - CM_Vec2BinaryOperation: md/ComfyMath/Nodes/CM_Vec2BinaryOperation.md + - CM_Vec2ScalarOperation: md/ComfyMath/Nodes/CM_Vec2ScalarOperation.md + - CM_Vec2ToScalarBinaryOperation: md/ComfyMath/Nodes/CM_Vec2ToScalarBinaryOperation.md + - CM_Vec2ToScalarUnaryOperation: md/ComfyMath/Nodes/CM_Vec2ToScalarUnaryOperation.md + - CM_Vec2UnaryCondition: md/ComfyMath/Nodes/CM_Vec2UnaryCondition.md + - CM_Vec2UnaryOperation: md/ComfyMath/Nodes/CM_Vec2UnaryOperation.md + - CM_Vec3BinaryCondition: md/ComfyMath/Nodes/CM_Vec3BinaryCondition.md + - CM_Vec3BinaryOperation: md/ComfyMath/Nodes/CM_Vec3BinaryOperation.md + - CM_Vec3ScalarOperation: md/ComfyMath/Nodes/CM_Vec3ScalarOperation.md + - CM_Vec3ToScalarBinaryOperation: md/ComfyMath/Nodes/CM_Vec3ToScalarBinaryOperation.md + - CM_Vec3ToScalarUnaryOperation: md/ComfyMath/Nodes/CM_Vec3ToScalarUnaryOperation.md + - CM_Vec3UnaryCondition: md/ComfyMath/Nodes/CM_Vec3UnaryCondition.md + - CM_Vec3UnaryOperation: md/ComfyMath/Nodes/CM_Vec3UnaryOperation.md + - CM_Vec4BinaryCondition: md/ComfyMath/Nodes/CM_Vec4BinaryCondition.md + - CM_Vec4BinaryOperation: md/ComfyMath/Nodes/CM_Vec4BinaryOperation.md + - CM_Vec4ScalarOperation: md/ComfyMath/Nodes/CM_Vec4ScalarOperation.md + - CM_Vec4ToScalarBinaryOperation: md/ComfyMath/Nodes/CM_Vec4ToScalarBinaryOperation.md + - CM_Vec4ToScalarUnaryOperation: md/ComfyMath/Nodes/CM_Vec4ToScalarUnaryOperation.md + - CM_Vec4UnaryCondition: md/ComfyMath/Nodes/CM_Vec4UnaryCondition.md + - CM_Vec4UnaryOperation: md/ComfyMath/Nodes/CM_Vec4UnaryOperation.md + - ComfyUI_ADV_CLIP_emb: + - Home: md/ComfyUI_ADV_CLIP_emb/index.md + - Nodes: + - BNK_AddCLIPSDXLParams: md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLParams.md + - BNK_AddCLIPSDXLRParams: md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_AddCLIPSDXLRParams.md + - BNK_CLIPTextEncodeAdvanced: md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeAdvanced.md + - BNK_CLIPTextEncodeSDXLAdvanced: md/ComfyUI_ADV_CLIP_emb/Nodes/BNK_CLIPTextEncodeSDXLAdvanced.md + - ComfyUI_IPAdapter_plus: + - Home: md/ComfyUI_IPAdapter_plus/index.md + - Nodes: + - IPAdapterApply: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApply.md + - IPAdapterApplyEncoded: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyEncoded.md + - IPAdapterApplyFaceID: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterApplyFaceID.md + - IPAdapterBatchEmbeds: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterBatchEmbeds.md + - IPAdapterEncoder: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterEncoder.md + - IPAdapterLoadEmbeds: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterLoadEmbeds.md + - IPAdapterModelLoader: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterModelLoader.md + - IPAdapterSaveEmbeds: md/ComfyUI_IPAdapter_plus/Nodes/IPAdapterSaveEmbeds.md + - InsightFaceLoader: md/ComfyUI_IPAdapter_plus/Nodes/InsightFaceLoader.md + - PrepImageForClipVision: md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForClipVision.md + - PrepImageForInsightFace: md/ComfyUI_IPAdapter_plus/Nodes/PrepImageForInsightFace.md + - SavePNG: + - Home: md/SavePNG/index.md + - Nodes: + - PNG_Save: md/SavePNG/Nodes/PNG_Save.md + - WAS_Extras: + - Home: md/WAS_Extras/index.md + - Nodes: + - BLVAEEncode: md/WAS_Extras/Nodes/BLVAEEncode.md + - CLIPTextEncodeList: md/WAS_Extras/Nodes/CLIPTextEncodeList.md + - CLIPTextEncodeSequence2: md/WAS_Extras/Nodes/CLIPTextEncodeSequence2.md + - ConditioningBlend: md/WAS_Extras/Nodes/ConditioningBlend.md + - DebugInput: md/WAS_Extras/Nodes/DebugInput.md + - KSamplerSeq: md/WAS_Extras/Nodes/KSamplerSeq.md + - KSamplerSeq2: md/WAS_Extras/Nodes/KSamplerSeq2.md + - VAEEncodeForInpaint (WAS): md/WAS_Extras/Nodes/VAEEncodeForInpaint + (WAS).md + - VividSharpen: md/WAS_Extras/Nodes/VividSharpen.md + - comfyui_segment_anything: + - Home: md/comfyui_segment_anything/index.md + - Nodes: + - GroundingDinoModelLoader (segment anything): md/comfyui_segment_anything/Nodes/GroundingDinoModelLoader + (segment anything).md + - GroundingDinoSAMSegment (segment anything): md/comfyui_segment_anything/Nodes/GroundingDinoSAMSegment + (segment anything).md + - InvertMask (segment anything): md/comfyui_segment_anything/Nodes/InvertMask + (segment anything).md + - IsMaskEmpty: md/comfyui_segment_anything/Nodes/IsMaskEmpty.md + - SAMModelLoader (segment anything): md/comfyui_segment_anything/Nodes/SAMModelLoader + (segment anything).md + - llmnodes: + - Home: md/llmnodes/index.md + - Nodes: + - DocumentNode: md/llmnodes/Nodes/DocumentNode.md + - DocumentPack: md/llmnodes/Nodes/DocumentPack.md + - LoadOpenAIModel: md/llmnodes/Nodes/LoadOpenAIModel.md + - plaiflowplayer_blocks: + - Home: md/plaiflowplayer_blocks/index.md + - Nodes: + - PlaiFlowPlayerAppFooter: md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppFooter.md + - PlaiFlowPlayerAppInfo: md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerAppInfo.md + - PlaiFlowPlayerInput: md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerInput.md + - PlaiFlowPlayerModelList: md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerModelList.md + - PlaiFlowPlayerOutput: md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerOutput.md + - PlaiFlowPlayerSAMSplit: md/plaiflowplayer_blocks/Nodes/PlaiFlowPlayerSAMSplit.md theme: name: material palette: