Skip to content

[Feat] support JoyAIImagePipeline #13437

Open
RuixiangMa wants to merge 4 commits intohuggingface:mainfrom
RuixiangMa:joyaiimage
Open

[Feat] support JoyAIImagePipeline #13437
RuixiangMa wants to merge 4 commits intohuggingface:mainfrom
RuixiangMa:joyaiimage

Conversation

@RuixiangMa
Copy link
Copy Markdown

@RuixiangMa RuixiangMa commented Apr 9, 2026

Support JoyAIImagePipeline

Prompt image
NO dog
add hat in the dog joyai_image_seqoffload_out
Rotate the dog to show the left side view left
Rotate the dog to show the right side view right

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Signed-off-by: Lancer <maruixiang6688@gmail.com>
@github-actions github-actions bot added models pipelines size/L PR with diff > 200 LOC labels Apr 9, 2026
@RuixiangMa RuixiangMa marked this pull request as draft April 9, 2026 11:42
Signed-off-by: Lancer <maruixiang6688@gmail.com>
@github-actions github-actions bot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 9, 2026
Signed-off-by: Lancer <maruixiang6688@gmail.com>
@github-actions github-actions bot added schedulers size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 9, 2026
Signed-off-by: Lancer <maruixiang6688@gmail.com>
@github-actions github-actions bot added tests size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 9, 2026
@RuixiangMa RuixiangMa marked this pull request as ready for review April 9, 2026 18:20
Copy link
Copy Markdown
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR, i left some initial comments

from .autoencoder_kl_wan import (
WanAttentionBlock as AttentionBlock,
)
from .autoencoder_kl_wan import (
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you not import from wan? let's use #Copied from isntead

self._joyai_force_vae_fp32 = True

@classmethod
def from_pretrained(cls, pretrained_model_name_or_path, **kwargs):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's going on here? is there any reason we cannot make eit work with DiffusionPipeline.from_pretrained?

prev_sample: torch.FloatTensor


class JoyAIFlowMatchDiscreteScheduler(SchedulerMixin, ConfigMixin):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@claude
can you tell me what's the difference between this and FlowMatchDiscreteSchedule?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants