Skip to content

feat: add MLX distributed E2E test for MPI workload - #3933

Open
Rajkaran-122 wants to merge 1 commit into
kubeflow:masterfrom
Rajkaran-122:feat/e2e-mlx-mpi-test
Open

feat: add MLX distributed E2E test for MPI workload#3933
Rajkaran-122 wants to merge 1 commit into
kubeflow:masterfrom
Rajkaran-122:feat/e2e-mlx-mpi-test

Conversation

@Rajkaran-122

@Rajkaran-122 Rajkaran-122 commented Aug 17, 2026

Copy link
Copy Markdown

This commit adds comprehensive E2E testing for the MLX distributed runtime to complement the existing DeepSpeed MPI test. The MLX runtime uses OpenMPI for distributed training and requires verification of the launcher/node dependency relationship.

Changes:

  • Add mlxRuntime constant to E2E test suite
  • Add MLX workload test case with multi-node MPI configuration
  • Verify launcher dependsOn node status (Ready) for proper MPI bootstrap
  • Test validates both launcher and node job completion status

This ensures the MLX distributed runtime works correctly with the MPI plugin and JobSet dependsOn API, providing parity with DeepSpeed runtime testing.

Co-Authored-By: yadavrajkaran854@gmail.com

Fixes #3174

@github-actions

Copy link
Copy Markdown
Contributor

🎉 Welcome to the Kubeflow Trainer! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

  • If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards.
  • Our team will review your PR soon! cc @kubeflow/kubeflow-trainer-team

Join the community:

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏

@google-oss-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tenzen-y for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Rajkaran-122 Rajkaran-122 changed the title test(e2e): Add MLX distributed E2E test for MPI workload test(e2e): add MLX distributed E2E test for MPI workload Aug 17, 2026
@Rajkaran-122 Rajkaran-122 changed the title test(e2e): add MLX distributed E2E test for MPI workload feat: add MLX distributed E2E test for MPI workload Aug 17, 2026
This commit adds comprehensive E2E testing for the MLX distributed runtime
to complement the existing DeepSpeed MPI test. The MLX runtime uses OpenMPI
for distributed training and requires verification of the launcher/node
dependency relationship.

Changes:
- Add mlxRuntime constant to E2E test suite
- Add MLX workload test case with multi-node MPI configuration
- Verify launcher dependsOn node status (Ready) for proper MPI bootstrap
- Test validates both launcher and node job completion status

This ensures the MLX distributed runtime works correctly with the MPI plugin
and JobSet dependsOn API, providing parity with DeepSpeed runtime testing.

Co-Authored-By: yadavrajkaran854@gmail.com
Signed-off-by: Rajkaran Yadav <yadavrajkaran854@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Needs Triage

Development

Successfully merging this pull request may close these issues.

Support local iceberg table with parquet files to local validation and e2es

2 participants