Upload video to get the motion

[LinMa910]

Hello，I want to use my own pkl or upload a video to get the motion, but when I use the video_demo and upload my video I only get the first frame to suit, So what can I do to make this correct? Thanks for your time!

[LinMa910]

Hello, I’m trying to use PHC with my own motion extracted from a video.

Problem 1 – Only first frame motion available
When I run my motion tracking pipeline on a video, I only get the first frame’s pose, so in PHC the avatar just stands or plays the default get-up motion. I would like to know:

How to process my video to get full, temporally consistent SMPL motion so PHC can replay it?

Are there recommended tools (HMR4D, VIBE, PHALP, etc.) and scripts to export a PHC-compatible .pkl covering all frames?

Problem 2 – Do I need to train PHC for my motion?
I have a .pkl file converted into PHC’s format (pose_aa, trans, etc.).
Question:

Can I directly load this into env.motion_file with an existing PHC model, or must I train a new controller for it?

If training is not needed, what preprocessing or scaling should I do to ensure compatibility?

Environment:

PHC commit: [commit hash]

Python version: [version]

Motion extractor: [name, e.g., VIBE/HMR4D]

Video length: [seconds]

What I’ve tried:

Converting my motion to .pkl via my own script

Running env=env_im_getup_mcp with env.motion_file=...

The avatar still plays default motion if loading fails

Expected behavior:

PHC loads the .pkl motion and plays it smoothly for all frames from my video.

Thanks for any guidance!

[ZhengyiLuo]

Hi! First frame avaible -- how long is your motion? The video_dmeo is for streaming from a webcam

[LinMa910]

Hi! First frame avaible -- how long is your motion? The video_dmeo is for streaming from a webcam

Yes, and I try to revise the code to upload a video, my video is almost 30s, and I using the pkl output from gvhmr project, which is from video to motion. Using this pkl with the code you survived to the style of PHC, but still get no matching motion with my video. It's like this:

phc.mp4

https://github.com/user-attachments/assets/7ce77477-5503-4737-b1ac-b0cf75f96757
So if I want to upload a video to get the motion, how can I do, would you please give me some advice? Thank you for your reply!