Thanks for the repo! Some questions about GameStatusClassifier and Player Skeletons

[arashout]

Hey! Thanks for creating and open-sourcing the repo, it's fun to play with!

I want to build an automated tool for creating clips of our volleyball games to trim down our 1+ hour video down to 20 minutes of just actual gameplay.

I'm having trouble getting the GameStatusClassifier to reliably recognize anything (probably because the camera angles and footage from our recreational games are much worse)

• What made you decide to use the VideoMAE model for detecting game state? I'm trying to read up on it but couldn't find anything relevant to sports analytics outside of https://arxiv.org/abs/2203.12602
• Also if I want to train / finetune the model, is it enough to have clips have "playing" vs "not playing"? e.g. 50 x 10 secs of both?

Also I noticed that we have the player skeleton data, but we don't use it for Action Recognition. What did you plan to do with the Skeleton data? Do you think it would be helpful if I trained a model for Action Detection?

Anyway thanks again for creating the repo, I'm not a machine learning engineer but having fun messing around with it

[masouduut94]

Hi Arash.

I'm a big fan of volleyball just like you. To develop this project, it took me 10 months to gather dataset from games online and also on my hometown games. It was completely a hobby project.

I used to work on a sports analytics company in which we used to work on various sports like cricket, table tennis, basketball and etc ...

I used many approaches and many type of features to find out actions inside the video.

For example to detect a person that enters in a video and throwing a ball, I used optical flow to detect it. But this method couldn't filter other background actions like other players moving far from the scene.
Sklethon-based approaches are not good for this task. Because differentiating between active play and non-active play and also service play is not easy to be done by such models.

The 3d models (models that get multiple frames as input) are good for complex tasks like action detection or video classification and also suitable for realtime inference.

I used 3d models in this project because I found 1 second by 1 second video classification quite reasonable and fast.

The only thing that was not easy was gathering the dataset and training I used over 10 to 12 videos of volleyball games on youtube.com channel named as volleyball watchdogs to annotate. After annotation, I used several video augmentations to add more data. The service was the category that I had the least dataset.

I recommend you to use llms available for automated annotation.

I hope you could retrain the videoMAE model. By the way new version is even better videoMAE2.

[arashout]

Hey Masoud, cool to hear from you!

Yes I'm going to re-train the videoMAE model. I didn't know about the V2 version I'll take a look!
But I'm not a machine learning so some of the discussion around hyperparameters is making my head spin haha 😕

I actually have a decent method, I purchased a subscription to Balltime (a sports analytics company that does this sort of thing) and I'm using their editted footage to build my own database of clips.

It's still a little bit of a pain because they don't make it easy for you to export that data (obviously they don't want you to train your own model on theirs, which is fair)

Sklethon-based approaches are not good for this task. Because differentiating between active play and non-active play and also service play is not easy to be done by such models.

This is a good point, I hadn't thought about the other players interrupting. I was thinking of running one model for PLAY/NO PLAY and then using pose detections with skeletons for actions.

I recommend you to use llms available for automated annotation.

Do you mean taking a frame where there is an Action happening, like a spike and sending that to an LLM to annotate?
That's an interesting take, I guess it would require having a decent model and then iterating on it?

I used several video augmentations to add more data.

Do you mean playing around with flipping / compression / scaling?

[MuhammadHammadBashir]

Hi Arashout, I am doing the same kind of system for amateur matches, can you give me your expert thoughts on a few points.

1. Did the game state classification work on your matches?
2. you managed to find action classification because in amaterus matches with low quality camera some players pose are not detected by yolo pose and mediapipe etc
3. Have you worked around bounce and score calculation

[arashout]

1. It's working okay I decided to just take the VideoMAE model and try to train that for now. And I reduced it to just 2 states, Play and no play since it's easier to collect data.

• It's not that good yet, I need to collect more data (I'm lazy haha)

2. Yeah I initially was trying to use the pose estimation and stuff, but I feel like it's not worth it.

3. No

@MuhammadHammadBashir