Tuning as in the reproducibility section

[sorgfresser]

Hey, since I did not find any further information on this so far: how exactly did you tune the hyperparams stated in the reproducibility section? I'm currently messing around with diart.tune for AMI with the new pyannote models but would like a fair comparison against the old DER values.

I will post any questions coming up regarding the tuning process here to have one concise place for them.
As of now:
1.) Did you use the whole dataset or just the dev set to tune?
2.) I used the AMI dataset by pyannote/AMI-diarization-setup. At least for me this resulted in IB4005.wav lacking the matching rttm. Where did you get AMI from?

[juanmc2005]

Hi @sorgfresser

1.) Did you use the whole dataset or just the dev set to tune?

I used only the dev split for tuning. The performance reported in the paper (and README) is on test

2.) I used the AMI dataset by pyannote/AMI-diarization-setup. At least for me this resulted in IB4005.wav lacking the matching rttm. Where did you get AMI from?

I got AMI from the same repo. It also resulted in that file being out of any split.