Computationally intensive quality measures, :dunn and :sillouette fail on large datasets

[chelate]

Comparing several quality indices, (thanks for these). These knn graph methods :dunn and :sillouette (which rely on graphical structures) have a hard termination and forced restart. In Pluto the error message is simply:

Malt.TerminatedWorkerException()

This is on julia 1.10.2+0.aarch64.apple.darwin14 with Clustering v"0.15.7"

I have 1.4 million datapoints, and 200 clusters. I don't know if calculating these indices on this large dataset is feasible?

[jaksle]

For both indices there are variants suitable for large datasets. It requires using different algorithms. It may not be worth it though, choosing a quality index is somewhat subjective anyway, so you may as well choose ones easier to compute.