Request for fine-grained prompts for CLIP-IQA

[mansur007]

Is your feature request related to a problem? Please describe.
Currently, CLIP-IQA uses ["Good photo.", "Bad photo."] as anchors.
However, a more fine-grained assessment should be possible according to the paper.
If you refer to the table 3 of the paper, you will see pairs like:
["Clean photo.", "Noisy photo."], ["Sharp photo.", "Blurry photo."], etc.
Would it be possible for such prompts to be supported?

Describe the solution you'd like
In the clip_iqa.py file, pre-computed tokens are downloaded for "Good photo." and "Bad photo.".
Would it be possible to add pre-computed tokens for other prompts as well?
Or, can you suggest a quick way for us to generate those tokens by ourselves?

Thank you!

[snk4tr]

Hello @mansur007!

Indeed, the initial paper mentioned several ways to apply prompt engineering to tune the metric. One way would be to use alternative prompts that you just mentioned, the other way would be to combine (ensemble) several prompts to cover broader scope and potentially improve generalizability.

While those modifications are possible, the choice of a pair of prompts is part of the metric design and firmly fixed. To confirm that, please check tables with comparisons of CLIP-IQA with other metrics, in all of them ["Good photo.", "Bad photo."] are used as anchors. Hence, we fixed the pair of prompts for users' convenience so their results reproduce the actual CLIP-IQA metric.

However, your proposition makes sense. Potentially, a user may want to use alternative pairs of prompts. We will support this functionality ASAP. For the moment, there is no elegant solution to your problem. I can only recommend you to clone the repo and modify prompts in the clipiqa.py file.

[mansur007]

@snk4tr Thank you for the info!
I don't know how to generate tokens for other prompts. Would you be able to guide me/point me to some resources I could use to generate tokens for custom prompts?
Thank you