Export `:tensorflow:serving:...` metrics by signature names

[jeongukjae]

Feature Request

If this is a feature request, please fill out the following form in full:

Describe the problem the feature is intended to solve

For now, tensorflow serving exports metrics by model like below.

...
:tensorflow:serving:request_count{model_name="test_model",status="OK"} 6
...
:tensorflow:serving:request_latency_bucket{model_name="test_model",API="predict",entrypoint="REST",le="10"} 0
:tensorflow:serving:request_latency_bucket{model_name="test_model",API="predict",entrypoint="REST",le="18"} 0
...
:tensorflow:serving:runtime_latency_bucket{model_name="test_model",API="Predict",runtime="TF1",le="10"} 0
:tensorflow:serving:runtime_latency_bucket{model_name="test_model",API="Predict",runtime="TF1",le="18"} 0
:tensorflow:serving:runtime_latency_bucket{model_name="test_model",API="Predict",runtime="TF1",le="32.4"} 0
...

We cannot collect metrics by signatures, even if the latencies of each signature are very different.

Related codes:

• serving/tensorflow_serving/servables/tensorflow/util.h

  Lines 118 to 119 in 21360c7

  	void RecordRuntimeLatency(const string& model_name, const string& api,
  	const string& runtime, int64_t latency_usec);

• serving/tensorflow_serving/servables/tensorflow/util.h

  Lines 122 to 123 in 21360c7

  	void RecordRequestLatency(const string& model_name, const string& api,
  	const string& entrypoint, int64_t latency_usec);

Describe the solution

It must be better if runtime latency and request latency are recorded with signature names.

Describe alternatives you've considered

Additional context

[singhniraj08]

@jeongukjae,

Are you still looking for a resolution? We are planning on prioritising the issues based on the community interests. Please let us know if this issue still persists with the latest TF Serving 1.12.1 release so that we can work on fixing it. Thank you for your contributions.

[jeongukjae]

@singhniraj08 I wrote a PR for this issue #2152
I think those patches are enough for this. Can you review that?

[singhniraj08]

@jeongukjae, Thank you for your contributions. We will discuss this internally and update this thread. Thanks