perfzero resnet benchmark is outdated

[vladfi1]

Due to refactoring in the tf models repo, the estimator benchmarks seem to have moved or no longer exist. For example, running the following command

$ python3 lib/benchmark.py \
--git_repos="https://github.com/tensorflow/models.git;benchmark" \
--python_path=models \
--benchmark_methods=official.r1.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu \
--gcloud_key_file_url=""

produces ModuleNotFoundError: No module named 'official.r1'. I've tried to change the path to something else -- using official.legacy.image_classification.resnet makes some progress -- but I haven't been able to figure it out.

[vladfi1]

I managed to get it working with the following command:

python3 lib/benchmark.py \
--git_repos="https://github.com/tensorflow/models.git;benchmark" \
--python_path=models \
--benchmark_methods=official.benchmark.keras_imagenet_benchmark.Resnet50KerasBenchmarkSynth.benchmark_1_gpu \
--gcloud_key_file_url=""

It would be very helpful to have a way of listing available benchmarks!

[wangdalin]

I managed to get it working with the following command:

python3 lib/benchmark.py \
--git_repos="https://github.com/tensorflow/models.git;benchmark" \
--python_path=models \
--benchmark_methods=official.benchmark.keras_imagenet_benchmark.Resnet50KerasBenchmarkSynth.benchmark_1_gpu \
--gcloud_key_file_url=""

It would be very helpful to have a way of listing available benchmarks!

@vladfi1 Thanks! Your command helps me a a lot!

[johnnynunez]

It's not working now