diff options
author | Yilun Chong <yilunchong@google.com> | 2017-12-13 14:34:52 -0800 |
---|---|---|
committer | Yilun Chong <yilunchong@google.com> | 2017-12-13 14:34:52 -0800 |
commit | 34843eddfe65686a86d12780d1ca709a997d83ad (patch) | |
tree | 670a2ed6cd727a214a4f59147c10ce2ee9c375a6 /benchmarks/README.md | |
parent | 5e732e35c1aea3a00903aae50f63cf1cf9166833 (diff) | |
download | protobuf-34843eddfe65686a86d12780d1ca709a997d83ad.tar.gz protobuf-34843eddfe65686a86d12780d1ca709a997d83ad.tar.bz2 protobuf-34843eddfe65686a86d12780d1ca709a997d83ad.zip |
Fix bugs
Diffstat (limited to 'benchmarks/README.md')
-rw-r--r-- | benchmarks/README.md | 32 |
1 files changed, 24 insertions, 8 deletions
diff --git a/benchmarks/README.md b/benchmarks/README.md index 00a63704..f266d41e 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -5,20 +5,31 @@ This directory contains benchmarking schemas and data sets that you can use to test a variety of performance scenarios against your protobuf language runtime. -The schema for the datasets is described in `benchmarks.proto`. +## Benchmark tools and build instructions -The benchmark is based on some submodules. To initialize the submodues: +First, you need to follow the instruction in the root directory's README to +build your language's protobuf, then: -For java: -``` -$ ./initialize_submodule.sh java -``` +### CPP +We are using [google/benchmark](https://github.com/google/benchmark) as the +benchmark tool for testing cpp. This is included as submodule under third_party +directory. To init and build this tools, you need to do this under root dirctory: -For java: ``` -$ ./initialize_submodule.sh cpp +$ cd third_party +$ git submodule update --init -r +$ cd benchmark && cmake -DCMAKE_BUILD_TYPE=Release && make && cd ../.. ``` +### JAVA +We're using maven to build the java benchmarks, which is the same as to build +the Java protobuf. There're no other tools need to install. We're using +[google/caliper](https://github.com/google/caliper) as benchmark tool, which +can be automaticly included by maven. + + +## Run instructions + To run all the benchmark dataset: For java: @@ -49,6 +60,8 @@ $ make cpp-benchmark $ ./cpp-benchmark $(specific generated dataset file name) ``` +## Benchmark datasets + There's some big testing data not included in the directory initially, you need to run the following command to download the testing data: @@ -57,10 +70,13 @@ $ ./download_data.sh ``` Each data set is in the format of benchmarks.proto: + 1. name is the benchmark dataset's name. 2. message_name is the benchmark's message type full name (including package and message name) 3. payload is the list of raw data. +The schema for the datasets is described in `benchmarks.proto`. + Benchmark likely want to run several benchmarks against each data set (parse, serialize, possibly JSON, possibly using different APIs, etc). |