diff options
| author | Yangyang Liu <yangyangliu@fb.com> | 2016-07-12 10:13:58 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-07-12 10:13:58 -0700 |
| commit | 68df47aca55e99406b7b67ef3d4b1008abf1b8b6 (patch) | |
| tree | 3f55af3bf21e5079e4b33e1f7e0e523e1b2e1fc5 /common/network-shuffle/pom.xml | |
| parent | d513c99c19e229f72d03006e251725a43c13fefd (diff) | |
| download | spark-68df47aca55e99406b7b67ef3d4b1008abf1b8b6.tar.gz spark-68df47aca55e99406b7b67ef3d4b1008abf1b8b6.tar.bz2 spark-68df47aca55e99406b7b67ef3d4b1008abf1b8b6.zip | |
[SPARK-16405] Add metrics and source for external shuffle service
## What changes were proposed in this pull request?
Since externalShuffleService is essential for spark, better monitoring for shuffle service is necessary. In order to do so, we added various metrics in shuffle service and imported into ExternalShuffleServiceSource for metric system.
Metrics added in shuffle service:
* registeredExecutorsSize
* openBlockRequestLatencyMillis
* registerExecutorRequestLatencyMillis
* blockTransferRateBytes
JIRA Issue: https://issues.apache.org/jira/browse/SPARK-16405
## How was this patch tested?
Some test cases are added to verify metrics as expected in metric system. Those unit test cases are shown in `ExternalShuffleBlockHandlerSuite `
Author: Yangyang Liu <yangyangliu@fb.com>
Closes #14080 from lovexi/yangyang-metrics.
Diffstat (limited to 'common/network-shuffle/pom.xml')
| -rw-r--r-- | common/network-shuffle/pom.xml | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/common/network-shuffle/pom.xml b/common/network-shuffle/pom.xml index 3c4f32933e..51c06b9e5a 100644 --- a/common/network-shuffle/pom.xml +++ b/common/network-shuffle/pom.xml @@ -55,6 +55,11 @@ </dependency> <dependency> + <groupId>io.dropwizard.metrics</groupId> + <artifactId>metrics-core</artifactId> + </dependency> + + <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-annotations</artifactId> </dependency> |