diff options
| author | Shixiong Zhu <shixiong@databricks.com> | 2016-12-13 09:53:22 -0800 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2016-12-13 09:53:22 -0800 |
| commit | fb3081d3b38a50aa5e023c603e1b191e57f7c876 (patch) | |
| tree | b7dfa91670eeb40dffd2ca27224ea27d8a2dabbe /scalastyle-config.xml | |
| parent | d53f18cae41c6c77a0cff3f1fd266e4c1b9ea79a (diff) | |
| download | spark-fb3081d3b38a50aa5e023c603e1b191e57f7c876.tar.gz spark-fb3081d3b38a50aa5e023c603e1b191e57f7c876.tar.bz2 spark-fb3081d3b38a50aa5e023c603e1b191e57f7c876.zip | |
[SPARK-13747][CORE] Fix potential ThreadLocal leaks in RPC when using ForkJoinPool
## What changes were proposed in this pull request?
Some places in SQL may call `RpcEndpointRef.askWithRetry` (e.g., ParquetFileFormat.buildReader -> SparkContext.broadcast -> ... -> BlockManagerMaster.updateBlockInfo -> RpcEndpointRef.askWithRetry), which will finally call `Await.result`. It may cause `java.lang.IllegalArgumentException: spark.sql.execution.id is already set` when running in Scala ForkJoinPool.
This PR includes the following changes to fix this issue:
- Remove `ThreadUtils.awaitResult`
- Rename `ThreadUtils. awaitResultInForkJoinSafely` to `ThreadUtils.awaitResult`
- Replace `Await.result` in RpcTimeout with `ThreadUtils.awaitResult`.
## How was this patch tested?
Jenkins
Author: Shixiong Zhu <shixiong@databricks.com>
Closes #16230 from zsxwing/fix-SPARK-13747.
Diffstat (limited to 'scalastyle-config.xml')
| -rw-r--r-- | scalastyle-config.xml | 1 |
1 files changed, 0 insertions, 1 deletions
diff --git a/scalastyle-config.xml b/scalastyle-config.xml index 48333851ef..1f48d71cc7 100644 --- a/scalastyle-config.xml +++ b/scalastyle-config.xml @@ -200,7 +200,6 @@ This file is divided into 3 sections: // scalastyle:off awaitresult Await.result(...) // scalastyle:on awaitresult - If your codes use ThreadLocal and may run in threads created by the user, use ThreadUtils.awaitResultInForkJoinSafely instead. ]]></customMessage> </check> |