diff options
| author | wangfei <wangfei_hello@126.com> | 2014-08-19 19:37:02 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-08-19 19:37:02 -0700 |
| commit | 0e3ab94d413fd70fff748fded42ab5e2ebd66fcc (patch) | |
| tree | f701a967ccaf450f40a1d908566d62034480dacc /sql | |
| parent | c7252b0097cfacd36f17357d195b12a59e503b35 (diff) | |
| download | spark-0e3ab94d413fd70fff748fded42ab5e2ebd66fcc.tar.gz spark-0e3ab94d413fd70fff748fded42ab5e2ebd66fcc.tar.bz2 spark-0e3ab94d413fd70fff748fded42ab5e2ebd66fcc.zip | |
[SQL] add note of use synchronizedMap in SQLConf
Refer to:
http://stackoverflow.com/questions/510632/whats-the-difference-between-concurrenthashmap-and-collections-synchronizedmap
Collections.synchronizedMap(map) creates a blocking Map which will degrade performance, albeit ensure consistency. So use ConcurrentHashMap(a more effective thread-safe hashmap) instead.
also update HiveQuerySuite to fix test error when changed to ConcurrentHashMap.
Author: wangfei <wangfei_hello@126.com>
Author: scwf <wangfei1@huawei.com>
Closes #1996 from scwf/sqlconf and squashes the following commits:
93bc0c5 [wangfei] revert change of HiveQuerySuite
0cc05dd [wangfei] add note for use synchronizedMap
3c224d31 [scwf] fix formate
a7bcb98 [scwf] use ConcurrentHashMap in sql conf, intead synchronizedMap
Diffstat (limited to 'sql')
| -rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala b/sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala index 4f2adb006f..5cc41a83cc 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala @@ -54,6 +54,7 @@ private[spark] object SQLConf { trait SQLConf { import SQLConf._ + /** Only low degree of contention is expected for conf, thus NOT using ConcurrentHashMap. */ @transient protected[spark] val settings = java.util.Collections.synchronizedMap( new java.util.HashMap[String, String]()) |