diff options
author | Wenchen Fan <wenchen@databricks.com> | 2016-04-28 21:57:58 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-04-28 21:57:58 -0700 |
commit | 6f9a18fe311925056cce83a44f187f122b6591cb (patch) | |
tree | ffbbd73a0399c6aad5ef6871e2e264c7ac9b3336 /sql | |
parent | 9c7c42bc6a35679cfffcdfb6feb26af834fec2e1 (diff) | |
download | spark-6f9a18fe311925056cce83a44f187f122b6591cb.tar.gz spark-6f9a18fe311925056cce83a44f187f122b6591cb.tar.bz2 spark-6f9a18fe311925056cce83a44f187f122b6591cb.zip |
[HOTFIX][CORE] fix a concurrence issue in NewAccumulator
## What changes were proposed in this pull request?
`AccumulatorContext` is not thread-safe, that's why all of its methods are synchronized. However, there is one exception: the `AccumulatorContext.originals`. `NewAccumulator` use it to check if it's registered, which is wrong as it's not synchronized.
This PR mark `AccumulatorContext.originals` as `private` and now all access to `AccumulatorContext` is synchronized.
## How was this patch tested?
I verified it locally. To be safe, we can let jenkins test it many times to make sure this problem is gone.
Author: Wenchen Fan <wenchen@databricks.com>
Closes #12773 from cloud-fan/debug.
Diffstat (limited to 'sql')
-rw-r--r-- | sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala index 0e6356b578..1095a73c58 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala @@ -334,10 +334,10 @@ class CachedTableSuite extends QueryTest with SQLTestUtils with SharedSQLContext sql("SELECT * FROM t2").count() AccumulatorContext.synchronized { - val accsSize = AccumulatorContext.originals.size + val accsSize = AccumulatorContext.numAccums sqlContext.uncacheTable("t1") sqlContext.uncacheTable("t2") - assert((accsSize - 2) == AccumulatorContext.originals.size) + assert((accsSize - 2) == AccumulatorContext.numAccums) } } |