diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-06-04 17:33:24 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-06-04 17:33:24 -0700 |
| commit | 74dc2a90bcb05b64c3e7efc02d1451b0cbc2adba (patch) | |
| tree | c14f89a7a2e99414a90806a9311018edfc0353b2 /project/SparkBuild.scala | |
| parent | 63bc0c4430680cce230dd7a10d34da0492351446 (diff) | |
| download | spark-74dc2a90bcb05b64c3e7efc02d1451b0cbc2adba.tar.gz spark-74dc2a90bcb05b64c3e7efc02d1451b0cbc2adba.tar.bz2 spark-74dc2a90bcb05b64c3e7efc02d1451b0cbc2adba.zip | |
[SPARK-8106] [SQL] Set derby.system.durability=test to speed up Hive compatibility tests
Derby has a `derby.system.durability` configuration property that can be used to disable I/O synchronization calls for writes. This sacrifices durability but can result in large performance gains, which is appropriate for tests.
We should enable this in our test system properties in order to speed up the Hive compatibility tests. I saw 2-3x speedups locally with this change.
See https://db.apache.org/derby/docs/10.8/ref/rrefproperdurability.html for more documentation of this property.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #6651 from JoshRosen/hive-compat-suite-speedup and squashes the following commits:
b7a08a2 [Josh Rosen] Set derby.system.durability=test in our unit tests.
Diffstat (limited to 'project/SparkBuild.scala')
| -rw-r--r-- | project/SparkBuild.scala | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index f65031fe25..ef3a175bac 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -504,6 +504,7 @@ object TestSettings { javaOptions in Test += "-Dspark.driver.allowMultipleContexts=true", javaOptions in Test += "-Dspark.unsafe.exceptionOnMemoryLeak=true", javaOptions in Test += "-Dsun.io.serialization.extendedDebugInfo=true", + javaOptions in Test += "-Dderby.system.durability=test", javaOptions in Test ++= System.getProperties.filter(_._1 startsWith "spark") .map { case (k,v) => s"-D$k=$v" }.toSeq, javaOptions in Test += "-ea", |