diff options
| author | Chen Chao <crazyjvm@gmail.com> | 2014-04-16 09:14:18 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-16 09:14:18 -0700 |
| commit | 9edd88782e0268439c5ab57400d6a7ab432fc269 (patch) | |
| tree | 1a5c8b05925aec2e3f7f6953b54cc49ee9bfae05 /dev/scalastyle | |
| parent | fec462c15321fa138d9654d64cc08468b6ab5ad1 (diff) | |
| download | spark-9edd88782e0268439c5ab57400d6a7ab432fc269.tar.gz spark-9edd88782e0268439c5ab57400d6a7ab432fc269.tar.bz2 spark-9edd88782e0268439c5ab57400d6a7ab432fc269.zip | |
update spark.default.parallelism
actually, the value 8 is only valid in mesos fine-grained mode :
<code>
override def defaultParallelism() = sc.conf.getInt("spark.default.parallelism", 8)
</code>
while in coarse-grained model including mesos coares-grained, the value of the property depending on core numbers!
<code>
override def defaultParallelism(): Int = {
conf.getInt("spark.default.parallelism", math.max(totalCoreCount.get(), 2))
}
</code>
Author: Chen Chao <crazyjvm@gmail.com>
Closes #389 from CrazyJvm/patch-2 and squashes the following commits:
84a7fe4 [Chen Chao] miss </li> at the end of every single line
04a9796 [Chen Chao] change format
ee0fae0 [Chen Chao] update spark.default.parallelism
Diffstat (limited to 'dev/scalastyle')
0 files changed, 0 insertions, 0 deletions