diff options
| author | Dhruve Ashar <dhruveashar@gmail.com> | 2016-04-07 10:39:21 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-04-07 10:39:21 -0500 |
| commit | 033d8081525a7137085ec898e2426a58056ee2b8 (patch) | |
| tree | 08755f863e3c2b9692fe2d0d6332d0645b09bbff /sql | |
| parent | 35e0db2d45e2f98d8b4d2c0d442ab19cd615830e (diff) | |
| download | spark-033d8081525a7137085ec898e2426a58056ee2b8.tar.gz spark-033d8081525a7137085ec898e2426a58056ee2b8.tar.bz2 spark-033d8081525a7137085ec898e2426a58056ee2b8.zip | |
[SPARK-12384] Enables spark-clients to set the min(-Xms) and max(*.memory config) j…
## What changes were proposed in this pull request?
Currently Spark clients are started with the same memory setting for Xms and Xms leading to reserving unnecessary higher amounts of memory.
This behavior is changed and the clients can now specify an initial heap size using the extraJavaOptions in the config for driver,executor and am individually.
Note, that only -Xms can be provided through this config option, if the client wants to set the max size(-Xmx), this has to be done via the *.memory configuration knobs which are currently supported.
## How was this patch tested?
Monitored executor and yarn logs in debug mode to verify the commands through which they are being launched in client and cluster mode. The driver memory was verified locally using jps -v. Setting up -Xmx parameter in the javaExtraOptions raises exception with the info provided.
Author: Dhruve Ashar <dhruveashar@gmail.com>
Closes #12115 from dhruve/impr/SPARK-12384.
Diffstat (limited to 'sql')
0 files changed, 0 insertions, 0 deletions