diff options
author | hyukjinkwon <gurwls223@gmail.com> | 2016-05-26 20:55:06 -0700 |
---|---|---|
committer | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2016-05-26 20:55:06 -0700 |
commit | 1c403733b89258e57daf7b8b0a2011981ad7ed8a (patch) | |
tree | 4b35e45f6fe8615a7bbc9889824e03333d529c85 /docs/sparkr.md | |
parent | 3fca635b4ed322208debcd89a539e42cdde6bbd4 (diff) | |
download | spark-1c403733b89258e57daf7b8b0a2011981ad7ed8a.tar.gz spark-1c403733b89258e57daf7b8b0a2011981ad7ed8a.tar.bz2 spark-1c403733b89258e57daf7b8b0a2011981ad7ed8a.zip |
[SPARK-8603][SPARKR] Use shell() instead of system2() for SparkR on Windows
## What changes were proposed in this pull request?
This PR corrects SparkR to use `shell()` instead of `system2()` on Windows.
Using `system2(...)` on Windows does not process windows file separator `\`. `shell(tralsate = TRUE, ...)` can treat this problem. So, this was changed to be chosen according to OS.
Existing tests were failed on Windows due to this problem. For example, those were failed.
```
8. Failure: sparkJars tag in SparkContext (test_includeJAR.R#34)
9. Failure: sparkJars tag in SparkContext (test_includeJAR.R#36)
```
The cases above were due to using of `system2`.
In addition, this PR also fixes some tests failed on Windows.
```
5. Failure: sparkJars sparkPackages as comma-separated strings (test_context.R#128)
6. Failure: sparkJars sparkPackages as comma-separated strings (test_context.R#131)
7. Failure: sparkJars sparkPackages as comma-separated strings (test_context.R#134)
```
The cases above were due to a weird behaviour of `normalizePath()`. On Linux, if the path does not exist, it just prints out the input but it prints out including the current path on Windows.
```r
# On Linus
path <- normalizePath("aa")
print(path)
[1] "aa"
# On Windows
path <- normalizePath("aa")
print(path)
[1] "C:\\Users\\aa"
```
## How was this patch tested?
Jenkins tests and manually tested in a Window machine as below:
Here is the [stdout](https://gist.github.com/HyukjinKwon/4bf35184f3a30f3bce987a58ec2bbbab) of testing.
Closes #7025
Author: hyukjinkwon <gurwls223@gmail.com>
Author: Hyukjin Kwon <gurwls223@gmail.com>
Author: Prakash PC <prakash.chinnu@gmail.com>
Closes #13165 from HyukjinKwon/pr/7025.
Diffstat (limited to 'docs/sparkr.md')
0 files changed, 0 insertions, 0 deletions