diff options
| author | Tejas Patil <tejasp@fb.com> | 2016-05-27 12:05:11 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-05-27 12:05:11 -0700 |
| commit | a96e4151a9d429cfaf457c07b4ce174890a3b39b (patch) | |
| tree | 569dc53323762261e23aeebda4e964a0280616a7 /docs/ml-classification-regression.md | |
| parent | b376a4eabc82d622ea26290345c01465af7a628d (diff) | |
| download | spark-a96e4151a9d429cfaf457c07b4ce174890a3b39b.tar.gz spark-a96e4151a9d429cfaf457c07b4ce174890a3b39b.tar.bz2 spark-a96e4151a9d429cfaf457c07b4ce174890a3b39b.zip | |
[SPARK-14400][SQL] ScriptTransformation does not fail the job for bad user command
## What changes were proposed in this pull request?
- Refer to the Jira for the problem: jira : https://issues.apache.org/jira/browse/SPARK-14400
- The fix is to check if the process has exited with a non-zero exit code in `hasNext()`. I have moved this and checking of writer thread exception to a separate method.
## How was this patch tested?
- Ran a job which had incorrect transform script command and saw that the job fails
- Existing unit tests for `ScriptTransformationSuite`. Added a new unit test
Author: Tejas Patil <tejasp@fb.com>
Closes #12194 from tejasapatil/script_transform.
Diffstat (limited to 'docs/ml-classification-regression.md')
0 files changed, 0 insertions, 0 deletions