diff options
| author | Tathagata Das <tathagata.das1565@gmail.com> | 2015-07-30 16:44:02 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-07-30 16:44:02 -0700 |
| commit | 1afdeb7b458f86e2641f062fb9ddc00e9c5c7531 (patch) | |
| tree | 014718dff32c1fef2bf0c794aa5fc8db220e713f /python/pyspark/mllib/linalg.py | |
| parent | 04c8409107710fc9a625ee513d68c149745539f3 (diff) | |
| download | spark-1afdeb7b458f86e2641f062fb9ddc00e9c5c7531.tar.gz spark-1afdeb7b458f86e2641f062fb9ddc00e9c5c7531.tar.bz2 spark-1afdeb7b458f86e2641f062fb9ddc00e9c5c7531.zip | |
[STREAMING] [TEST] [HOTFIX] Fixed Kinesis test to not throw weird errors when Kinesis tests are enabled without AWS keys
If Kinesis tests are enabled by env ENABLE_KINESIS_TESTS = 1 but no AWS credentials are found, the desired behavior is the fail the test using with
```
Exception encountered when attempting to run a suite with class name: org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDSuite *** ABORTED *** (3 seconds, 5 milliseconds)
[info] java.lang.Exception: Kinesis tests enabled, but could get not AWS credentials
```
Instead KinesisStreamSuite fails with
```
[info] - basic operation *** FAILED *** (3 seconds, 35 milliseconds)
[info] java.lang.IllegalArgumentException: requirement failed: Stream not yet created, call createStream() to create one
[info] at scala.Predef$.require(Predef.scala:233)
[info] at org.apache.spark.streaming.kinesis.KinesisTestUtils.streamName(KinesisTestUtils.scala:77)
[info] at org.apache.spark.streaming.kinesis.KinesisTestUtils$$anonfun$deleteStream$1.apply(KinesisTestUtils.scala:150)
[info] at org.apache.spark.streaming.kinesis.KinesisTestUtils$$anonfun$deleteStream$1.apply(KinesisTestUtils.scala:150)
[info] at org.apache.spark.Logging$class.logWarning(Logging.scala:71)
[info] at org.apache.spark.streaming.kinesis.KinesisTestUtils.logWarning(KinesisTestUtils.scala:39)
[info] at org.apache.spark.streaming.kinesis.KinesisTestUtils.deleteStream(KinesisTestUtils.scala:150)
[info] at org.apache.spark.streaming.kinesis.KinesisStreamSuite$$anonfun$3.apply$mcV$sp(KinesisStreamSuite.scala:111)
[info] at org.apache.spark.streaming.kinesis.KinesisStreamSuite$$anonfun$3.apply(KinesisStreamSuite.scala:86)
[info] at org.apache.spark.streaming.kinesis.KinesisStreamSuite$$anonfun$3.apply(KinesisStreamSuite.scala:86)
```
This is because attempting to delete a non-existent Kinesis stream throws uncaught exception. This PR fixes it.
Author: Tathagata Das <tathagata.das1565@gmail.com>
Closes #7809 from tdas/kinesis-test-hotfix and squashes the following commits:
7c372e6 [Tathagata Das] Fixed test
Diffstat (limited to 'python/pyspark/mllib/linalg.py')
0 files changed, 0 insertions, 0 deletions