diff options
| author | Burak Yavuz <brkyvz@gmail.com> | 2015-11-09 17:18:49 -0800 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-11-09 17:18:49 -0800 |
| commit | 26062d22607e1f9854bc2588ba22a4e0f8bba48c (patch) | |
| tree | c9a3c0e2a0bf59a481af2637e700500799569de0 /extras/kinesis-asl/pom.xml | |
| parent | 61f9c8711c79f35d67b0456155866da316b131d9 (diff) | |
| download | spark-26062d22607e1f9854bc2588ba22a4e0f8bba48c.tar.gz spark-26062d22607e1f9854bc2588ba22a4e0f8bba48c.tar.bz2 spark-26062d22607e1f9854bc2588ba22a4e0f8bba48c.zip | |
[SPARK-11198][STREAMING][KINESIS] Support de-aggregation of records during recovery
While the KCL handles de-aggregation during the regular operation, during recovery we use the lower level api, and therefore need to de-aggregate the records.
tdas Testing is an issue, we need protobuf magic to do the aggregated records. Maybe we could depend on KPL for tests?
Author: Burak Yavuz <brkyvz@gmail.com>
Closes #9403 from brkyvz/kinesis-deaggregation.
Diffstat (limited to 'extras/kinesis-asl/pom.xml')
| -rw-r--r-- | extras/kinesis-asl/pom.xml | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/extras/kinesis-asl/pom.xml b/extras/kinesis-asl/pom.xml index ef72d97eae..519a920279 100644 --- a/extras/kinesis-asl/pom.xml +++ b/extras/kinesis-asl/pom.xml @@ -65,6 +65,12 @@ <version>${aws.java.sdk.version}</version> </dependency> <dependency> + <groupId>com.amazonaws</groupId> + <artifactId>amazon-kinesis-producer</artifactId> + <version>${aws.kinesis.producer.version}</version> + <scope>test</scope> + </dependency> + <dependency> <groupId>org.mockito</groupId> <artifactId>mockito-core</artifactId> <scope>test</scope> |