diff options
author | Sean Owen <sowen@cloudera.com> | 2017-02-16 12:32:45 +0000 |
---|---|---|
committer | Sean Owen <sowen@cloudera.com> | 2017-02-16 12:32:45 +0000 |
commit | 0e2405490f2056728d1353abbac6f3ea177ae533 (patch) | |
tree | 1a9ec960faec7abcb8d8fbac43b6a6dc633d2297 /external/kafka-0-10-sql | |
parent | 3871d94a695d47169720e877f77ff1e4bede43ee (diff) | |
download | spark-0e2405490f2056728d1353abbac6f3ea177ae533.tar.gz spark-0e2405490f2056728d1353abbac6f3ea177ae533.tar.bz2 spark-0e2405490f2056728d1353abbac6f3ea177ae533.zip |
[SPARK-19550][BUILD][CORE][WIP] Remove Java 7 support
- Move external/java8-tests tests into core, streaming, sql and remove
- Remove MaxPermGen and related options
- Fix some reflection / TODOs around Java 8+ methods
- Update doc references to 1.7/1.8 differences
- Remove Java 7/8 related build profiles
- Update some plugins for better Java 8 compatibility
- Fix a few Java-related warnings
For the future:
- Update Java 8 examples to fully use Java 8
- Update Java tests to use lambdas for simplicity
- Update Java internal implementations to use lambdas
## How was this patch tested?
Existing tests
Author: Sean Owen <sowen@cloudera.com>
Closes #16871 from srowen/SPARK-19493.
Diffstat (limited to 'external/kafka-0-10-sql')
-rw-r--r-- | external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala | 3 |
1 files changed, 1 insertions, 2 deletions
diff --git a/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala b/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala index 02b23111af..9c5dceca2d 100644 --- a/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala +++ b/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala @@ -259,7 +259,7 @@ private[kafka010] class KafkaSource( val preferredLoc = if (numExecutors > 0) { // This allows cached KafkaConsumers in the executors to be re-used to read the same // partition in every batch. - Some(sortedExecutors(floorMod(tp.hashCode, numExecutors))) + Some(sortedExecutors(Math.floorMod(tp.hashCode, numExecutors))) } else None KafkaSourceRDDOffsetRange(tp, fromOffset, untilOffset, preferredLoc) }.filter { range => @@ -347,5 +347,4 @@ private[kafka010] object KafkaSource { if (a.host == b.host) { a.executorId > b.executorId } else { a.host > b.host } } - def floorMod(a: Long, b: Int): Int = ((a % b).toInt + b) % b } |