aboutsummaryrefslogtreecommitdiff
path: root/external
Commit message (Collapse)AuthorAgeFilesLines
* [maven-release-plugin] prepare release v1.1.0-rc4v1.1.0Patrick Wendell2014-09-036-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-rc3"Patrick Wendell2014-09-026-6/+6
| | | | This reverts commit b2d0493b223c5f98a593bb6d7372706cc02bebad.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-09-026-6/+6
| | | | This reverts commit 865e6f63f63f5e881a02d1a4e3b4c5d0e86fcd8e.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-306-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc3Patrick Wendell2014-08-306-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-rc3"Patrick Wendell2014-08-306-6/+6
| | | | This reverts commit 2b2e02265f80e4c5172c1e498aa9ba2c6b91c6c9.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-306-6/+6
| | | | This reverts commit 8b5f0dbd8d32a25a4e7ba3ebe1a4c3c6310aeb85.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-306-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc3Patrick Wendell2014-08-306-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-rc2"Patrick Wendell2014-08-296-6/+6
| | | | This reverts commit 711aebb329ca28046396af1e34395a0df92b5327.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-296-6/+6
| | | | This reverts commit a4a7a241441489a0d31365e18476ae2e1c34464d.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-296-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc2Patrick Wendell2014-08-296-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-rc1"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit f07183249b74dd857069028bf7d570b35f265585.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit f8f7a0c9dce764ece8acdc41d35bbf448dba7e92.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-286-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc1Patrick Wendell2014-08-286-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-rc1"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 58b0be6a29eab817d350729710345e9f39e4c506.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 78e3c036eee7113b2ed144eec5061e070b479e56.
* Revert "[maven-release-plugin] prepare release v1.1.0-rc1"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 79e86ef3e1a3ee03a7e3b166a5c7dee11c6d60d7.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit a118ea5c59d653f5a3feda21455ba60bc722b3b1.
* Revert "Revert "[maven-release-plugin] prepare for next development iteration""Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 71ec0140f7e121bdba3d19e8219e91a5e9d1e320.
* Revert "Revert "[maven-release-plugin] prepare release v1.1.0-rc1""Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 56070f12f455bae645cba887a74c72b12f1085f8.
* Revert "[maven-release-plugin] prepare release v1.1.0-rc1"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit da4b94c86c9dd0d624b3040aa4b9449be9f60fc3.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 96926c5a42c5970ed74c50db5bd9c68cacf92207.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-286-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc1Patrick Wendell2014-08-286-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-rc1"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit 79e86ef3e1a3ee03a7e3b166a5c7dee11c6d60d7.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-286-6/+6
| | | | This reverts commit a118ea5c59d653f5a3feda21455ba60bc722b3b1.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-286-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc1Patrick Wendell2014-08-286-6/+6
|
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-276-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-rc1Patrick Wendell2014-08-276-6/+6
|
* Revert "[maven-release-plugin] prepare release v1.1.0-snapshot2"Patrick Wendell2014-08-276-6/+6
| | | | This reverts commit e1535ad3c6f7400f2b7915ea91da9c60510557ba.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-276-6/+6
| | | | This reverts commit 9af3fb7385d1f9f221962f1d2d725ff79bd82033.
* [SPARK-3154][STREAMING] Make FlumePollingInputDStream shutdown cleaner.Hari Shreedharan2014-08-275-86/+236
| | | | | | | | | | | | | | | | | | | | | | | | | | | Currently lot of errors get thrown from Avro IPC layer when the dstream or sink is shutdown. This PR cleans it up. Some refactoring is done in the receiver code to put all of the RPC code into a single Try and just recover from that. The sink code has also been cleaned up. Author: Hari Shreedharan <hshreedharan@apache.org> Closes #2065 from harishreedharan/clean-flume-shutdown and squashes the following commits: f93a07c [Hari Shreedharan] Formatting fixes. d7427cc [Hari Shreedharan] More fixes! a0a8852 [Hari Shreedharan] Fix race condition, hopefully! Minor other changes. 4c9ed02 [Hari Shreedharan] Remove unneeded list in Callback handler. Other misc changes. 8fee36f [Hari Shreedharan] Scala-library is required, else maven build fails. Also catch InterruptedException in TxnProcessor. 445e700 [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into clean-flume-shutdown 87232e0 [Hari Shreedharan] Refactor Flume Input Stream. Clean up code, better error handling. 9001d26 [Hari Shreedharan] Change log level to debug in TransactionProcessor#shutdown method e7b8d82 [Hari Shreedharan] Incorporate review feedback 598efa7 [Hari Shreedharan] Clean up some exception handling code e1027c6 [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into clean-flume-shutdown ed608c8 [Hari Shreedharan] [SPARK-3154][STREAMING] Make FlumePollingInputDStream shutdown cleaner. (cherry picked from commit 6f671d04fa98f97fd48c5e749b9f47dd4a8b4f44) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* SPARK-2798 [BUILD] Correct several small errors in Flume module pom.xml filesSean Owen2014-08-252-15/+12
| | | | | | | | | | | | | | | | | | | (EDIT) Since the scalatest issue was since resolved, this is now about a few small problems in the Flume Sink `pom.xml` - `scalatest` is not declared as a test-scope dependency - Its Avro version doesn't match the rest of the build - Its Flume version is not synced with the other Flume module - The other Flume module declares its dependency on Flume Sink slightly incorrectly, hard-coding the Scala 2.10 version - It depends on Scala Lang directly, which it shouldn't Author: Sean Owen <sowen@cloudera.com> Closes #1726 from srowen/SPARK-2798 and squashes the following commits: a46e2c6 [Sean Owen] scalatest to test scope, harmonize Avro and Flume versions, remove direct Scala dependency, fix '2.10' in Flume dependency (cherry picked from commit cd30db566a327ddf63cd242c758e46ce2d9479df) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [SPARK-3169] Removed dependency on spark streaming test from spark flume sinkTathagata Das2014-08-222-11/+17
| | | | | | | | | | | | | | | | Due to maven bug https://jira.codehaus.org/browse/MNG-1378, maven could not resolve spark streaming classes required by the spark-streaming test-jar dependency of external/flume-sink. There is no particular reason that the external/flume-sink has to depend on Spark Streaming at all, so I am eliminating this dependency. Also I have removed the exclusions present in the Flume dependencies, as there is no reason to exclude them (they were excluded in the external/flume module to prevent dependency collisions with Spark). Since Jenkins will test the sbt build and the unit test, I only tested maven compilation locally. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #2101 from tdas/spark-sink-pom-fix and squashes the following commits: 8f42621 [Tathagata Das] Added Flume sink exclusions back, and added netty to test dependencies 93b559f [Tathagata Das] Removed dependency on spark streaming test from spark flume sink (cherry picked from commit 3004074152b7261c2a968bb8e94ec7c41a7b43c1) Signed-off-by: Patrick Wendell <pwendell@gmail.com>
* Revert "HOTFIX:Temporarily removing flume sink test in 1.1 branch"Patrick Wendell2014-08-221-0/+204
| | | | This reverts commit 1d5e84a99076d3e0168dd2f4626c7911e7ba49e7.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-216-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-snapshot2Patrick Wendell2014-08-216-6/+6
|
* HOTFIX:Temporarily removing flume sink test in 1.1 branchPatrick Wendell2014-08-201-204/+0
|
* [HOTFIX][STREAMING] Allow the JVM/Netty to decide which port to bind to in ↵Hari Shreedharan2014-08-202-29/+34
| | | | | | | | | | | | | | | | | | Flume Polling Tests. Author: Hari Shreedharan <harishreedharan@gmail.com> Closes #1820 from harishreedharan/use-free-ports and squashes the following commits: b939067 [Hari Shreedharan] Remove unused import. 67856a8 [Hari Shreedharan] Remove findFreePort. 0ea51d1 [Hari Shreedharan] Make some changes to getPort to use map on the serverOpt. 1fb0283 [Hari Shreedharan] Merge branch 'master' of https://github.com/apache/spark into use-free-ports b351651 [Hari Shreedharan] Allow Netty to choose port, and query it to decide the port to bind to. Leaving findFreePort as is, if other tests want to use it at some point. e6c9620 [Hari Shreedharan] Making sure the second sink uses the correct port. 11c340d [Hari Shreedharan] Add info about race condition to scaladoc. e89d135 [Hari Shreedharan] Adding Scaladoc. 6013bb0 [Hari Shreedharan] [STREAMING] Find free ports to use before attempting to create Flume Sink in Flume Polling Suite
* [HOTFIX][Streaming] Handle port collisions in flume polling testAndrew Or2014-08-201-1/+31
| | | | | | | | | | | | | This is failing my tests in #1777. @tdas Author: Andrew Or <andrewor14@gmail.com> Closes #1803 from andrewor14/fix-flaky-streaming-test and squashes the following commits: ea11a03 [Andrew Or] Catch all exceptions caused by BindExceptions 54a0ca0 [Andrew Or] Merge branch 'master' of github.com:apache/spark into fix-flaky-streaming-test 664095c [Andrew Or] Tone down bind exception message af3ddc9 [Andrew Or] Handle port collisions in flume polling test
* Revert "[maven-release-plugin] prepare release v1.1.0-snapshot1"Patrick Wendell2014-08-206-6/+6
| | | | This reverts commit d428d88418d385d1d04e1b0adcb6b068efe9c7b0.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-08-206-6/+6
| | | | This reverts commit c204a742a9eb9d3fd318e0f059bd00cbfb8b2c14.
* [SPARK-3054][STREAMING] Add unit tests for Spark Sink.Hari Shreedharan2014-08-204-2/+212
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds unit tests for Spark Sink. It also removes the private[flume] for Spark Sink, since the sink is instantiated from Flume configuration (looks like this is ignored by reflection which is used by Flume, but we should still remove it anyway). Author: Hari Shreedharan <hshreedharan@apache.org> Author: Hari Shreedharan <hshreedharan@cloudera.com> Closes #1958 from harishreedharan/spark-sink-test and squashes the following commits: e3110b9 [Hari Shreedharan] Add a sleep to allow sink to commit the transactions 120b81e [Hari Shreedharan] Fix complexity in threading model in test 4df5be6 [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into spark-sink-test c9190d1 [Hari Shreedharan] Indentation and spaces changes 7fedc5a [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into spark-sink-test abc20cb [Hari Shreedharan] Minor test changes 7b9b649 [Hari Shreedharan] Merge branch 'master' into spark-sink-test f2c56c9 [Hari Shreedharan] Update SparkSinkSuite.scala a24aac8 [Hari Shreedharan] Remove unused var c86d615 [Hari Shreedharan] [SPARK-3054][STREAMING] Add unit tests for Spark Sink. (cherry picked from commit 8c5a2226932c572898c76eb6fab9283f02ad4103) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-08-076-6/+6
|
* [maven-release-plugin] prepare release v1.1.0-snapshot1Patrick Wendell2014-08-076-6/+6
|
* [SPARK-1022][Streaming][HOTFIX] Fixed zookeeper dependency of KafkaTathagata Das2014-08-051-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://github.com/apache/spark/pull/1751 caused maven builds to fail. ``` ~/Apache/spark(branch-1.1|✔) ➤ mvn -U -DskipTests clean install . . . [error] Apache/spark/external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala:36: object NIOServerCnxnFactory is not a member of package org.apache.zookeeper.server [error] import org.apache.zookeeper.server.NIOServerCnxnFactory [error] ^ [error] Apache/spark/external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala:199: not found: type NIOServerCnxnFactory [error] val factory = new NIOServerCnxnFactory() [error] ^ [error] two errors found [error] Compile failed at Aug 5, 2014 1:42:36 PM [0.503s] ``` The problem is how SBT and Maven resolves multiple versions of the same library, which in this case, is Zookeeper. Observing and comparing the dependency trees from Maven and SBT showed this. Spark depends on ZK 3.4.5 whereas Apache Kafka transitively depends on upon ZK 3.3.4. SBT decides to evict 3.3.4 and use the higher version 3.4.5. But Maven decides to stick to the closest (in the tree) dependent version of 3.3.4. And 3.3.4 does not have NIOServerCnxnFactory. The solution in this patch excludes zookeeper from the apache-kafka dependency in streaming-kafka module so that it just inherits zookeeper from Spark core. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #1797 from tdas/kafka-zk-fix and squashes the following commits: 94b3931 [Tathagata Das] Fixed zookeeper dependency of Kafka (cherry picked from commit ee7f30856bf3f7b9a4f1d3641b6bc2cc4e842b0e) Signed-off-by: Patrick Wendell <pwendell@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-16 07:58:44 +0000