aboutsummaryrefslogtreecommitdiff
path: root/extras
Commit message (Collapse)AuthorAgeFilesLines
* Preparing development version 1.4.0-SNAPSHOTPatrick Wendell2015-05-273-3/+3
|
* Preparing Spark release v1.4.0-rc3Patrick Wendell2015-05-273-3/+3
|
* Preparing development version 1.4.0-SNAPSHOTPatrick Wendell2015-05-233-3/+3
|
* Preparing Spark release v1.4.0-rc2Patrick Wendell2015-05-233-3/+3
|
* Preparing development version 1.4.1-SNAPSHOTPatrick Wendell2015-05-233-3/+3
|
* Preparing Spark release v1.4.0-rc2-testPatrick Wendell2015-05-233-3/+3
|
* Preparing development version 1.4.1-SNAPSHOTPatrick Wendell2015-05-233-3/+3
|
* Preparing Spark release 1.4.0-rc2-testPatrick Wendell2015-05-233-3/+3
|
* [SPARK-7838] [STREAMING] Set scope for kinesis streamTathagata Das2015-05-221-3/+6
| | | | | | | | | | | | Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6369 from tdas/SPARK-7838 and squashes the following commits: 87d1c7f [Tathagata Das] Addressed comment 37775d8 [Tathagata Das] set scope for kinesis stream (cherry picked from commit baa89838cca96fa091c9e5ce62be01e1a265d820) Signed-off-by: Andrew Or <andrew@databricks.com>
* [SPARK-7788] Made KinesisReceiver.onStart() non-blockingTathagata Das2015-05-221-5/+25
| | | | | | | | | | | | | | | KinesisReceiver calls worker.run() which is a blocking call (while loop) as per source code of kinesis-client library - https://github.com/awslabs/amazon-kinesis-client/blob/v1.2.1/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/Worker.java. This results in infinite loop while calling sparkStreamingContext.stop(stopSparkContext = false, stopGracefully = true) perhaps because ReceiverTracker is never able to register the receiver (it's receiverInfo field is a empty map) causing it to be stuck in infinite loop while waiting for running flag to be set to false. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6348 from tdas/SPARK-7788 and squashes the following commits: 2584683 [Tathagata Das] Added receiver id in thread name 6cf1cd4 [Tathagata Das] Made KinesisReceiver.onStart non-blocking (cherry picked from commit 1c388a9985999e043fa002618a357bc8f0a8b65a) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [SPARK-7722] [STREAMING] Added Kinesis to style checkerTathagata Das2015-05-211-1/+1
| | | | | | | | | | | Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6325 from tdas/SPARK-7722 and squashes the following commits: 9ab35b2 [Tathagata Das] Fixed styles in Kinesis (cherry picked from commit 311fab6f1b00db1a581d77be5196dd045f93d83d) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [SPARK-7787] [STREAMING] Fix serialization issue of SerializableAWSCredentialsTathagata Das2015-05-212-18/+17
| | | | | | | | | | | | | Lack of default constructor causes deserialization to fail. This occurs only when the AWS credentials are explicitly specified through KinesisUtils. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6316 from tdas/SPARK-7787 and squashes the following commits: 248ca5c [Tathagata Das] Fixed serializability (cherry picked from commit 4b7ff3092c53827817079e0810563cbb0b9d0747) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* Preparing development version 1.4.0-SNAPSHOTPatrick Wendell2015-05-203-3/+3
|
* Preparing Spark release rc-testPatrick Wendell2015-05-203-3/+3
|
* Preparing development version 1.4.0-SNAPSHOTpwendell2015-05-203-3/+3
|
* Preparing Spark release rc-testpwendell2015-05-203-3/+3
|
* Preparing development version 1.4.0-SNAPSHOTjenkins2015-05-203-3/+3
|
* Preparing Spark release rc-testjenkins2015-05-203-3/+3
|
* Preparing development version 1.4.0-SNAPSHOTPatrick Wendell2015-05-203-3/+3
|
* Preparing Spark release rc-testPatrick Wendell2015-05-203-3/+3
|
* Preparing development version 1.4.1-SNAPSHOTPatrick Wendell2015-05-193-3/+3
|
* Preparing Spark release v1.4.0-rc1Patrick Wendell2015-05-193-3/+3
|
* Revert "Preparing Spark release v1.4.0-rc1"Patrick Wendell2015-05-193-3/+3
| | | | This reverts commit 79fb01a3be07b5086134a6fe103248e9a33a9500.
* Revert "Preparing development version 1.4.1-SNAPSHOT"Patrick Wendell2015-05-193-3/+3
| | | | This reverts commit a1d896b85bd3fb88284f8b6758d7e5f0a1bb9eb3.
* Preparing development version 1.4.1-SNAPSHOTPatrick Wendell2015-05-193-3/+3
|
* Preparing Spark release v1.4.0-rc1Patrick Wendell2015-05-193-3/+3
|
* Revert "Preparing Spark release v1.4.0-rc1"Patrick Wendell2015-05-193-3/+3
| | | | This reverts commit 38ccef36c1551dc36d9444f47df11ae34c1e139e.
* Revert "Preparing development version 1.4.1-SNAPSHOT"Patrick Wendell2015-05-193-3/+3
| | | | This reverts commit 40190ce22622cadd41f740a763fba061281c2966.
* [HOTFIX] Fixing style failures in Kinesis sourcePatrick Wendell2015-05-192-4/+6
|
* Preparing development version 1.4.1-SNAPSHOTPatrick Wendell2015-05-193-3/+3
|
* Preparing Spark release v1.4.0-rc1Patrick Wendell2015-05-193-3/+3
|
* Revert "Preparing Spark release v1.4.0-rc1"Patrick Wendell2015-05-183-3/+3
| | | | This reverts commit e8e97e3a630dea3c68702e26bc56f61044b2db71.
* Revert "Preparing development version 1.4.1-SNAPSHOT"Patrick Wendell2015-05-183-3/+3
| | | | This reverts commit 758ca74bab7c342f94442f69476c6b9543ac1228.
* Preparing development version 1.4.1-SNAPSHOTPatrick Wendell2015-05-193-3/+3
|
* Preparing Spark release v1.4.0-rc1Patrick Wendell2015-05-193-3/+3
|
* [SPARK-7692] Updated Kinesis examplesTathagata Das2015-05-182-237/+268
| | | | | | | | | | | | | | | | | | | - Updated Kinesis examples to use stable API - Cleaned up comments, etc. - Renamed KinesisWordCountProducerASL to KinesisWordProducerASL Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6249 from tdas/kinesis-examples and squashes the following commits: 7cc307b [Tathagata Das] More tweaks f080872 [Tathagata Das] More cleanup 841987f [Tathagata Das] Small update 011cbe2 [Tathagata Das] More fixes b0d74f9 [Tathagata Das] Updated examples. (cherry picked from commit 3a6003866ade45974b43a9e785ec35fb76a32b99) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [SPARK-6514] [SPARK-5960] [SPARK-6656] [SPARK-7679] [STREAMING] [KINESIS] ↵Tathagata Das2015-05-175-118/+346
| | | | | | | | | | | | | | | | | | | Updates to the Kinesis API SPARK-6514 - Use correct region SPARK-5960 - Allow AWS Credentials to be directly passed SPARK-6656 - Specify kinesis application name explicitly SPARK-7679 - Upgrade to latest KCL and AWS SDK. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6147 from tdas/kinesis-api-update and squashes the following commits: f23ea77 [Tathagata Das] Updated versions and updated APIs 373b201 [Tathagata Das] Updated Kinesis API (cherry picked from commit ca4257aec658aaa87f4f097dd7534033d5f13ddc) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [SPARK-6440][CORE]Handle IPv6 addresses properly when constructing URInyaapa2015-04-131-1/+2
| | | | | | | | | | Author: nyaapa <nyaapa@gmail.com> Closes #5424 from nyaapa/master and squashes the following commits: 6b717aa [nyaapa] [SPARK-6440][CORE] Remove Utils.localIpAddressHostname, Utils.localIpAddressURI and Utils.getAddressHostName; make Utils.localIpAddress private; rename Utils.localHostURI into Utils.localHostNameForURI; use Utils.localHostName in org.apache.spark.streaming.kinesis.KinesisReceiver and org.apache.spark.sql.hive.thriftserver.SparkSQLEnv 2098081 [nyaapa] [SPARK-6440][CORE] style fixes and use getHostAddress instead of getHostName 84763d7 [nyaapa] [SPARK-6440][CORE]Handle IPv6 addresses properly when constructing URI
* [SPARK-6758]block the right jetty package in logWangTaoTheTonic2015-04-093-5/+5
| | | | | | | | | | | | https://issues.apache.org/jira/browse/SPARK-6758 I am not sure if it is ok to block them in test resources too (as we shade jetty in assembly?). Author: WangTaoTheTonic <wangtao111@huawei.com> Closes #5406 from WangTaoTheTonic/SPARK-6758 and squashes the following commits: e09605b [WangTaoTheTonic] block the right jetty package
* [SPARK-6371] [build] Update version to 1.4.0-SNAPSHOT.Marcelo Vanzin2015-03-203-3/+3
| | | | | | | | | | | | | | | | Author: Marcelo Vanzin <vanzin@cloudera.com> Closes #5056 from vanzin/SPARK-6371 and squashes the following commits: 63220df [Marcelo Vanzin] Merge branch 'master' into SPARK-6371 6506f75 [Marcelo Vanzin] Use more fine-grained exclusion. 178ba71 [Marcelo Vanzin] Oops. 75b2375 [Marcelo Vanzin] Exclude VertexRDD in MiMA. a45a62c [Marcelo Vanzin] Work around MIMA warning. 1d8a670 [Marcelo Vanzin] Re-group jetty exclusion. 0e8e909 [Marcelo Vanzin] Ignore ml, don't ignore graphx. cef4603 [Marcelo Vanzin] Indentation. 296cf82 [Marcelo Vanzin] [SPARK-6371] [build] Update version to 1.4.0-SNAPSHOT.
* SPARK-6182 [BUILD] spark-parent pom needs to be published for both 2.10 and 2.11Sean Owen2015-03-053-3/+3
| | | | | | | | | | Option 1 of 2: Convert spark-parent module name to spark-parent_2.10 / spark-parent_2.11 Author: Sean Owen <sowen@cloudera.com> Closes #4912 from srowen/SPARK-6182.1 and squashes the following commits: eff60de [Sean Owen] Convert spark-parent module name to spark-parent_2.10 / spark-parent_2.11
* SPARK-4682 [CORE] Consolidate various 'Clock' classesSean Owen2015-02-193-20/+17
| | | | | | | | | | | | | | | Another one from JoshRosen 's wish list. The first commit is much smaller and removes 2 of the 4 Clock classes. The second is much larger, necessary for consolidating the streaming one. I put together implementations in the way that seemed simplest. Almost all the change is standardizing class and method names. Author: Sean Owen <sowen@cloudera.com> Closes #4514 from srowen/SPARK-4682 and squashes the following commits: 5ed3a03 [Sean Owen] Javadoc Clock classes; make ManualClock private[spark] 169dd13 [Sean Owen] Add support for legacy org.apache.spark.streaming clock class names 277785a [Sean Owen] Reduce the net change in this patch by reversing some unnecessary syntax changes along the way b5e53df [Sean Owen] FakeClock -> ManualClock; getTime() -> getTimeMillis() 160863a [Sean Owen] Consolidate Streaming Clock class into common util Clock 7c956b2 [Sean Owen] Consolidate Clocks except for Streaming Clock
* [SPARK-5735] Replace uses of EasyMock with MockitoJosh Rosen2015-02-132-141/+127
| | | | | | | | | | | | | | | | | | | This patch replaces all uses of EasyMock with Mockito. There are two motivations for this: 1. We should use a single mocking framework in our tests in order to keep things consistent. 2. EasyMock may be responsible for non-deterministic unit test failures due to its Objensis dependency (see SPARK-5626 for more details). Most of these changes are fairly mechanical translations of Mockito code to EasyMock, although I made a small change that strengthens the assertions in one test in KinesisReceiverSuite. Author: Josh Rosen <joshrosen@databricks.com> Closes #4578 from JoshRosen/SPARK-5735-remove-easymock and squashes the following commits: 0ab192b [Josh Rosen] Import sorting plus two minor changes to more closely match old semantics. 977565b [Josh Rosen] Remove EasyMock from build. fae1d8f [Josh Rosen] Remove EasyMock usage in KinesisReceiverSuite. 7cca486 [Josh Rosen] Remove EasyMock usage in MesosSchedulerBackendSuite fc5e94d [Josh Rosen] Remove EasyMock in CacheManagerSuite
* [SPARK-5155] Build fails with spark-ganglia-lgpl profileKousuke Saruta2015-02-011-1/+1
| | | | | | | | | | | | Build fails with spark-ganglia-lgpl profile at the moment. This is because pom.xml for spark-ganglia-lgpl is not updated. This PR is related to #4218, #4209 and #3812. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes #4303 from sarutak/fix-ganglia-pom-for-metric and squashes the following commits: 5cf455f [Kousuke Saruta] Fixed pom.xml for ganglia in order to use io.dropwizard.metrics
* SPARK-4159 [CORE] Maven build doesn't run JUnit test suitesSean Owen2015-01-064-29/+4
| | | | | | | | | | | | | | | | | | This PR: - Reenables `surefire`, and copies config from `scalatest` (which is itself an old fork of `surefire`, so similar) - Tells `surefire` to test only Java tests - Enables `surefire` and `scalatest` for all children, and in turn eliminates some duplication. For me this causes the Scala and Java tests to be run once each, it seems, as desired. It doesn't affect the SBT build but works for Maven. I still need to verify that all of the Scala tests and Java tests are being run. Author: Sean Owen <sowen@cloudera.com> Closes #3651 from srowen/SPARK-4159 and squashes the following commits: 2e8a0af [Sean Owen] Remove specialized SPARK_HOME setting for REPL, YARN tests as it appears to be obsolete 12e4558 [Sean Owen] Append to unit-test.log instead of overwriting, so that both surefire and scalatest output is preserved. Also standardize/correct comments a bit. e6f8601 [Sean Owen] Reenable Java tests by reenabling surefire with config cloned from scalatest; centralize test config in the parent
* [SPARK-4501][Core] - Create build/mvn to automatically download ↵Brennon York2014-12-271-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | maven/zinc/scalac Creates a top level directory script (as `build/mvn`) to automatically download zinc and the specific version of scala used to easily build spark. This will also download and install maven if the user doesn't already have it and all packages are hosted under the `build/` directory. Tested on both Linux and OSX OS's and both work. All commands pass through to the maven binary so it acts exactly as a traditional maven call would. Author: Brennon York <brennon.york@capitalone.com> Closes #3707 from brennonyork/SPARK-4501 and squashes the following commits: 0e5a0e4 [Brennon York] minor incorrect doc verbage (with -> this) 9b79e38 [Brennon York] fixed merge conflicts with dev/run-tests, properly quoted args in sbt/sbt, fixed bug where relative paths would fail if passed in from build/mvn d2d41b6 [Brennon York] added blurb about leverging zinc with build/mvn b979c58 [Brennon York] updated the merge conflict c5634de [Brennon York] updated documentation to overview build/mvn, updated all points where sbt/sbt was referenced with build/sbt b8437ba [Brennon York] set progress bars for curl and wget when not run on jenkins, no progress bar when run on jenkins, moved sbt script to build/sbt, wrote stub and warning under sbt/sbt which calls build/sbt, modified build/sbt to use the correct directory, fixed bug in build/sbt-launch-lib.bash to correctly pull the sbt version be11317 [Brennon York] added switch to silence download progress only if AMPLAB_JENKINS is set 28d0a99 [Brennon York] updated to remove the python dependency, uses grep instead 7e785a6 [Brennon York] added silent and quiet flags to curl and wget respectively, added single echo output to denote start of a download if download is needed 14a5da0 [Brennon York] removed unnecessary zinc output on startup 1af4a94 [Brennon York] fixed bug with uppercase vs lowercase variable 3e8b9b3 [Brennon York] updated to properly only restart zinc if it was freshly installed a680d12 [Brennon York] Added comments to functions and tested various mvn calls bb8cc9d [Brennon York] removed package files ef017e6 [Brennon York] removed OS complexities, setup generic install_app call, removed extra file complexities, removed help, removed forced install (defaults now), removed double-dash from cli 07bf018 [Brennon York] Updated to specifically handle pulling down the correct scala version f914dea [Brennon York] Beginning final portions of localized scala home 69c4e44 [Brennon York] working linux and osx installers for purely local mvn build 4a1609c [Brennon York] finalizing working linux install for maven to local ./build/apache-maven folder cbfcc68 [Brennon York] Changed the default sbt/sbt to build/sbt and added a build/mvn which will automatically download, install, and execute maven with zinc for easier build capability
* Bumping version to 1.3.0-SNAPSHOT.Marcelo Vanzin2014-11-183-3/+3
| | | | | | | | | | | | Author: Marcelo Vanzin <vanzin@cloudera.com> Closes #3277 from vanzin/version-1.3 and squashes the following commits: 7c3c396 [Marcelo Vanzin] Added temp repo to sbt build. 5f404ff [Marcelo Vanzin] Add another exclusion. 19457e7 [Marcelo Vanzin] Update old version to 1.2, add temporary 1.2 repo. 3c8d705 [Marcelo Vanzin] Workaround for MIMA checks. e940810 [Marcelo Vanzin] Bumping version to 1.3.0-SNAPSHOT.
* [SPARK-3748] Log thread name in unit test logsReynold Xin2014-10-012-2/+2
| | | | | | | | | | Thread names are useful for correlating failures. Author: Reynold Xin <rxin@apache.org> Closes #2600 from rxin/log4j and squashes the following commits: 83ffe88 [Reynold Xin] [SPARK-3748] Log thread name in unit test logs
* SPARK-3639 | Removed settings master in examplesaniketbhatnagar2014-09-262-13/+9
| | | | | | | | | | | | This patch removes setting of master as local in Kinesis examples so that users can set it using submit-job. Author: aniketbhatnagar <aniket.bhatnagar@gmail.com> Closes #2536 from aniketbhatnagar/Kinesis-Examples-Master-Unset and squashes the following commits: c9723ac [aniketbhatnagar] Merge remote-tracking branch 'origin/Kinesis-Examples-Master-Unset' into Kinesis-Examples-Master-Unset fec8ead [aniketbhatnagar] SPARK-3639 | Removed settings master in examples 31cdc59 [aniketbhatnagar] SPARK-3639 | Removed settings master in examples
* [SPARK-3452] Maven build should skip publishing artifacts people shouldn...Prashant Sharma2014-09-141-0/+14
| | | | | | | | | | | | | | | | ...'t depend on Publish local in maven term is `install` and publish otherwise is `deploy` So disabled both for following projects. Author: Prashant Sharma <prashant.s@imaginea.com> Closes #2329 from ScrapCodes/SPARK-3452/maven-skip-install and squashes the following commits: 257b79a [Prashant Sharma] [SPARK-3452] Maven build should skip publishing artifacts people shouldn't depend on
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-23 12:12:52 +0000