aboutsummaryrefslogtreecommitdiff
path: root/pom.xml
Commit message (Collapse)AuthorAgeFilesLines
* graph -> graphx in pom.xmlAnkur Dave2014-01-101-1/+1
|
* Merge remote-tracking branch 'spark-upstream/master' into HEADAnkur Dave2014-01-081-109/+76
|\ | | | | | | | | | | | | | | | | | | | | Conflicts: README.md core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala core/src/main/scala/org/apache/spark/util/collection/PrimitiveKeyOpenHashMap.scala pom.xml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * Add CDH Repository to Maven BuildPatrick Wendell2014-01-081-0/+5
| |
| * Merge pull request #313 from tdas/project-refactorPatrick Wendell2014-01-071-23/+6
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored the streaming project to separate external libraries like Twitter, Kafka, Flume, etc. At a high level, these are the following changes. 1. All the external code was put in `SPARK_HOME/external/` as separate SBT projects and Maven modules. Their artifact names are `spark-streaming-twitter`, `spark-streaming-kafka`, etc. Both SparkBuild.scala and pom.xml files have been updated. References to external libraries and repositories have been removed from the settings of root and streaming projects/modules. 2. To avail the external functionality (say, creating a Twitter stream), the developer has to `import org.apache.spark.streaming.twitter._` . For Scala API, the developer has to call `TwitterUtils.createStream(streamingContext, ...)`. For the Java API, the developer has to call `TwitterUtils.createStream(javaStreamingContext, ...)`. 3. Each external project has its own scala and java unit tests. Note the unit tests of each external library use classes of the streaming unit tests (`TestSuiteBase`, `LocalJavaStreamingContext`, etc.). To enable this code sharing among test classes, `dependsOn(streaming % "compile->compile,test->test")` was used in the SparkBuild.scala . In the streaming/pom.xml, an additional `maven-jar-plugin` was necessary to capture this dependency (see comment inside the pom.xml for more information). 4. Jars of the external projects have been added to examples project but not to the assembly project. 5. In some files, imports have been rearrange to conform to the Spark coding guidelines.
| | * Merge remote-tracking branch 'apache/master' into project-refactorTathagata Das2014-01-061-49/+17
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: examples/src/main/java/org/apache/spark/streaming/examples/JavaFlumeEventCount.java streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/test/java/org/apache/spark/streaming/JavaAPISuite.java streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala streaming/src/test/scala/org/apache/spark/streaming/TestSuiteBase.scala
| | * | Added pom.xml for external projects and removed unnecessary dependencies and ↵Tathagata Das2013-12-311-23/+6
| | | | | | | | | | | | | | | | repositoris from other poms and sbt.
| * | | Merge pull request #338 from ScrapCodes/ning-upgradePatrick Wendell2014-01-061-1/+1
| |\ \ \ | | | | | | | | | | | | | | | SPARK-1005 Ning upgrade
| | * | | SPARK-1005 Ning upgradePrashant Sharma2014-01-061-1/+1
| | | |/ | | |/|
| * / | Change protobuf version for yarn alpha back to 2.4.1Thomas Graves2014-01-061-1/+0
| |/ /
| * | Using name yarn-alpha/yarn instead of yarn-2.0/yarn-2.2Raymond Liu2014-01-031-2/+2
| | |
| * | Change profile name new-yarn to hadoop2.2-yarnRaymond Liu2014-01-031-1/+1
| | |
| * | Fix pom for yarn code reorgnaize commitRaymond Liu2014-01-031-46/+9
| | |
| * | restore core/pom.xml file modificationliguoqiang2014-01-011-5/+5
| | |
| * | Merge pull request #73 from falaki/ApproximateDistinctCountReynold Xin2013-12-311-0/+5
| |\ \ | | | | | | | | | | | | | | | | | | | | Approximate distinct count Added countApproxDistinct() to RDD and countApproxDistinctByKey() to PairRDDFunctions to approximately count distinct number of elements and distinct number of values per key, respectively. Both functions use HyperLogLog from stream-lib for counting. Both functions take a parameter that controls the trade-off between accuracy and memory consumption. Also added Scala docs and test suites for both methods.
| | * | Using origin versionHossein Falaki2013-12-301-118/+138
| | |\|
| | * | Added stream-lib dependency to Maven buildHossein Falaki2013-10-181-0/+5
| | | |
| * | | upgrade Netty from 4.0.0.Beta2 to 4.0.13.FinalBinh Nguyen2013-12-241-1/+1
| | |/ | |/|
| * | Clean-upPatrick Wendell2013-12-161-0/+1
| | |
| * | CleanupPatrick Wendell2013-12-161-6/+0
| | |
| * | Remove trailing slashes from repository specifications.Patrick Wendell2013-12-161-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The correct format is to not have a trailing slash. For me this caused non-deterministic failures due to issues fetching certain artifacts. The issue was that some of the maven caches would fail to fetch the artifact (due to the way that the artifact path was concatenated with the repository) and this short-circuited the download process in a silent way. Here is what the log output looked like: Downloading: http://repo.maven.apache.org/maven2/org/spark-project/akka/akka-remote_2.10/2.2.3-shaded-protobuf/akka-remote_2.10-2.2.3-shaded-protobuf.pom [WARNING] The POM for org.spark-project.akka:akka-remote_2.10:jar:2.2.3-shaded-protobuf is missing, no dependency information available This was pretty brutal to debug since there was no error message anywhere and the path *looks* correct as reported by the Maven log.
| * | Attempt with extra repositoriesPatrick Wendell2013-12-161-33/+43
| | |
| * | Use scala.binary.version in POMsMark Hamstra2013-12-151-8/+9
| | |
| * | Fix maven build issues in 2.10 branchPatrick Wendell2013-12-131-0/+4
| | |
| * | Disabled yarn 2.2 and added a message in the sbt buildPrashant Sharma2013-12-121-30/+30
| | |
| * | Merge branch 'master' into akka-bug-fixPrashant Sharma2013-12-111-9/+52
| |\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala pom.xml project/SparkBuild.scala streaming/pom.xml yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
| | * | Fix pom.xml for maven buildRaymond Liu2013-12-031-9/+52
| | | |
| * | | Style fixes and addressed review comments at #221Prashant Sharma2013-12-101-9/+8
| | | |
| * | | Incorporated Patrick's feedback comment on #211 and made maven ↵Prashant Sharma2013-12-071-51/+5
| | | | | | | | | | | | | | | | build/dep-resolution atleast a bit faster.
| * | | Merge branch 'master' into scala-2.10-wipPrashant Sharma2013-11-251-0/+5
| |\| | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/rdd/RDD.scala project/SparkBuild.scala
| * | | Merge branch 'master' into scala-2.10Raymond Liu2013-11-141-0/+6
| |\ \ \
| * \ \ \ Merge branch 'master' into scala-2.10Raymond Liu2013-11-131-45/+81
| |\ \ \ \
| * \ \ \ \ Merge branch 'scala-2.10' of github.com:ScrapCodes/spark into scala-2.10Prashant Sharma2013-10-101-3/+3
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala project/SparkBuild.scala
| | * \ \ \ \ Merge branch 'master' into wip-merge-masterPrashant Sharma2013-10-081-1/+2
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: bagel/pom.xml core/pom.xml core/src/test/scala/org/apache/spark/ui/UISuite.scala examples/pom.xml mllib/pom.xml pom.xml project/SparkBuild.scala repl/pom.xml streaming/pom.xml tools/pom.xml In scala 2.10, a shorter representation is used for naming artifacts so changed to shorter scala version for artifacts and made it a property in pom.
| | * \ \ \ \ \ Merge branch 'master' into scala-2.10Prashant Sharma2013-10-011-2/+1
| | |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressUI.scala docs/_config.yml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * | | | | | | | scala 2.10 requires Java 1.6,Martin Weindel2013-10-051-3/+9
| |/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | using Scala 2.10.3, resolved maven-scala-plugin warning
| * | | | | | | Sync with master and some build fixesPrashant Sharma2013-09-261-1/+2
| |\ \ \ \ \ \ \
| * | | | | | | | fixed maven build for scala 2.10Prashant Sharma2013-09-261-24/+18
| | | | | | | | |
| * | | | | | | | version changed 2.9.3 -> 2.10 in shell script.Prashant Sharma2013-09-151-8/+0
| | | | | | | | |
| * | | | | | | | Merge branch 'master' of git://github.com/mesos/spark into scala-2.10Prashant Sharma2013-09-151-127/+107
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/SparkContext.scala project/SparkBuild.scala
| * \ \ \ \ \ \ \ \ Merged with masterPrashant Sharma2013-09-061-90/+254
| |\ \ \ \ \ \ \ \ \
| * \ \ \ \ \ \ \ \ \ Merge branch 'master' into master-mergePrashant Sharma2013-07-121-53/+19
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: README.md core/pom.xml core/src/main/scala/spark/deploy/JsonProtocol.scala core/src/main/scala/spark/deploy/LocalSparkCluster.scala core/src/main/scala/spark/deploy/master/Master.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala pom.xml project/SparkBuild.scala streaming/src/main/scala/spark/streaming/receivers/ActorReceiver.scala
| * | | | | | | | | | | Removed some unnecessary code and fixed dependenciesPrashant Sharma2013-07-111-1/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Merge branch 'master' into master-mergePrashant Sharma2013-07-031-18/+90
| |\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/RDDCheckpointData.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/Utils.scala core/src/main/scala/spark/api/python/PythonRDD.scala core/src/main/scala/spark/deploy/client/Client.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/ZippedRDD.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockManagerMaster.scala core/src/main/scala/spark/storage/BlockManagerMasterActor.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala core/src/test/scala/spark/SizeEstimatorSuite.scala pom.xml project/SparkBuild.scala repl/src/main/scala/spark/repl/SparkILoop.scala repl/src/test/scala/spark/repl/ReplSuite.scala streaming/src/main/scala/spark/streaming/StreamingContext.scala streaming/src/main/scala/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/util/MasterFailureTest.scala
| * | | | | | | | | | | | Fixed maven build without netty fixPrashant Sharma2013-06-141-3/+3
| | | | | | | | | | | | |
| * | | | | | | | | | | | Updated to latest stable scala 2.10.1 and akka 2.1.2Prashant Sharma2013-05-011-1/+1
| | | | | | | | | | | | |
| * | | | | | | | | | | | Manually merged scala-2.10 and masterPrashant Sharma2013-04-221-20/+45
| |\ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | | Updated maven build for scala 2.10.folone2013-01-201-26/+31
| | | | | | | | | | | | | |
* | | | | | | | | | | | | | Merge branch 'master' of github.com:apache/incubator-sparkReynold Xin2013-11-251-0/+11
|\ \ \ \ \ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|_|_|_|_|_|_|/ / | |/| | | | | | | | | | | |
| * | | | | | | | | | | | | Fix Maven build for metrics-graphiteLiGuoqiang2013-11-251-0/+5
| | |_|_|_|_|_|_|_|_|_|/ / | |/| | | | | | | | | | |
| * | | | | | | | | | | | Allow spark on yarn to be run from HDFS. Allows the spark.jar, app.jar, and ↵tgravescs2013-11-041-0/+6
| | |_|_|_|_|_|_|_|_|/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | log4j.properties to be put into hdfs.
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-18 21:00:22 +0000