spark - Mirror of Apache Spark

	Commit message (Collapse)	Author	Age	Files	Lines
*	Fix finding of assembly JAR, as well as some pointers to ./run	Matei Zaharia	2013-08-29	3	-2/+3
\|
*	Fix PySpark for assembly run and include it in dist	Matei Zaharia	2013-08-29	3	-0/+28
\|
*	Change build and run instructions to use assemblies	Matei Zaharia	2013-08-29	2	-2/+2
\| \| \| \| \| \| \| \| \| \| \| \| \| \| \| \|	This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.
*	Merge pull request #871 from pwendell/expose-local	Patrick Wendell	2013-08-28	1	-1/+1
\|\ \| \| \| \|	Expose `isLocal` in SparkContext.
\| *	Make local variable public	Patrick Wendell	2013-08-28	1	-1/+1
\| \|
* \|	Merge pull request #865 from tgravescs/fixtmpdir	Matei Zaharia	2013-08-28	1	-0/+22
\|\ \ \| \| \| \| \| \|	Spark on Yarn should use yarn approved directories for spark.local.dir and tmp
\| * \|	Change Executor to only look at the env variable SPARK_YARN_MODE	Y.CORP.YAHOO.COM\tgraves	2013-08-28	1	-1/+1
\| \| \|
\| * \|	Updated based on review comments.	Y.CORP.YAHOO.COM\tgraves	2013-08-27	1	-9/+6
\| \| \|
\| * \|	Allow for Executors to have different directories then the Spark Master for Yarn	Y.CORP.YAHOO.COM\tgraves	2013-08-27	1	-0/+25
\| \|/
* \|	Added worker state to the cluster master JSON ui.	Reynold Xin	2013-08-26	1	-1/+2
\| \|
* \|	Revert "Merge pull request #841 from rxin/json"	Reynold Xin	2013-08-26	6	-64/+70
\|/ \| \| \| \|	This reverts commit 1fb1b0992838c8cdd57eec45793e67a0490f1a52, reversing changes made to c69c48947d5102c81a9425cb380d861c3903685c.
*	Merge pull request #854 from markhamstra/pomUpdate	Matei Zaharia	2013-08-22	1	-0/+4
\|\ \| \| \| \|	Synced sbt and maven builds to use the same dependencies, etc.
\| *	Synced sbt and maven builds	Mark Hamstra	2013-08-21	1	-0/+4
\| \|
* \|	Merge pull request #832 from alig/coalesce	Matei Zaharia	2013-08-22	4	-46/+389
\|\ \ \| \|/ \|/\|	Coalesced RDD with locality
\| *	Merged in from upstream to use TaskLocation instead of strings	Ali Ghodsi	2013-08-20	2	-8/+11
\| \|
\| *	added curly braces to make the code more consistent	Ali Ghodsi	2013-08-20	1	-1/+2
\| \|
\| *	indent	Ali Ghodsi	2013-08-20	1	-1/+1
\| \|
\| *	Bug in test fixed	Ali Ghodsi	2013-08-20	1	-3/+3
\| \|
\| *	Added a test to make sure no locality preferences are ignored	Ali Ghodsi	2013-08-20	1	-0/+5
\| \|
\| *	Simpler code	Ali Ghodsi	2013-08-20	2	-5/+4
\| \|
\| *	simpler code	Ali Ghodsi	2013-08-20	1	-16/+7
\| \|
\| *	Fixed almost all of Matei's feedback	Ali Ghodsi	2013-08-20	2	-31/+26
\| \|
\| *	fixed Matei's comments	Ali Ghodsi	2013-08-20	3	-73/+99
\| \|
\| *	making CoalescedRDDPartition public	Ali Ghodsi	2013-08-20	1	-2/+1
\| \|
\| *	comment in the test to make it more understandable	Ali Ghodsi	2013-08-20	1	-1/+1
\| \|
\| *	Coalescer now uses current preferred locations for derived RDDs. Made run() ↵	Ali Ghodsi	2013-08-20	4	-34/+59
\| \| \| \| \| \| \| \|	in DAGScheduler thread safe and added a method to be able to ask it for preferred locations. Added a similar method that wraps the former inside SparkContext.
\| *	added one test that will test a future functionality	Ali Ghodsi	2013-08-20	1	-1/+10
\| \|
\| *	Added error messages to the tests to make failed tests less cryptic	Ali Ghodsi	2013-08-20	1	-7/+7
\| \|
\| *	fixed matei's comments	Ali Ghodsi	2013-08-20	1	-15/+16
\| \|
\| *	Made a function object that returns the coalesced groups	Ali Ghodsi	2013-08-20	1	-30/+35
\| \|
\| *	several of Reynold's suggestions implemented	Ali Ghodsi	2013-08-20	1	-15/+14
\| \|
\| *	space removed	Ali Ghodsi	2013-08-20	1	-1/+1
\| \|
\| *	use count rather than foreach	Ali Ghodsi	2013-08-20	1	-2/+1
\| \|
\| *	made preferredLocation a val of the surrounding case class	Ali Ghodsi	2013-08-20	1	-10/+3
\| \|
\| *	Fix bug in tests	Ali Ghodsi	2013-08-20	2	-6/+6
\| \|
\| *	Renamed split to partition	Ali Ghodsi	2013-08-20	1	-11/+11
\| \|
\| *	word wrap before 100 chars per line	Ali Ghodsi	2013-08-20	2	-41/+51
\| \|
\| *	added goals inline as comment	Ali Ghodsi	2013-08-20	1	-0/+21
\| \|
\| *	Large scale load and locality tests for the coalesced partitions added	Ali Ghodsi	2013-08-20	2	-63/+118
\| \|
\| *	Bug, should compute slack wrt parent partition size, not number of bins	Ali Ghodsi	2013-08-20	1	-2/+2
\| \|
\| *	load balancing coalescer	Ali Ghodsi	2013-08-20	2	-11/+218
\| \|
* \|	Removed meaningless types	Mark Hamstra	2013-08-20	1	-1/+1
\|/
*	Merge remote-tracking branch 'jey/hadoop-agnostic'	Matei Zaharia	2013-08-20	29	-2255/+178
\|\ \| \| \| \| \| \| \| \|	Conflicts: core/src/main/scala/spark/PairRDDFunctions.scala
\| *	Fix Maven build with Hadoop 0.23.9	Jey Kottalam	2013-08-18	1	-0/+8
\| \|
\| *	Maven build now also works with YARN	Jey Kottalam	2013-08-16	1	-70/+0
\| \|
\| *	Don't mark hadoop-client as 'provided'	Jey Kottalam	2013-08-16	1	-1/+0
\| \|
\| *	Maven build now works with CDH hadoop-2.0.0-mr1	Jey Kottalam	2013-08-16	1	-52/+0
\| \|
\| *	Initial changes to make Maven build agnostic of hadoop version	Jey Kottalam	2013-08-16	1	-58/+5
\| \|
\| *	Rename HadoopWriter to SparkHadoopWriter since it's outside of our package	Jey Kottalam	2013-08-15	2	-6/+6
\| \|
\| *	Fix newTaskAttemptID to work under YARN	Jey Kottalam	2013-08-15	1	-1/+19
\| \|