Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Merge pull request #677 from jerryshao/fix_stage_clean | Matei Zaharia | 2013-07-04 | 1 | -2/+6 |
|\ | | | | | Clean StageToInfos periodically when spark.cleaner.ttl is enabled | ||||
| * | Clean StageToInfos periodically when spark.cleaner.ttl is enabled | jerryshao | 2013-07-05 | 1 | -2/+6 |
|/ | |||||
* | Merge pull request #666 from c0s/master | Matei Zaharia | 2013-07-01 | 1 | -0/+5 |
|\ | | | | | hbase dependency is missed in hadoop2-yarn profile of examples module | ||||
| * | Fixing missed hbase dependency in examples hadoop2-yarn profile | Konstantin Boudnik | 2013-07-01 | 1 | -0/+5 |
|/ | |||||
* | Clarify that PySpark is not supported on Windows | root | 2013-07-01 | 1 | -3/+2 |
| | |||||
* | Fixed PySpark perf regression by not using socket.makefile(), and improved | root | 2013-07-01 | 3 | -22/+50 |
| | | | | | | | debuggability by letting "print" statements show up in the executor's stderr Conflicts: core/src/main/scala/spark/api/python/PythonRDD.scala | ||||
* | Fix performance bug with new Python code not using buffered streams | root | 2013-07-01 | 2 | -17/+19 |
| | |||||
* | Increase SLF4j version in Maven too | Matei Zaharia | 2013-06-30 | 1 | -1/+1 |
| | |||||
* | Update docs on SCALA_LIBRARY_PATH | Matei Zaharia | 2013-06-30 | 2 | -13/+9 |
| | |||||
* | Made use of spark.executor.memory setting consistent and documented it | Matei Zaharia | 2013-06-30 | 5 | -34/+43 |
| | | | | | | Conflicts: core/src/main/scala/spark/SparkContext.scala | ||||
* | Merge pull request #577 from skumargithub/master | Matei Zaharia | 2013-06-29 | 1 | -0/+50 |
|\ | | | | | Example of cumulative counting using updateStateByKey | ||||
| * | Removed unused code, clarified intent of the program, batch size to 1 second | unknown | 2013-05-06 | 1 | -5/+3 |
| | | |||||
| * | Modified as per TD's suggestions | unknown | 2013-04-30 | 1 | -17/+6 |
| | | |||||
| * | Examaple of cumulative counting using updateStateByKey | unknown | 2013-04-22 | 1 | -0/+63 |
| | | |||||
* | | Remove Twitter4J specific repo since it's in Maven central | Matei Zaharia | 2013-06-29 | 2 | -13/+1 |
| | | |||||
* | | Initialize Twitter4J OAuth from system properties instead of prompting | Matei Zaharia | 2013-06-29 | 4 | -38/+23 |
| | | |||||
* | | Merge remote-tracking branch 'mrpotes/master' | Matei Zaharia | 2013-06-29 | 6 | -50/+97 |
|\ \ | |||||
| * | | Fix usage and parameter extraction | James Phillpotts | 2013-06-25 | 3 | -12/+9 |
| | | | |||||
| * | | Include a default OAuth implementation, and update examples and ↵ | James Phillpotts | 2013-06-25 | 6 | -28/+81 |
| | | | | | | | | | | | | JavaStreamingContext | ||||
| * | | Twitter API v1 is retired - username/password auth no longer possible | James Phillpotts | 2013-06-24 | 1 | -15/+1 |
| | | | |||||
| * | | Allow other twitter authorizations than username/password | James Phillpotts | 2013-06-21 | 2 | -9/+20 |
| | | | |||||
* | | | Merge pull request #664 from pwendell/test-fix | Matei Zaharia | 2013-06-27 | 1 | -1/+0 |
|\ \ \ | | | | | | | | | Removing incorrect test statement | ||||
| * | | | Removing incorrect test statement | Patrick Wendell | 2013-06-27 | 1 | -1/+0 |
| | | | | |||||
* | | | | Look at JAVA_HOME before PATH to determine Java executable | Matei Zaharia | 2013-06-27 | 1 | -4/+5 |
| | | | | |||||
* | | | | Simplify Python docs a little to do substring search | Matei Zaharia | 2013-06-26 | 1 | -4/+3 |
| | | | | |||||
* | | | | Fixes to compute-classpath on Windows | Matei Zaharia | 2013-06-26 | 2 | -2/+3 |
| | | | | |||||
* | | | | Merge pull request #663 from stephenh/option_and_getenv | Matei Zaharia | 2013-06-26 | 1 | -17/+6 |
|\ \ \ \ | | | | | | | | | | | Be cute with Option and getenv. | ||||
| * | | | | Wrap lines. | Stephen Haberman | 2013-06-26 | 1 | -1/+3 |
| | | | | | |||||
| * | | | | Be cute with Option and getenv. | Stephen Haberman | 2013-06-26 | 1 | -17/+4 |
|/ / / / | |||||
* | | | | Don't use forward slash in exclusion for JAR signature files | Matei Zaharia | 2013-06-25 | 1 | -1/+1 |
| | | | | |||||
* | | | | Refactored tests to share SparkContexts in some of them | Matei Zaharia | 2013-06-25 | 10 | -425/+373 |
| | | | | | | | | | | | | | | | | | | | | Creating these seems to take a while and clutters the output with Akka stuff, so it would be nice to share them. | ||||
* | | | | Formatting | Matei Zaharia | 2013-06-25 | 1 | -3/+4 |
| | | | | |||||
* | | | | Added a local-cluster mode test to ReplSuite | Matei Zaharia | 2013-06-25 | 1 | -5/+26 |
| | | | | |||||
* | | | | Fix computation of classpath when we launch java directly | Matei Zaharia | 2013-06-25 | 6 | -107/+189 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous version assumed that a CLASSPATH environment variable was set by the "run" script when launching the process that starts the ExecutorRunner, but unfortunately this is not true in tests. Instead, we factor the classpath calculation into an extenral script and call that. NOTE: This includes a Windows version but hasn't yet been tested there. | ||||
* | | | | Some fixes to the launch-java-directly change: | Matei Zaharia | 2013-06-25 | 3 | -31/+138 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Split SPARK_JAVA_OPTS into multiple command-line arguments if it contains spaces; this splitting follows quoting rules in bash - Add the Scala JARs to the classpath if they're not in the CLASSPATH variable because the ExecutorRunner is launched with "scala" (this can happen when using local-cluster URLs in spark-shell) | ||||
* | | | | Fixed deprecated use of expect in SizeEstimatorSuite | Matei Zaharia | 2013-06-25 | 1 | -36/+36 |
| | | | | |||||
* | | | | Merge remote-tracking branch 'cgrothaus/SPARK-698' | Matei Zaharia | 2013-06-25 | 3 | -13/+45 |
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | Conflicts: run | ||||
| * | | | | Incorporate feedback from mateiz: | Christoph Grothaus | 2013-02-24 | 3 | -25/+16 |
| | | | | | | | | | | | | | | | | | | | | | | | | | - we do not need getEnvOrEmpty - Instead of saving SPARK_NONDAEMON_JAVA_OPTS, it would be better to modify the scripts to use a different variable name for the JAVA_OPTS they do eventually use | ||||
| * | | | | Fix SPARK-698. From ExecutorRunner, launch java directly instead via the run ↵ | Christoph Grothaus | 2013-02-20 | 3 | -4/+45 |
| | | | | | | | | | | | | | | | | | | | | scripts. | ||||
* | | | | | Merge pull request #661 from mesos/streaming | Matei Zaharia | 2013-06-25 | 12 | -103/+148 |
|\ \ \ \ \ | | | | | | | | | | | | | Kafka fixes and DStream.count fix for master | ||||
| * \ \ \ \ | Merge branch 'master' into streaming | Tathagata Das | 2013-06-24 | 289 | -3293/+13250 |
| |\ \ \ \ \ | |/ / / / / |/| | | | | | | | | | | | | | | | | | Conflicts: .gitignore | ||||
* | | | | | | Clone the zero value for each key in foldByKey | Matei Zaharia | 2013-06-23 | 2 | -3/+34 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The old version reused the object within each task, leading to overwriting of the object when a mutable type is used, which is expected to be common in fold. Conflicts: core/src/test/scala/spark/ShuffleSuite.scala | ||||
* | | | | | | Fix search path for REPL class loader to really find added JARs | Matei Zaharia | 2013-06-22 | 2 | -17/+25 |
| | | | | | | |||||
* | | | | | | ADD_JARS environment variable for spark-shell | Matei Zaharia | 2013-06-22 | 2 | -4/+15 |
| | | | | | | |||||
* | | | | | | Merge pull request #648 from shivaram/netty-dbg | Matei Zaharia | 2013-06-22 | 7 | -86/+151 |
|\ \ \ \ \ \ | |_|_|/ / / |/| | | | | | Shuffle fixes and cleanup | ||||
| * | | | | | Fix some comments and style | Shivaram Venkataraman | 2013-06-13 | 5 | -14/+8 |
| | | | | | | |||||
| * | | | | | Merge branch 'netty-dbg' of github.com:shivaram/spark into netty-dbg | Shivaram Venkataraman | 2013-06-12 | 4 | -39/+94 |
| |\ \ \ \ \ | |||||
| | * | | | | | Clean up variables and counters in BlockFetcherIterator | Shivaram Venkataraman | 2013-06-06 | 1 | -23/+31 |
| | | | | | | | |||||
| | * | | | | | Pass in bufferSize to BufferedOutputStream | Shivaram Venkataraman | 2013-06-05 | 1 | -1/+1 |
| | | | | | | | |||||
| | * | | | | | Don't write zero block files with java serializer | Shivaram Venkataraman | 2013-06-05 | 4 | -18/+61 |
| | | | | | | |