aboutsummaryrefslogtreecommitdiff
path: root/repl
Commit message (Collapse)AuthorAgeFilesLines
* Revert GraphX changes to SparkILoopInitAnkur Dave2014-01-101-10/+2
| | | | | | The changes were to support a custom banner in spark-shell for use by graphx-shell, but once GraphX is merged into Spark, a separate shell will be unnecessary.
* Merge remote-tracking branch 'spark-upstream/master' into HEADAnkur Dave2014-01-0815-1570/+2045
|\ | | | | | | | | | | | | | | | | | | | | Conflicts: README.md core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala core/src/main/scala/org/apache/spark/util/collection/PrimitiveKeyOpenHashMap.scala pom.xml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * Merge pull request #327 from lucarosellini/masterMatei Zaharia2014-01-083-3/+73
| |\ | | | | | | | | | | | | | | | | | | | | | | | | Added ‘-i’ command line option to Spark REPL We had to create a new implementation of both scala.tools.nsc.CompilerCommand and scala.tools.nsc.Settings, because using scala.tools.nsc.GenericRunnerSettings would bring in other options (-howtorun, -save and -execute) which don’t make sense in Spark. Any new Spark specific command line option could now be added to org.apache.spark.repl.SparkRunnerSettings class. Since the behavior of loading a script from the command line should be the same as loading it using the “:load” command inside the shell, the script should be loaded when the SparkContext is available, that’s why we had to move the call to ‘loadfiles(settings)’ _after_ the call to postInitialization(). This still doesn’t work if ‘isAsync = true’.
| | * Added license header and removed @author tagLuca Rosellini2014-01-072-4/+34
| | |
| | * Added ‘-i’ command line option to spark REPL.Luca Rosellini2014-01-033-3/+43
| | | | | | | | | | | | | | | | | | | | | We had to create a new implementation of both scala.tools.nsc.CompilerCommand and scala.tools.nsc.Settings, because using scala.tools.nsc.GenericRunnerSettings would bring in other options (-howtorun, -save and -execute) which don’t make sense in Spark. Any new Spark specific command line option could now be added to org.apache.spark.repl.SparkRunnerSettings class. Since the behavior of loading a script from the command line should be the same as loading it using the “:load” command inside the shell, the script should be loaded when the SparkContext is available, that’s why we had to move the call to ‘loadfiles(settings)’ _after_ the call to postInitialization(). This still doesn’t work if ‘isAsync = true’.
| * | Merge remote-tracking branch 'apache-github/master' into remove-binariesPatrick Wendell2014-01-031-1/+0
| |\ \ | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/test/scala/org/apache/spark/DriverSuite.scala docs/python-programming-guide.md
| | * \ Merge branch 'scripts-reorg' of github.com:shane-huang/incubator-spark into ↵Prashant Sharma2014-01-021-1/+0
| | |\ \ | | | |/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | spark-915-segregate-scripts Conflicts: bin/spark-shell core/pom.xml core/src/main/scala/org/apache/spark/SparkContext.scala core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala core/src/main/scala/org/apache/spark/ui/UIWorkloadGenerator.scala core/src/test/scala/org/apache/spark/DriverSuite.scala python/run-tests sbin/compute-classpath.sh sbin/spark-class sbin/stop-slaves.sh
| | | * deprecate "spark" script and SPAKR_CLASSPATH environment variableAndrew xia2013-10-121-1/+0
| | | |
| * | | fixed review commentsPrashant Sharma2014-01-031-1/+3
| |/ /
| * | Miscellaneous fixes from code review.Matei Zaharia2014-01-011-1/+1
| | | | | | | | | | | | | | | | | | Also replaced SparkConf.getOrElse with just a "get" that takes a default value, and added getInt, getLong, etc to make code that uses this simpler later on.
| * | Various fixes to configuration codeMatei Zaharia2013-12-282-7/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Got rid of global SparkContext.globalConf - Pass SparkConf to serializers and compression codecs - Made SparkConf public instead of private[spark] - Improved API of SparkContext and SparkConf - Switched executor environment vars to be passed through SparkConf - Fixed some places that were still using system properties - Fixed some tests, though others are still failing This still fails several tests in core, repl and streaming, likely due to properties not being set or cleared correctly (some of the tests run fine in isolation).
| * | spark-544, introducing SparkConf and related configuration overhaul.Prashant Sharma2013-12-252-8/+6
| | |
| * | Use scala.binary.version in POMsMark Hamstra2013-12-151-7/+7
| | |
| * | Review comments on the PR for scala 2.10 migration.Prashant Sharma2013-12-131-1/+0
| | |
| * | Style fixes and addressed review comments at #221Prashant Sharma2013-12-101-7/+7
| | |
| * | Incorporated Patrick's feedback comment on #211 and made maven ↵Prashant Sharma2013-12-071-1/+1
| | | | | | | | | | | | build/dep-resolution atleast a bit faster.
| * | Fixed compile time warnings and formatting post merge.Prashant Sharma2013-11-261-65/+74
| | |
| * | Various merge correctionsAaron Davidson2013-11-142-12/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've diff'd this patch against my own -- since they were both created independently, this means that two sets of eyes have gone over all the merge conflicts that were created, so I'm feeling significantly more confident in the resulting PR. @rxin has looked at the changes to the repl and is resoundingly confident that they are correct.
| * | Merge branch 'master' into scala-2.10Raymond Liu2013-11-141-2/+34
| |\ \
| * \ \ Merge branch 'master' into scala-2.10Raymond Liu2013-11-131-7/+29
| |\ \ \
| * \ \ \ Merge branch 'master' into wip-merge-masterPrashant Sharma2013-10-081-8/+8
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: bagel/pom.xml core/pom.xml core/src/test/scala/org/apache/spark/ui/UISuite.scala examples/pom.xml mllib/pom.xml pom.xml project/SparkBuild.scala repl/pom.xml streaming/pom.xml tools/pom.xml In scala 2.10, a shorter representation is used for naming artifacts so changed to shorter scala version for artifacts and made it a property in pom.
| * \ \ \ \ Merge branch 'master' into scala-2.10Prashant Sharma2013-10-012-2/+2
| |\ \ \ \ \ | | | |_|_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressUI.scala docs/_config.yml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * | | | | fixed maven build for scala 2.10Prashant Sharma2013-09-261-6/+6
| | | | | |
| * | | | | ported repl improvements from masterPrashant Sharma2013-09-152-2/+11
| | | | | |
| * | | | | Merge branch 'master' of git://github.com/mesos/spark into scala-2.10Prashant Sharma2013-09-151-12/+0
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/SparkContext.scala project/SparkBuild.scala
| * | | | | | Fixed repl suitePrashant Sharma2013-09-152-7/+7
| | | | | | |
| * | | | | | Few more fixes to tests broken during mergePrashant Sharma2013-09-101-50/+0
| | | | | | |
| * | | | | | Merged with masterPrashant Sharma2013-09-0618-372/+369
| |\ \ \ \ \ \
| * | | | | | | Added add jars functionality to new repl, which was dropped while merging ↵Prashant Sharma2013-07-123-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | with old.
| * | | | | | | Removed an unnecessary test casePrashant Sharma2013-07-111-101/+0
| | | | | | | |
| * | | | | | | Merge branch 'master' into master-mergePrashant Sharma2013-07-033-14/+91
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/RDDCheckpointData.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/Utils.scala core/src/main/scala/spark/api/python/PythonRDD.scala core/src/main/scala/spark/deploy/client/Client.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/ZippedRDD.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockManagerMaster.scala core/src/main/scala/spark/storage/BlockManagerMasterActor.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala core/src/test/scala/spark/SizeEstimatorSuite.scala pom.xml project/SparkBuild.scala repl/src/main/scala/spark/repl/SparkILoop.scala repl/src/test/scala/spark/repl/ReplSuite.scala streaming/src/main/scala/spark/streaming/StreamingContext.scala streaming/src/main/scala/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/util/MasterFailureTest.scala
| * \ \ \ \ \ \ \ Merge pull request #652 from ScrapCodes/scala-2.10Matei Zaharia2013-06-141-0/+2
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Fixed maven build without netty fix
| | * | | | | | | | Fixed maven build without netty fixPrashant Sharma2013-06-141-0/+2
| | | | | | | | | |
| * | | | | | | | | Fix StandaloneClusterReplSuite to allow running multiple testsMatei Zaharia2013-06-082-16/+5
| |/ / / / / / / /
| * | | | | | | | Fixied other warningsPrashant Sharma2013-04-291-1/+1
| | | | | | | | |
| * | | | | | | | scala 2.10 and master mergePrashant Sharma2013-04-243-16/+2
| | | | | | | | |
| * | | | | | | | Manually merged scala-2.10 and masterPrashant Sharma2013-04-223-16/+44
| |\ \ \ \ \ \ \ \
| * | | | | | | | | Added more testsPrashant Sharma2013-04-193-12/+84
| | | | | | | | | |
| * | | | | | | | | Changed spark context as lazy val to become just val.Prashant Sharma2013-04-162-3/+5
| | | | | | | | | |
| * | | | | | | | | Added standalone cluster repl suitePrashant Sharma2013-04-153-47/+100
| | | | | | | | | |
| * | | | | | | | | Added class wrappers instead of object and incorporated most of matei commentsPrashant Sharma2013-04-102-20/+39
| | | | | | | | | |
| * | | | | | | | | Made shell loading synchronous as async loading confuses with out of order ↵Prashant Sharma2013-04-101-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | status messages
| * | | | | | | | | Fixed broken tests by last commit for repl.Prashant Sharma2013-03-251-8/+7
| | | | | | | | | |
| * | | | | | | | | Ctrl-D hang bug fixed!Prashant Sharma2013-03-202-1/+13
| | | | | | | | | |
| * | | | | | | | | fixed autocompletion apparent hang due to loggingprashant2013-03-193-8/+9
| | | | | | | | | |
| * | | | | | | | | porting of repl to scala-2.10Prashant Sharma2013-03-1710-1507/+1895
| | | | | | | | | |
* | | | | | | | | | Merge branch 'master' of github.com:apache/incubator-sparkReynold Xin2013-11-252-4/+42
|\ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|_|_|_|/ | |/| | | | | | | |
| * | | | | | | | | Propagate the SparkContext local property from the thread that calls the ↵Reynold Xin2013-11-092-4/+42
| | |_|_|_|_|_|_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | spark-repl to the actual execution thread.
* / | | | | | | | This commit adds a new graphx-shell which is essentially the same asJoseph E. Gonzalez2013-11-041-83/+91
|/ / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the spark shell but with GraphX packages automatically imported and with Kryo serialization enabled for GraphX types. In addition the graphx-shell has a nifty new logo. To make these changes minimally invasive in the SparkILoop.scala I added some additional environment variables: SPARK_BANNER_TEXT: If set this string is displayed instead of the spark logo SPARK_SHELL_INIT_BLOCK: if set this expression is evaluated in the spark shell after the spark context is created.
* | | | | | | | Makes Spark SIMR ready.Ali Ghodsi2013-10-241-0/+14
| | | | | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-18 20:20:26 +0000