aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark/files.py
Commit message (Collapse)AuthorAgeFilesLines
* Add Apache license headers and LICENSE and NOTICE filesMatei Zaharia2013-07-161-0/+17
|
* Do not launch JavaGateways on workers (SPARK-674).Josh Rosen2013-02-011-1/+1
| | | | | | | | | | | The problem was that the gateway was being initialized whenever the pyspark.context module was loaded. The fix uses lazy initialization that occurs only when SparkContext instances are actually constructed. I also made the gateway and jvm variables private. This change results in ~3-4x performance improvement when running the PySpark unit tests.
* Allow PySpark's SparkFiles to be used from driverJosh Rosen2013-01-231-3/+17
| | | | Fix minor documentation formatting issues.
* Don't download files to master's working directory.Josh Rosen2013-01-211-0/+24
This should avoid exceptions caused by existing files with different contents. I also removed some unused code.