1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
|
---
layout: post
title: Spark Release 1.3.1
categories: []
tags: []
status: publish
type: post
published: true
meta:
_edit_last: '4'
_wpas_done_all: '1'
---
Spark 1.3.1 is a maintenance release containing stability fixes. This release is based on the [branch-1.3](https://github.com/apache/spark/tree/branch-1.3) maintenance branch of Spark. We recommend all 1.3.0 users to upgrade to this stable release. Contributions to this release came from 60 developers.
To download Spark 1.3.1 visit the <a href="{{site.url}}downloads.html">downloads</a> page.
### Fixes
Spark 1.3.1 contains several bug fixes in Spark SQL and assorted fixes in other components. Some of the more important fixes are highlighted below. You can visit the [Spark issue tracker](https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20fixVersion%20%3D%201.3.1%20ORDER%20BY%20priority%2C%20component) for the full list of fixes.
#### Spark SQL
* Unable to use reserved words in DDL ([SPARK-6250](http://issues.apache.org/jira/browse/SPARK-6250))
* Parquet no longer caches metadata ([SPARK-6575](http://issues.apache.org/jira/browse/SPARK-6575))
* Bug when joining two Parquet tables ([SPARK-6851](http://issues.apache.org/jira/browse/SPARK-6851))
* Unable to read parquet data generated by Spark 1.1.1 ([SPARK-6315](http://issues.apache.org/jira/browse/SPARK-6315))
* Parquet data source may use wrong Hadoop FileSystem ([SPARK-6330](http://issues.apache.org/jira/browse/SPARK-6330))
#### Spark Streaming
* Potential for data loss during WAL recovery ([SPARK-6222](http://issues.apache.org/jira/browse/SPARK-6222))
#### PySpark
* Potential program hang when calling collect ([SPARK-6667](http://issues.apache.org/jira/browse/SPARK-6667))
#### Spark Core
* Thread safety issue in Netty shuffle ([SPARK-6578](http://issues.apache.org/jira/browse/SPARK-6578))
* Memory leak in output committer map ([SPARK-6737](http://issues.apache.org/jira/browse/SPARK-6737))
* Unable to perform fetch file when local directories run NFS ([SPARK-6313](http://issues.apache.org/jira/browse/SPARK-6313))
* NPE when cancelling and using mix of job groups ([SPARK-6414](http://issues.apache.org/jira/browse/SPARK-6414))
### Contributors
The following developers contributed to this release:
* Adam Budde -- Bug fixes in SQL
* Andrew Or -- Bug fixes in Core
* Andrey Zagrebin -- Improvement in SQL
* Bill Chambers -- Documentation in Core
* Cheng Lian -- Bug fixes and improvement in SQL
* Chet Mancini -- Improvements in Core and SQL
* Christophe Preaud -- Documentation in Core and YARN
* Daoyuan Wang -- New features in SQL
* Davies Liu -- Improvements in PySpark and SQL; bug fixes in tests, PySpark, and SQL; improvement in SQL
* Dean Chen -- Bug fixes in Core
* Doing Done -- Bug fixes in Core and SQL
* Hung Lin -- Bug fixes in scheduler
* Ilya Ganelin -- Improvements in Core
* Imran Rashid -- Bug fixes in Core
* Iulian Dragos -- Bug fixes in Core
* Jayson Sunshine -- Documentation in Core
* Jeremy Freeman -- Bug fixes in Streaming and MLlib
* Jongyoul Lee -- Improvements in Mesos; bug fixes in Core
* Joseph K. Bradley -- Documentation in PySpark, Streaming, SQL, MLlib, and Core
* Josh Rosen -- Improvements in Core; bug fixes in Java API, Core, scheduler, and Streaming
* Kai Sasaki -- Documentation in Core and MLlib; bug fixes in MLlib and PySpark
* Kalle Jepsen -- Improvements in PySpark
* Kamil Smuga -- Bug fixes in Core and PySpark
* Kay Ousterhout -- Bug fixes in Core, tests, and Web UI
* Kevin (Sangwoo) Kim -- Bug fixes in Core
* Kousuke Saruta -- Improvements in Streaming and tests
* Lev Khomich -- Improvements in Core
* Liang-Chi Hsieh -- Bug fixes in SQL
* Liangliang Gu -- Bug fixes in spark submit
* Lomig Megard -- Documentation in Core
* Marcelo Vanzin -- Bug fixes in Core and YARN
* Matt Aasted -- Bug fixes in EC2
* Michael Armbrust -- Improvements in SQL; documentation in Core; bug fixes in SQL; improvement in Core and SQL
* Michael Griffiths -- Bug fixes in Windows and Core
* Milan Straka -- Bug fixes in PySpark
* Nan Zhu -- Bug fixes in Core and SQL
* Nathan McCarthy -- Bug fixes in Core
* Pei-Lun Lee -- Bug fixes in SQL
* Peter Parente -- Improvements in Core
* Peter Rudenko -- Documentation in Core
* Reynold Xin -- Improvements in Core; documentation in Core; bug fixes in Core; improvement in SQL
* Sean Owen -- Bug fixes in Core, tests, and SQL
* Shixiong Zhu -- Bug fixes in Core
* Tathagata Das -- Improvements in Core and Streaming; bug fixes in Streaming
* Thomas Graves -- Bug fixes in Core
* Tijo Thomas -- Bug fixes in Core and SQL
* Venkata Ramana Gollamudi -- Bug fixes in SQL
* Vinod KC -- Bug fixes in Core and SQL
* Volodymyr Lyubinets -- Improvements and bug fixes in SQL
* Xiangrui Meng -- New features in MLlib and PySpark; bug fixes in PySpark, MLlib, and SQL; documentation in Core and MLlib
* Yadong Qi -- Improvements in SQL
* Yanbo Liang -- Bug fixes in MLlib and SQL
* Yash Datta -- Improvements in SQL
* Yin Huai -- Improvements and bug fixes in SQL
* Yp Cat -- Bug fixes in SQL
* Yu ISHIKAWA -- Improvements in MLlib
* Yuri Saito -- Bug fixes in SQL
* Zhang, Liye -- Bug fixes in Core and Web UI
* Zhichao Li -- Bug fixes in Streaming and Web UI
* Zhichao Zhang -- Documentation in Core
_Thanks to everyone who contributed!_
|
