diff options
author | Kousuke Saruta <sarutak@oss.nttdata.co.jp> | 2015-12-18 14:05:06 -0800 |
---|---|---|
committer | Michael Armbrust <michael@databricks.com> | 2015-12-18 14:05:06 -0800 |
commit | 6eba655259d2bcea27d0147b37d5d1e476e85422 (patch) | |
tree | f25a36ec77cb23da0ec848fa5b0b2f72cc8cb07d /sbin/stop-slaves.sh | |
parent | 41ee7c57abd9f52065fd7ffb71a8af229603371d (diff) | |
download | spark-6eba655259d2bcea27d0147b37d5d1e476e85422.tar.gz spark-6eba655259d2bcea27d0147b37d5d1e476e85422.tar.bz2 spark-6eba655259d2bcea27d0147b37d5d1e476e85422.zip |
[SPARK-12404][SQL] Ensure objects passed to StaticInvoke is Serializable
Now `StaticInvoke` receives `Any` as a object and `StaticInvoke` can be serialized but sometimes the object passed is not serializable.
For example, following code raises Exception because `RowEncoder#extractorsFor` invoked indirectly makes `StaticInvoke`.
```
case class TimestampContainer(timestamp: java.sql.Timestamp)
val rdd = sc.parallelize(1 to 2).map(_ => TimestampContainer(System.currentTimeMillis))
val df = rdd.toDF
val ds = df.as[TimestampContainer]
val rdd2 = ds.rdd <----------------- invokes extractorsFor indirectory
```
I'll add test cases.
Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Author: Michael Armbrust <michael@databricks.com>
Closes #10357 from sarutak/SPARK-12404.
Diffstat (limited to 'sbin/stop-slaves.sh')
0 files changed, 0 insertions, 0 deletions