diff options
| author | Liwei Lin <lwlin7@gmail.com> | 2016-06-10 13:01:29 -0700 |
|---|---|---|
| committer | Shixiong Zhu <shixiong@databricks.com> | 2016-06-10 13:01:29 -0700 |
| commit | fb219029dd1b8d2783c3e202361401048296595c (patch) | |
| tree | 3a1668e08c9204bf30c6c471b0268be691fa7691 /repl/scala-2.10 | |
| parent | 5c16ad0d522e5124a6977533077afb7b38fc42a1 (diff) | |
| download | spark-fb219029dd1b8d2783c3e202361401048296595c.tar.gz spark-fb219029dd1b8d2783c3e202361401048296595c.tar.bz2 spark-fb219029dd1b8d2783c3e202361401048296595c.zip | |
[SPARK-15871][SQL] Add `assertNotPartitioned` check in `DataFrameWriter`
## What changes were proposed in this pull request?
It doesn't make sense to specify partitioning parameters, when we write data out from Datasets/DataFrames into `jdbc` tables or streaming `ForeachWriter`s.
This patch adds `assertNotPartitioned` check in `DataFrameWriter`.
<table>
<tr>
<td align="center"><strong>operation</strong></td>
<td align="center"><strong>should check not partitioned?</strong></td>
</tr>
<tr>
<td align="center">mode</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">outputMode</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">trigger</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">format</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">option/options</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">partitionBy</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">bucketBy</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">sortBy</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">save</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">queryName</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">startStream</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">foreach</td>
<td align="center">yes</td>
</tr>
<tr>
<td align="center">insertInto</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">saveAsTable</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">jdbc</td>
<td align="center">yes</td>
</tr>
<tr>
<td align="center">json</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">parquet</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">orc</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">text</td>
<td align="center"></td>
</tr>
<tr>
<td align="center">csv</td>
<td align="center"></td>
</tr>
</table>
## How was this patch tested?
New dedicated tests.
Author: Liwei Lin <lwlin7@gmail.com>
Closes #13597 from lw-lin/add-assertNotPartitioned.
Diffstat (limited to 'repl/scala-2.10')
0 files changed, 0 insertions, 0 deletions