diff options
| author | Liang-Chi Hsieh <viirya@gmail.com> | 2015-04-15 13:01:29 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-04-15 13:02:25 -0700 |
| commit | e3e4e9a38b25174ed8bb460ba2b375813ebf3b4b (patch) | |
| tree | 94345d56d01146bfe44167ba29d1d6ac2f69a1a2 | |
| parent | b75b3070740803480d235b0c9a86673721344f30 (diff) | |
| download | spark-e3e4e9a38b25174ed8bb460ba2b375813ebf3b4b.tar.gz spark-e3e4e9a38b25174ed8bb460ba2b375813ebf3b4b.tar.bz2 spark-e3e4e9a38b25174ed8bb460ba2b375813ebf3b4b.zip | |
[SPARK-6800][SQL] Update doc for JDBCRelation's columnPartition
JIRA https://issues.apache.org/jira/browse/SPARK-6800
Author: Liang-Chi Hsieh <viirya@gmail.com>
Closes #5488 from viirya/fix_jdbc_where and squashes the following commits:
51386c8 [Liang-Chi Hsieh] Update code comment.
1dcc929 [Liang-Chi Hsieh] Update document.
3eb74d6 [Liang-Chi Hsieh] Revert and modify doc.
df11783 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into fix_jdbc_where
3e7db15 [Liang-Chi Hsieh] Fix wrong logic to generate WHERE clause for JDBC.
| -rw-r--r-- | docs/sql-programming-guide.md | 5 | ||||
| -rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala | 4 |
2 files changed, 6 insertions, 3 deletions
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 332618edf0..03500867df 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -1371,7 +1371,10 @@ the Data Sources API. The following options are supported: <td> These options must all be specified if any of them is specified. They describe how to partition the table when reading in parallel from multiple workers. - <code>partitionColumn</code> must be a numeric column from the table in question. + <code>partitionColumn</code> must be a numeric column from the table in question. Notice + that <code>lowerBound</code> and <code>upperBound</code> are just used to decide the + partition stride, not for filtering the rows in table. So all rows in the table will be + partitioned and returned. </td> </tr> </table> diff --git a/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala b/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala index c25ef58e6f..b237fe684c 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala @@ -873,8 +873,8 @@ class SQLContext(@transient val sparkContext: SparkContext) * passed to this function. * * @param columnName the name of a column of integral type that will be used for partitioning. - * @param lowerBound the minimum value of `columnName` to retrieve - * @param upperBound the maximum value of `columnName` to retrieve + * @param lowerBound the minimum value of `columnName` used to decide partition stride + * @param upperBound the maximum value of `columnName` used to decide partition stride * @param numPartitions the number of partitions. the range `minValue`-`maxValue` will be split * evenly into this many partitions * |