diff options
| author | sureshthalamati <suresh.thalamati@gmail.com> | 2015-09-15 19:41:38 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2015-09-15 19:41:38 -0700 |
| commit | 64c29afcb787d9f176a197c25314295108ba0471 (patch) | |
| tree | 5dcf40c33d6317743ebe2d25ad91826d44a1ee44 /docs/mllib-feature-extraction.md | |
| parent | 35a19f3357d2ec017cfefb90f1018403e9617de4 (diff) | |
| download | spark-64c29afcb787d9f176a197c25314295108ba0471.tar.gz spark-64c29afcb787d9f176a197c25314295108ba0471.tar.bz2 spark-64c29afcb787d9f176a197c25314295108ba0471.zip | |
[SPARK-9078] [SQL] Allow jdbc dialects to override the query used to check the table.
Current implementation uses query with a LIMIT clause to find if table already exists. This syntax works only in some database systems. This patch changes the default query to the one that is likely to work on most databases, and adds a new method to the JdbcDialect abstract class to allow dialects to override the default query.
I looked at using the JDBC meta data calls, it turns out there is no common way to find the current schema, catalog..etc. There is a new method Connection.getSchema() , but that is available only starting jdk1.7 , and existing jdbc drivers may not have implemented it. Other option was to use jdbc escape syntax clause for LIMIT, not sure on how well this supported in all the databases also. After looking at all the jdbc metadata options my conclusion was most common way is to use the simple select query with 'where 1 =0' , and allow dialects to customize as needed
Author: sureshthalamati <suresh.thalamati@gmail.com>
Closes #8676 from sureshthalamati/table_exists_spark-9078.
Diffstat (limited to 'docs/mllib-feature-extraction.md')
0 files changed, 0 insertions, 0 deletions