diff options
| author | Tor Myklebust <tmyklebu@gmail.com> | 2015-02-02 19:50:14 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-02-02 19:50:14 -0800 |
| commit | 8f471a66db0571a76a21c0d93312197fee16174a (patch) | |
| tree | fcb1817c8b074956a7af39aa36b0f1629bef5483 /sql/core/pom.xml | |
| parent | 1bcd46574e442e20f55709d70573f271ce44e5b9 (diff) | |
| download | spark-8f471a66db0571a76a21c0d93312197fee16174a.tar.gz spark-8f471a66db0571a76a21c0d93312197fee16174a.tar.bz2 spark-8f471a66db0571a76a21c0d93312197fee16174a.zip | |
[SPARK-5472][SQL] A JDBC data source for Spark SQL.
This pull request contains a Spark SQL data source that can pull data from, and can put data into, a JDBC database.
I have tested both read and write support with H2, MySQL, and Postgres. It would surprise me if both read and write support worked flawlessly out-of-the-box for any other database; different databases have different names for different JDBC data types and different meanings for SQL types with the same name. However, this code is designed (see `DriverQuirks.scala`) to make it *relatively* painless to add support for another database by augmenting the type mapping contained in this PR.
Author: Tor Myklebust <tmyklebu@gmail.com>
Closes #4261 from tmyklebu/master and squashes the following commits:
cf167ce [Tor Myklebust] Work around other Java tests ruining TestSQLContext.
67893bf [Tor Myklebust] Move the jdbcRDD methods into SQLContext itself.
585f95b [Tor Myklebust] Dependencies go into the project's pom.xml.
829d5ba [Tor Myklebust] Merge branch 'master' of https://github.com/apache/spark
41647ef [Tor Myklebust] Hide a couple things that don't need to be public.
7318aea [Tor Myklebust] Fix scalastyle warnings.
a09eeac [Tor Myklebust] JDBC data source for Spark SQL.
176bb98 [Tor Myklebust] Add test deps for JDBC support.
Diffstat (limited to 'sql/core/pom.xml')
| -rw-r--r-- | sql/core/pom.xml | 24 |
1 files changed, 24 insertions, 0 deletions
diff --git a/sql/core/pom.xml b/sql/core/pom.xml index 3e9ef07df9..1a0c77d282 100644 --- a/sql/core/pom.xml +++ b/sql/core/pom.xml @@ -76,6 +76,30 @@ <artifactId>scalacheck_${scala.binary.version}</artifactId> <scope>test</scope> </dependency> + <dependency> + <groupId>com.h2database</groupId> + <artifactId>h2</artifactId> + <version>1.4.183</version> + <scope>test</scope> + </dependency> + <dependency> + <groupId>mysql</groupId> + <artifactId>mysql-connector-java</artifactId> + <version>5.1.34</version> + <scope>test</scope> + </dependency> + <dependency> + <groupId>org.postgresql</groupId> + <artifactId>postgresql</artifactId> + <version>9.3-1102-jdbc41</version> + <scope>test</scope> + </dependency> + <dependency> + <groupId>com.spotify</groupId> + <artifactId>docker-client</artifactId> + <version>2.7.5</version> + <scope>test</scope> + </dependency> </dependencies> <build> <outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory> |