diff options
| author | Nong Li <nong@databricks.com> | 2015-12-29 18:47:41 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-12-29 18:47:41 -0800 |
| commit | b600bccf41a7b1958e33d8301a19214e6517e388 (patch) | |
| tree | 93a01d5b6a39d2c2506c5581e6174ab9ffa8ba6d /sql/hive/pom.xml | |
| parent | 270a659584b6c1c304a9f9a331c56287672e00b0 (diff) | |
| download | spark-b600bccf41a7b1958e33d8301a19214e6517e388.tar.gz spark-b600bccf41a7b1958e33d8301a19214e6517e388.tar.bz2 spark-b600bccf41a7b1958e33d8301a19214e6517e388.zip | |
[SPARK-12362][SQL][WIP] Inline Hive Parser
This is a WIP. The PR has been taken over from nongli (see https://github.com/apache/spark/pull/10420). I have removed some additional dead code, and fixed a few issues which were caused by the fact that the inlined Hive parser is newer than the Hive parser we currently use in Spark.
I am submitting this PR in order to get some feedback and testing done. There is quite a bit of work to do:
- [ ] Get it to pass jenkins build/test.
- [ ] Aknowledge Hive-project for using their parser.
- [ ] Refactorings between HiveQl and the java classes.
- [ ] Create our own ASTNode and integrate the current implicit extentions.
- [ ] Move remaining ```SemanticAnalyzer``` and ```ParseUtils``` functionality to ```HiveQl```.
- [ ] Removing Hive dependencies from the parser. This will require some edits in the grammar files.
- [ ] Introduce our own context which needs to contain a ```TokenRewriteStream```.
- [ ] Add ```useSQL11ReservedKeywordsForIdentifier``` and ```allowQuotedId``` to the catalyst or sql configuration.
- [ ] Remove ```HiveConf``` from grammar files &HiveQl, and pass in our own configuration.
- [ ] Moving the parser into sql/core.
cc nongli rxin
Author: Herman van Hovell <hvanhovell@questtec.nl>
Author: Nong Li <nong@databricks.com>
Author: Nong Li <nongli@gmail.com>
Closes #10509 from hvanhovell/SPARK-12362.
Diffstat (limited to 'sql/hive/pom.xml')
| -rw-r--r-- | sql/hive/pom.xml | 22 |
1 files changed, 22 insertions, 0 deletions
diff --git a/sql/hive/pom.xml b/sql/hive/pom.xml index e9885f6682..ffabb92179 100644 --- a/sql/hive/pom.xml +++ b/sql/hive/pom.xml @@ -232,6 +232,7 @@ <configuration> <sources> <source>v${hive.version.short}/src/main/scala</source> + <source>${project.build.directory/generated-sources/antlr</source> </sources> </configuration> </execution> @@ -260,6 +261,27 @@ </execution> </executions> </plugin> + + + <plugin> + <groupId>org.antlr</groupId> + <artifactId>antlr3-maven-plugin</artifactId> + <executions> + <execution> + <goals> + <goal>antlr</goal> + </goals> + </execution> + </executions> + <configuration> + <sourceDirectory>${basedir}/src/main/antlr3</sourceDirectory> + <includes> + <include>**/SparkSqlLexer.g</include> + <include>**/SparkSqlParser.g</include> + </includes> + </configuration> + </plugin> + </plugins> </build> </project> |