diff options
| author | Herman van Hovell <hvanhovell@questtec.nl> | 2016-03-28 12:31:12 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-03-28 12:31:12 -0700 |
| commit | 600c0b69cab4767e8e5a6f4284777d8b9d4bd40e (patch) | |
| tree | bae635ab17a8b58400127f20bbbe5acaecc92f98 /dev/deps | |
| parent | 1528ff4c9affe1df103c4b3abd56a86c71d8b753 (diff) | |
| download | spark-600c0b69cab4767e8e5a6f4284777d8b9d4bd40e.tar.gz spark-600c0b69cab4767e8e5a6f4284777d8b9d4bd40e.tar.bz2 spark-600c0b69cab4767e8e5a6f4284777d8b9d4bd40e.zip | |
[SPARK-13713][SQL] Migrate parser from ANTLR3 to ANTLR4
### What changes were proposed in this pull request?
The current ANTLR3 parser is quite complex to maintain and suffers from code blow-ups. This PR introduces a new parser that is based on ANTLR4.
This parser is based on the [Presto's SQL parser](https://github.com/facebook/presto/blob/master/presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4). The current implementation can parse and create Catalyst and SQL plans. Large parts of the HiveQl DDL and some of the DML functionality is currently missing, the plan is to add this in follow-up PRs.
This PR is a work in progress, and work needs to be done in the following area's:
- [x] Error handling should be improved.
- [x] Documentation should be improved.
- [x] Multi-Insert needs to be tested.
- [ ] Naming and package locations.
### How was this patch tested?
Catalyst and SQL unit tests.
Author: Herman van Hovell <hvanhovell@questtec.nl>
Closes #11557 from hvanhovell/ngParser.
Diffstat (limited to 'dev/deps')
| -rw-r--r-- | dev/deps/spark-deps-hadoop-2.2 | 1 | ||||
| -rw-r--r-- | dev/deps/spark-deps-hadoop-2.3 | 1 | ||||
| -rw-r--r-- | dev/deps/spark-deps-hadoop-2.4 | 1 | ||||
| -rw-r--r-- | dev/deps/spark-deps-hadoop-2.6 | 1 | ||||
| -rw-r--r-- | dev/deps/spark-deps-hadoop-2.7 | 1 |
5 files changed, 5 insertions, 0 deletions
diff --git a/dev/deps/spark-deps-hadoop-2.2 b/dev/deps/spark-deps-hadoop-2.2 index 512675a599..7c2f88bdb1 100644 --- a/dev/deps/spark-deps-hadoop-2.2 +++ b/dev/deps/spark-deps-hadoop-2.2 @@ -3,6 +3,7 @@ RoaringBitmap-0.5.11.jar ST4-4.0.4.jar activation-1.1.jar antlr-runtime-3.5.2.jar +antlr4-runtime-4.5.2-1.jar aopalliance-1.0.jar apache-log4j-extras-1.2.17.jar arpack_combined_all-0.1.jar diff --git a/dev/deps/spark-deps-hadoop-2.3 b/dev/deps/spark-deps-hadoop-2.3 index 31f8694fed..f4d600038d 100644 --- a/dev/deps/spark-deps-hadoop-2.3 +++ b/dev/deps/spark-deps-hadoop-2.3 @@ -3,6 +3,7 @@ RoaringBitmap-0.5.11.jar ST4-4.0.4.jar activation-1.1.1.jar antlr-runtime-3.5.2.jar +antlr4-runtime-4.5.2-1.jar aopalliance-1.0.jar apache-log4j-extras-1.2.17.jar arpack_combined_all-0.1.jar diff --git a/dev/deps/spark-deps-hadoop-2.4 b/dev/deps/spark-deps-hadoop-2.4 index 0fa8bccab0..7c5e2c35bd 100644 --- a/dev/deps/spark-deps-hadoop-2.4 +++ b/dev/deps/spark-deps-hadoop-2.4 @@ -3,6 +3,7 @@ RoaringBitmap-0.5.11.jar ST4-4.0.4.jar activation-1.1.1.jar antlr-runtime-3.5.2.jar +antlr4-runtime-4.5.2-1.jar aopalliance-1.0.jar apache-log4j-extras-1.2.17.jar arpack_combined_all-0.1.jar diff --git a/dev/deps/spark-deps-hadoop-2.6 b/dev/deps/spark-deps-hadoop-2.6 index 8d2f6e6e32..03d9a51057 100644 --- a/dev/deps/spark-deps-hadoop-2.6 +++ b/dev/deps/spark-deps-hadoop-2.6 @@ -3,6 +3,7 @@ RoaringBitmap-0.5.11.jar ST4-4.0.4.jar activation-1.1.1.jar antlr-runtime-3.5.2.jar +antlr4-runtime-4.5.2-1.jar aopalliance-1.0.jar apache-log4j-extras-1.2.17.jar apacheds-i18n-2.0.0-M15.jar diff --git a/dev/deps/spark-deps-hadoop-2.7 b/dev/deps/spark-deps-hadoop-2.7 index a114c4ae8d..5765071a1c 100644 --- a/dev/deps/spark-deps-hadoop-2.7 +++ b/dev/deps/spark-deps-hadoop-2.7 @@ -3,6 +3,7 @@ RoaringBitmap-0.5.11.jar ST4-4.0.4.jar activation-1.1.1.jar antlr-runtime-3.5.2.jar +antlr4-runtime-4.5.2-1.jar aopalliance-1.0.jar apache-log4j-extras-1.2.17.jar apacheds-i18n-2.0.0-M15.jar |