diff options
| author | Davies Liu <davies@databricks.com> | 2015-06-22 18:03:59 -0700 |
|---|---|---|
| committer | Davies Liu <davies@databricks.com> | 2015-06-22 18:03:59 -0700 |
| commit | 6b7f2ceafdcbb014791909747c2210b527305df9 (patch) | |
| tree | 7a99294d26f19626b878fe6c8c6d889884f80df8 /project/MimaExcludes.scala | |
| parent | 860a49ef20cea5711a7f54de0053ea33647e56a7 (diff) | |
| download | spark-6b7f2ceafdcbb014791909747c2210b527305df9.tar.gz spark-6b7f2ceafdcbb014791909747c2210b527305df9.tar.bz2 spark-6b7f2ceafdcbb014791909747c2210b527305df9.zip | |
[SPARK-8307] [SQL] improve timestamp from parquet
This PR change to convert julian day to unix timestamp directly (without Calendar and Timestamp).
cc adrian-wang rxin
Author: Davies Liu <davies@databricks.com>
Closes #6759 from davies/improve_ts and squashes the following commits:
849e301 [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts
b0e4cad [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts
8e2d56f [Davies Liu] address comments
634b9f5 [Davies Liu] fix mima
4891efb [Davies Liu] address comment
bfc437c [Davies Liu] fix build
ae5979c [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts
602b969 [Davies Liu] remove jodd
2f2e48c [Davies Liu] fix test
8ace611 [Davies Liu] fix mima
212143b [Davies Liu] fix mina
c834108 [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts
a3171b8 [Davies Liu] Merge branch 'master' of github.com:apache/spark into improve_ts
5233974 [Davies Liu] fix scala style
361fd62 [Davies Liu] address comments
ea196d4 [Davies Liu] improve timestamp from parquet
Diffstat (limited to 'project/MimaExcludes.scala')
| -rw-r--r-- | project/MimaExcludes.scala | 12 |
1 files changed, 11 insertions, 1 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 015d0296dd..7a748fb5e3 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -54,7 +54,17 @@ object MimaExcludes { ProblemFilters.exclude[MissingMethodProblem]( "org.apache.spark.streaming.kafka.KafkaTestUtils.waitUntilLeaderOffset"), // SQL execution is considered private. - excludePackage("org.apache.spark.sql.execution") + excludePackage("org.apache.spark.sql.execution"), + // NanoTime and CatalystTimestampConverter is only used inside catalyst, + // not needed anymore + ProblemFilters.exclude[MissingClassProblem]( + "org.apache.spark.sql.parquet.timestamp.NanoTime"), + ProblemFilters.exclude[MissingClassProblem]( + "org.apache.spark.sql.parquet.timestamp.NanoTime$"), + ProblemFilters.exclude[MissingClassProblem]( + "org.apache.spark.sql.parquet.CatalystTimestampConverter"), + ProblemFilters.exclude[MissingClassProblem]( + "org.apache.spark.sql.parquet.CatalystTimestampConverter$") ) case v if v.startsWith("1.4") => Seq( |