diff options
| author | Cheng Lian <lian.cs.zju@gmail.com> | 2014-10-13 13:50:27 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-10-13 13:50:27 -0700 |
| commit | 9eb49d4134e23a15142fb592d54d920e89bd8786 (patch) | |
| tree | cdbeae8115304340b3edb55cdd6eb8078b4054ca /external/flume/src/test | |
| parent | 9d9ca91fef70eca6fc576be9c99aed5d8ce6e68b (diff) | |
| download | spark-9eb49d4134e23a15142fb592d54d920e89bd8786.tar.gz spark-9eb49d4134e23a15142fb592d54d920e89bd8786.tar.bz2 spark-9eb49d4134e23a15142fb592d54d920e89bd8786.zip | |
[SPARK-3809][SQL] Fixes test suites in hive-thriftserver
As scwf pointed out, `HiveThriftServer2Suite` isn't effective anymore after the Thrift server was made a daemon. On the other hand, these test suites were known flaky, PR #2214 tried to fix them but failed because of unknown Jenkins build error. This PR fixes both sets of issues.
In this PR, instead of watching `start-thriftserver.sh` output, the test code start a `tail` process to watch the log file. A `Thread.sleep` has to be introduced because the `kill` command used in `stop-thriftserver.sh` is not synchronous.
As for the root cause of the mysterious Jenkins build failure. Please refer to [this comment](https://github.com/apache/spark/pull/2675#issuecomment-58464189) below for details.
----
(Copied from PR description of #2214)
This PR fixes two issues of `HiveThriftServer2Suite` and brings 1 enhancement:
1. Although metastore, warehouse directories and listening port are randomly chosen, all test cases share the same configuration. Due to parallel test execution, one of the two test case is doomed to fail
2. We caught any exceptions thrown from a test case and print diagnosis information, but forgot to re-throw the exception...
3. When the forked server process ends prematurely (e.g., fails to start), the `serverRunning` promise is completed with a failure, preventing the test code to keep waiting until timeout.
So, embarrassingly, this test suite was failing continuously for several days but no one had ever noticed it... Fortunately no bugs in the production code were covered under the hood.
Author: Cheng Lian <lian.cs.zju@gmail.com>
Author: wangfei <wangfei1@huawei.com>
Closes #2675 from liancheng/fix-thriftserver-tests and squashes the following commits:
1c384b7 [Cheng Lian] Minor code cleanup, restore the logging level hack in TestHive.scala
7805c33 [wangfei] reset SPARK_TESTING to avoid loading Log4J configurations in testing class paths
af2b5a9 [Cheng Lian] Removes log level hacks from TestHiveContext
d116405 [wangfei] make sure that log4j level is INFO
ee92a82 [Cheng Lian] Relaxes timeout
7fd6757 [Cheng Lian] Fixes test suites in hive-thriftserver
Diffstat (limited to 'external/flume/src/test')
0 files changed, 0 insertions, 0 deletions