Added support for Json and valid input to conformance tests.

This was enabled by the recent open-sourcing of JSON
support and MessageDifferencer.

MessageDifferencer allows the conformance suite to expand
because it allows us to write tests for payloads that parse
successfully.  To verify the testee's output payload, we
need to parse it back into a message and compare the message
instances.  Comparing output bytes vs. a golden message is
*not* valid, because protobufs do not have a canonical
encoding (especially in the presence of maps, which have
no prescribed serialization order).

We only add one small JSON test for now, but with the
framework in place we now have the foundation to dramatically
expand the coverage of the conformance test suite.

Also added the ability for the testee to skip tests that
exercise features that are unimplemented.  This allows
Java (which currently has no JSON support) to skip tests
involving JSON.

Change-Id: I697b4363da432b61ae3b638b4287c4cda1af4deb

1 files changed, 23 insertions, 2 deletions

diff --git a/conformance/conformance_test.h b/conformance/conformance_test.h
index 764a8d33..cadda828 100644
--- a/conformance/conformance_test.h
+++ b/conformance/conformance_test.h
@@ -39,6 +39,8 @@
 #define CONFORMANCE_CONFORMANCE_TEST_H
 
 #include <string>
+#include <google/protobuf/stubs/common.h>
+#include <google/protobuf/util/type_resolver.h>
 #include <google/protobuf/wire_format_lite.h>
 
 namespace conformance {
@@ -98,10 +100,22 @@ class ConformanceTestSuite {
 
  private:
   void ReportSuccess(const std::string& test_name);
-  void ReportFailure(const std::string& test_name, const char* fmt, ...);
+  void ReportFailure(const string& test_name,
+                     const conformance::ConformanceRequest& request,
+                     const conformance::ConformanceResponse& response,
+                     const char* fmt, ...);
+  void ReportSkip(const string& test_name,
+                  const conformance::ConformanceRequest& request,
+                  const conformance::ConformanceResponse& response);
   void RunTest(const std::string& test_name,
                const conformance::ConformanceRequest& request,
                conformance::ConformanceResponse* response);
+  void RunValidInputTest(const string& test_name, const string& input,
+                         conformance::WireFormat input_format,
+                         const string& equivalent_text_format,
+                         conformance::WireFormat requested_output);
+  void RunValidJsonTest(const string& test_name, const string& input_json,
+                        const string& equivalent_text_format);
   void ExpectParseFailureForProto(const std::string& proto,
                                   const std::string& test_name);
   void ExpectHardParseFailureForProto(const std::string& proto,
@@ -110,7 +124,7 @@ class ConformanceTestSuite {
   bool CheckSetEmpty(const set<string>& set_to_check, const char* msg);
   ConformanceTestRunner* runner_;
   int successes_;
-  int failures_;
+  int expected_failures_;
   bool verbose_;
   std::string output_;
 
@@ -127,6 +141,13 @@ class ConformanceTestSuite {
 
   // The set of tests that succeeded, but weren't expected to.
   std::set<std::string> unexpected_succeeding_tests_;
+
+  // The set of tests that the testee opted out of;
+  std::set<std::string> skipped_;
+
+  google::protobuf::internal::scoped_ptr<google::protobuf::util::TypeResolver>
+      type_resolver_;
+  std::string type_url_;
 };
 
 }  // namespace protobuf