Exhaustivity: TreeMakers as boolean propositions

We check exhaustivity by representing a match as a formula in
finite-domain propositional logic (FDPL) that is false when
the match may fail. The variables in the formula represent
tested trees in the match (type tests/value equality tests).

The approximation uses the same framework as the CSE analysis.
A matrix of tree makers is turned into a DAG, where sharing
represents the same value/type being tested.

We reduce FDPL to Boolean PL as follows.
For all assignments, V_i = c_i_j, we introduce a proposition
P_i_j that is true iff V_i is equal to the constant c_i_j,
for a given i, and all j, P_i_j are mutually exclusive
(a variable cannot have multiple values).
If the variable's domain is closed, we assert that one of P_i_j
must be true for each i and some j. The conjunction of these
propositions constitutes the equality axioms.

After going through negational normal form to conjunctive normal
form, we use a small SAT solver (the DPLL algorithm) to find
a model under which the equational axioms hold but the match fails.
The formula: EqAxioms /\ -MatchSucceeds.
Note that match failure expresses nicely in CNF: the
negation of each case (which yields a disjunction) is anded.

We then turn this model into variable assignments
(what's the variable (not) equal to, along with recursive
assignments for its fields). Valid assignments (no ill-typed
field assignments) are then presented to the user as counter examples.
A counter example is a value, a type test, a constructor call
or a wildcard. We prune the example set and only report the
most general examples. (Finally, we sort the output to yield stable,
i.e. testable, warning messages.)

A match is only checked for exhaustivity when the type of the
selector is "checkable" (has a sealed type or is a tuple
with at least one component of sealed type).

We consider statically known guard outcomes, but generally back
off (don't check exhaustivity) when a match has guards
or user-defined extractor calls. (Sometimes constant folding
lets us statically decide a guard.)
We ignore possibly failing null checks (which are performed
before calling extractors, for example), though this could
be done easily in the current framework. The problem is false
positives. People don't usually put nulls in tuples or lists.

To improve the exhaustivity checks, we rewrite `List()` to Nil.
TODO: more general rewrite of List(a, b, ..., z) to `a :: b :: ... :: z`.
When presenting counter examples, we represent lists in the
user-friendly List(a,...,z) format. (Similarly for tuples.)

There are no exhaustivity checks for a match-defined PartialFunction.

misc notes:
- fix pure case of dpll solver
  impure set (symbol that occurs both as a positive and negative literal)
  was always empty since I was looking for literals (which are not equal if positivity is not equal)
  but should have been looking for symbols

- FDPL -> BoolPL translation collects all syms in props
  since propForEqualsTo generates an Or, must traverse the prop
  rather than assuming only top-level Syms are relevant...

  also, propForEqualsTo will not assume Or'ing a whole domain is equivalent to True
  (which it isn't, since the type test may fail in general)

- improve counter example description
  - treat as constructor call when we either have definite type information about a real class,
    or we have no equality information at all, but the variable's type is a class
    and we gathered constraints about its fields (typically when selector is a tuple)
  - flatten a :: b :: ... :: Nil to List(a, b, ...)
  - don't print tuple constructor names, so instead of "Tuple2(a, b)", say "(a, b)"

- filter out more statically impossible subtypes
  the static types convey more information than is actually checkable at run time
  this is good, as it allows us to narrow down the subtypes of a sealed type,

  however, when modeling the corresponding run-time checks we need to "erase" the
  uncheckable parts (using existentials so that the types are still well-kinded),
  so that we can use static subtyping as a sound model for dynamic type checks

- experimental java enum handling seals enum class
    before, we created a refinement class as the placeholder for the sealed children
    it seems more direct to use the enum class for that
    this way, the new pattern matcher's exhaustiveness checker just works for java enums

26 files changed, 170 insertions, 114 deletions

diff --git a/test/files/neg/exhausting.check b/test/files/neg/exhausting.check
index 0bef21e077..7140b99428 100644
--- a/test/files/neg/exhausting.check
+++ b/test/files/neg/exhausting.check
@@ -1,29 +1,25 @@
-exhausting.scala:20: error: match is not exhaustive!
-missing combination              *           Nil
-
+exhausting.scala:21: error: match may not be exhaustive.
+It would fail on the following input: List(_, _, _)
   def fail1[T](xs: List[T]) = xs match {
                               ^
-exhausting.scala:24: error: match is not exhaustive!
-missing combination            Nil
-
+exhausting.scala:27: error: match may not be exhaustive.
+It would fail on the following input: Nil
   def fail2[T](xs: List[T]) = xs match {
                               ^
-exhausting.scala:27: error: match is not exhaustive!
-missing combination           Bar3
-
+exhausting.scala:32: error: match may not be exhaustive.
+It would fail on the following input: List(<not in (1, 2)>)
+  def fail3a(xs: List[Int]) = xs match {
+                              ^
+exhausting.scala:39: error: match may not be exhaustive.
+It would fail on the following input: Bar3
   def fail3[T](x: Foo[T]) = x match {
                             ^
-exhausting.scala:31: error: match is not exhaustive!
-missing combination           Bar2          Bar2
-
+exhausting.scala:47: error: match may not be exhaustive.
+It would fail on the following inputs: (Bar1, Bar2), (Bar1, Bar3), (Bar2, Bar1), (Bar2, Bar2)
   def fail4[T <: AnyRef](xx: (Foo[T], Foo[T])) = xx match {
                                                  ^
-exhausting.scala:36: error: match is not exhaustive!
-missing combination           Bar1          Bar2
-missing combination           Bar1          Bar3
-missing combination           Bar2          Bar1
-missing combination           Bar2          Bar2
-
+exhausting.scala:56: error: match may not be exhaustive.
+It would fail on the following inputs: (Bar1, Bar2), (Bar1, Bar3), (Bar2, Bar1), (Bar2, Bar2)
   def fail5[T](xx: (Foo[T], Foo[T])) = xx match {
                                        ^
-5 errors found
+6 errors found
diff --git a/test/files/neg/exhausting.flags b/test/files/neg/exhausting.flags
index b7eb21d5f5..85d8eb2ba2 100644
--- a/test/files/neg/exhausting.flags
+++ b/test/files/neg/exhausting.flags
@@ -1 +1 @@
--Xfatal-warnings -Xoldpatmat
+-Xfatal-warnings
diff --git a/test/files/neg/exhausting.scala b/test/files/neg/exhausting.scala
index 14b05695aa..5554ee2671 100644
--- a/test/files/neg/exhausting.scala
+++ b/test/files/neg/exhausting.scala
@@ -16,30 +16,46 @@ object Test {
   def ex3[T](xx: (Foo[T], Foo[T])) = xx match {
     case (_: Foo[_], _: Foo[_]) => ()
   }
-  
+
+  // fails for: ::(_, ::(_, ::(_, _)))
   def fail1[T](xs: List[T]) = xs match {
     case Nil            => "ok"
     case x :: y :: Nil  => "ok"
   }
+
+  // fails for: Nil
   def fail2[T](xs: List[T]) = xs match {
     case _ :: _ => "ok"
   }
+
+  // fails for: ::(<not in (2, 1)>, _)
+  def fail3a(xs: List[Int]) = xs match {
+    case 1 :: _ =>
+    case 2 :: _ =>
+    case Nil =>
+  }
+
+  // fails for: Bar3
   def fail3[T](x: Foo[T]) = x match {
     case Bar1   => "ok"
     case Bar2   => "ok"
   }
+  // fails for: (Bar1, Bar2)
+  // fails for: (Bar1, Bar3)
+  // fails for: (Bar2, Bar2)
+  // fails for: (Bar2, Bar1)
   def fail4[T <: AnyRef](xx: (Foo[T], Foo[T])) = xx match {
     case (Bar1, Bar1) => ()
     case (Bar2, Bar3) => ()
     case (Bar3, _) => ()
   }
+  // fails for: (Bar1, Bar2)
+  // fails for: (Bar1, Bar3)
+  // fails for: (Bar2, Bar1)
+  // fails for: (Bar2, Bar2)
   def fail5[T](xx: (Foo[T], Foo[T])) = xx match {
     case (Bar1, Bar1) => ()
     case (Bar2, Bar3) => ()
     case (Bar3, _) => ()
   }
-  
-  def main(args: Array[String]): Unit = {
-    
-  }
 }
diff --git a/test/files/neg/pat_unreachable.flags b/test/files/neg/pat_unreachable.flags
index ba80cad69b..cb8324a345 100644
--- a/test/files/neg/pat_unreachable.flags
+++ b/test/files/neg/pat_unreachable.flags
@@ -1 +1 @@
--Xoldpatmat
+-Xoldpatmat
\ No newline at end of file
diff --git a/test/files/neg/patmatexhaust.check b/test/files/neg/patmatexhaust.check
index 5426d61d31..6172811e13 100644
--- a/test/files/neg/patmatexhaust.check
+++ b/test/files/neg/patmatexhaust.check
@@ -1,54 +1,41 @@
-patmatexhaust.scala:7: error: match is not exhaustive!
-missing combination            Baz
-
+patmatexhaust.scala:7: error: match may not be exhaustive.
+It would fail on the following input: Baz
     def ma1(x:Foo) = x match {
                      ^
-patmatexhaust.scala:11: error: match is not exhaustive!
-missing combination            Bar
-
+patmatexhaust.scala:11: error: match may not be exhaustive.
+It would fail on the following input: Bar(_)
     def ma2(x:Foo) = x match {
                      ^
-patmatexhaust.scala:23: error: match is not exhaustive!
-missing combination           Kult          Kult
-missing combination           Qult          Qult
-
+patmatexhaust.scala:23: error: match may not be exhaustive.
+It would fail on the following inputs: (Kult(_), Kult(_)), (Qult(), Qult())
     def ma3(x:Mult) = (x,x) match { // not exhaustive
                       ^
-patmatexhaust.scala:49: error: match is not exhaustive!
-missing combination             Gp
-missing combination             Gu
-
+patmatexhaust.scala:49: error: match may not be exhaustive.
+It would fail on the following inputs: Gp(), Gu
     def ma4(x:Deep) = x match { // missing cases: Gu, Gp
                       ^
-patmatexhaust.scala:53: error: match is not exhaustive!
-missing combination             Gp
-
-    def ma5(x:Deep) = x match { // Gp
+patmatexhaust.scala:53: error: match may not be exhaustive.
+It would fail on the following input: Gp()
+    def ma5(x:Deep) = x match {
                       ^
-patmatexhaust.scala:59: error: match is not exhaustive!
-missing combination            Nil
-
+patmatexhaust.scala:59: error: match may not be exhaustive.
+It would fail on the following input: Nil
   def ma6()  = List(1,2) match { // give up
                    ^
-patmatexhaust.scala:75: error: match is not exhaustive!
-missing combination              B
-
+patmatexhaust.scala:75: error: match may not be exhaustive.
+It would fail on the following input: B()
   def ma9(x: B) = x match {
                   ^
-patmatexhaust.scala:100: error: match is not exhaustive!
-missing combination             C1
-
+patmatexhaust.scala:100: error: match may not be exhaustive.
+It would fail on the following input: C1()
     def ma10(x: C) = x match { // not exhaustive: C1 is not sealed.
                      ^
-patmatexhaust.scala:114: error: match is not exhaustive!
-missing combination             D1
-missing combination             D2
-
+patmatexhaust.scala:114: error: match may not be exhaustive.
+It would fail on the following inputs: D1, D2()
     def ma10(x: C) = x match {  // not exhaustive: C1 has subclasses.
                      ^
-patmatexhaust.scala:126: error: match is not exhaustive!
-missing combination             C1
-
+patmatexhaust.scala:126: error: match may not be exhaustive.
+It would fail on the following input: C1()
     def ma10(x: C) = x match { // not exhaustive: C1 is not abstract.
                      ^
 10 errors found
diff --git a/test/files/neg/patmatexhaust.flags b/test/files/neg/patmatexhaust.flags
index b7eb21d5f5..85d8eb2ba2 100644
--- a/test/files/neg/patmatexhaust.flags
+++ b/test/files/neg/patmatexhaust.flags
@@ -1 +1 @@
--Xfatal-warnings -Xoldpatmat
+-Xfatal-warnings
diff --git a/test/files/neg/patmatexhaust.scala b/test/files/neg/patmatexhaust.scala
index 9297e09d0d..ceb960ee97 100644
--- a/test/files/neg/patmatexhaust.scala
+++ b/test/files/neg/patmatexhaust.scala
@@ -50,7 +50,7 @@ class TestSealedExhaustive { // compile only
       case Ga => 
     }
 
-    def ma5(x:Deep) = x match { // Gp
+    def ma5(x:Deep) = x match {
       case Gu =>
       case _ if 1 == 0 => 
       case Ga =>
diff --git a/test/files/neg/sealed-java-enums.check b/test/files/neg/sealed-java-enums.check
index 9303c2df9c..20d00c8e91 100644
--- a/test/files/neg/sealed-java-enums.check
+++ b/test/files/neg/sealed-java-enums.check
@@ -1,9 +1,5 @@
-sealed-java-enums.scala:5: error: match is not exhaustive!
-missing combination        BLOCKED
-missing combination          State
-missing combination     TERMINATED
-missing combination  TIMED_WAITING
-
+sealed-java-enums.scala:5: error: match may not be exhaustive.
+It would fail on the following inputs: BLOCKED, TERMINATED, TIMED_WAITING
   def f(state: State) = state match {
                         ^
 one error found
diff --git a/test/files/neg/sealed-java-enums.flags b/test/files/neg/sealed-java-enums.flags
index 312f3a87ec..e709c65918 100644
--- a/test/files/neg/sealed-java-enums.flags
+++ b/test/files/neg/sealed-java-enums.flags
@@ -1 +1 @@
--Xexperimental -Xfatal-warnings -Xoldpatmat
+-Xexperimental -Xfatal-warnings
diff --git a/test/files/neg/switch.check b/test/files/neg/switch.check
index 7212c1a22b..8955c94b32 100644
--- a/test/files/neg/switch.check
+++ b/test/files/neg/switch.check
@@ -1,10 +1,10 @@
 switch.scala:28: error: could not emit switch for @switch annotated match
   def fail1(c: Char) = (c: @switch) match {
-                                    ^
+                            ^
 switch.scala:38: error: could not emit switch for @switch annotated match
   def fail2(c: Char) = (c: @switch @unchecked) match {
-                                               ^
+                                    ^
 switch.scala:45: error: could not emit switch for @switch annotated match
   def fail3(c: Char) = (c: @unchecked @switch) match {
-                                               ^
+                                       ^
 three errors found
diff --git a/test/files/neg/switch.flags b/test/files/neg/switch.flags
deleted file mode 100644
index 809e9ff2f2..0000000000
--- a/test/files/neg/switch.flags
+++ /dev/null
@@ -1 +0,0 @@
- -Xoldpatmat
diff --git a/test/files/neg/t3098.check b/test/files/neg/t3098.check
index 403da281c8..85829747b9 100644
--- a/test/files/neg/t3098.check
+++ b/test/files/neg/t3098.check
@@ -1,6 +1,5 @@
-b.scala:3: error: match is not exhaustive!
-missing combination              C
-
+b.scala:3: error: match may not be exhaustive.
+It would fail on the following input: (_ : C)
   def f = (null: T) match {
                ^
 one error found
diff --git a/test/files/neg/t3098.flags b/test/files/neg/t3098.flags
index b7eb21d5f5..85d8eb2ba2 100644
--- a/test/files/neg/t3098.flags
+++ b/test/files/neg/t3098.flags
@@ -1 +1 @@
--Xfatal-warnings -Xoldpatmat
+-Xfatal-warnings
diff --git a/test/files/neg/t3683a.check b/test/files/neg/t3683a.check
index 18e80dd5e8..3de3ad784e 100644
--- a/test/files/neg/t3683a.check
+++ b/test/files/neg/t3683a.check
@@ -1,6 +1,5 @@
-t3683a.scala:14: error: match is not exhaustive!
-missing combination             XX
-
+t3683a.scala:14: error: match may not be exhaustive.
+It would fail on the following input: XX()
     w match {
     ^
 one error found
diff --git a/test/files/neg/t3683a.flags b/test/files/neg/t3683a.flags
index b7eb21d5f5..85d8eb2ba2 100644
--- a/test/files/neg/t3683a.flags
+++ b/test/files/neg/t3683a.flags
@@ -1 +1 @@
--Xfatal-warnings -Xoldpatmat
+-Xfatal-warnings
diff --git a/test/files/neg/t3692-new.flags b/test/files/neg/t3692-new.flags
index 82becdfbfd..cb8324a345 100644
--- a/test/files/neg/t3692-new.flags
+++ b/test/files/neg/t3692-new.flags
@@ -1 +1 @@
- -Xoldpatmat
\ No newline at end of file
+-Xoldpatmat
\ No newline at end of file
diff --git a/test/files/neg/t3692-old.flags b/test/files/neg/t3692-old.flags
index 82becdfbfd..cb8324a345 100644
--- a/test/files/neg/t3692-old.flags
+++ b/test/files/neg/t3692-old.flags
@@ -1 +1 @@
- -Xoldpatmat
\ No newline at end of file
+-Xoldpatmat
\ No newline at end of file
diff --git a/test/files/pos/exhaust_alternatives.flags b/test/files/pos/exhaust_alternatives.flags
new file mode 100644
index 0000000000..e8fb65d50c
--- /dev/null
+++ b/test/files/pos/exhaust_alternatives.flags
@@ -0,0 +1 @@
+-Xfatal-warnings
\ No newline at end of file
diff --git a/test/files/pos/exhaust_alternatives.scala b/test/files/pos/exhaust_alternatives.scala
new file mode 100644
index 0000000000..cc81d0be7d
--- /dev/null
+++ b/test/files/pos/exhaust_alternatives.scala
@@ -0,0 +1,10 @@
+sealed abstract class X
+sealed case class A(x: Boolean) extends X
+case object B extends X
+
+object Test {
+  def test(x: X) = x match {
+    case A(true) =>
+    case A(false) | B =>
+  }
+}
\ No newline at end of file
diff --git a/test/files/pos/exhaustive_heuristics.scala b/test/files/pos/exhaustive_heuristics.scala
new file mode 100644
index 0000000000..f6bea455a5
--- /dev/null
+++ b/test/files/pos/exhaustive_heuristics.scala
@@ -0,0 +1,16 @@
+// tests exhaustivity doesn't give warnings (due to its heuristic rewrites kicking in or it backing off)
+object Test {
+  // List() => Nil
+  List(1) match {
+    case List() =>
+    case x :: xs =>
+  }
+
+  // we don't look into guards
+  val turnOffChecks = true
+  List(1) match {
+    case _ if turnOffChecks =>
+  }
+
+  // TODO: we back off when there are any user-defined extractors
+}
\ No newline at end of file
diff --git a/test/files/pos/t3097.scala b/test/files/pos/t3097.scala
deleted file mode 100644
index a034b960f7..0000000000
--- a/test/files/pos/t3097.scala
+++ /dev/null
@@ -1,31 +0,0 @@
-package seal
-
-sealed trait ISimpleValue
-
-sealed trait IListValue extends ISimpleValue {
-  def items: List[IAtomicValue[_]]
-}
-sealed trait IAtomicValue[O] extends ISimpleValue {
-  def data: O
-}
-
-sealed trait IAbstractDoubleValue[O] extends IAtomicValue[O] { }
-sealed trait IDoubleValue extends IAbstractDoubleValue[Double]
-
-case class ListValue(val items: List[IAtomicValue[_]]) extends IListValue
-class DoubleValue(val data: Double) extends IDoubleValue {
-  def asDouble = data
-}
-
-object Test {
-  /**
-   * @param args the command line arguments
-   */
-  def main(args: Array[String]): Unit = {
-    val v: ISimpleValue = new DoubleValue(1)
-    v match {
-      case m: IListValue => println("list")
-      case a: IAtomicValue[_] => println("atomic")
-    }
-  }
-}
diff --git a/test/files/pos/virtpatmat_exhaust.scala b/test/files/pos/virtpatmat_exhaust.scala
new file mode 100644
index 0000000000..a2f47c88c8
--- /dev/null
+++ b/test/files/pos/virtpatmat_exhaust.scala
@@ -0,0 +1,24 @@
+sealed trait Option {}
+case class Choice(a: Option, b: Option) extends Option;
+case class Some(x: Boolean) extends Option;
+case object None extends Option;
+
+object test {
+
+// drop any case and it will report an error
+// note that booleans are taken into account
+  def f(opt: Option) = opt match {
+    case Choice(None, None) => 1;
+    case Choice(None, Some(_)) => 1;
+    case Choice(None, Choice(_, _)) => 1;
+    case Choice(Some(true), None) => 1;
+    case Choice(Some(false), None) => 1;
+    case Choice(Some(_), Some(_)) => 1;
+    case Choice(Some(_), Choice(_, _)) => 1;
+    case Choice(Choice(_, _), None) => 1;
+    case Choice(Choice(_, _), Some(_)) => 1;
+    case Choice(Choice(_, _), Choice(_, _)) => 1;
+    case Some(b) => 4;
+    case None => 5;
+  }
+}
diff --git a/test/files/pos/virtpatmat_exhaust_unchecked.flags b/test/files/pos/virtpatmat_exhaust_unchecked.flags
new file mode 100644
index 0000000000..85d8eb2ba2
--- /dev/null
+++ b/test/files/pos/virtpatmat_exhaust_unchecked.flags
@@ -0,0 +1 @@
+-Xfatal-warnings
diff --git a/test/files/pos/virtpatmat_exhaust_unchecked.scala b/test/files/pos/virtpatmat_exhaust_unchecked.scala
new file mode 100644
index 0000000000..641f2b4f9a
--- /dev/null
+++ b/test/files/pos/virtpatmat_exhaust_unchecked.scala
@@ -0,0 +1,24 @@
+sealed trait Option {}
+case class Choice(a: Option, b: Option) extends Option;
+case class Some(x: Boolean) extends Option;
+case object None extends Option;
+
+object test {
+
+// drop any case and it will report an error
+// note that booleans are taken into account
+  def f(opt: Option) = (opt: @unchecked) match {
+    case Choice(None, None) => 1;
+    case Choice(None, Some(_)) => 1;
+    case Choice(None, Choice(_, _)) => 1;
+    case Choice(Some(true), None) => 1;
+    // case Choice(Some(false), None) => 1;
+    case Choice(Some(_), Some(_)) => 1;
+    case Choice(Some(_), Choice(_, _)) => 1;
+    case Choice(Choice(_, _), None) => 1;
+    case Choice(Choice(_, _), Some(_)) => 1;
+    case Choice(Choice(_, _), Choice(_, _)) => 1;
+    case Some(b) => 4;
+    case None => 5;
+  }
+}
diff --git a/test/files/run/t3097.check b/test/files/run/t3097.check
new file mode 100644
index 0000000000..63695f771b
--- /dev/null
+++ b/test/files/run/t3097.check
@@ -0,0 +1 @@
+atomic
diff --git a/test/files/run/t3097.scala b/test/files/run/t3097.scala
new file mode 100644
index 0000000000..4aaf8056ca
--- /dev/null
+++ b/test/files/run/t3097.scala
@@ -0,0 +1,18 @@
+sealed trait ISimpleValue
+
+sealed trait IListValue extends ISimpleValue
+sealed trait IAtomicValue[O] extends ISimpleValue
+
+sealed trait IAbstractDoubleValue[O] extends IAtomicValue[O]
+sealed trait IDoubleValue extends IAbstractDoubleValue[Double]
+
+case class ListValue(val items: List[IAtomicValue[_]]) extends IListValue
+class DoubleValue(val data: Double) extends IDoubleValue
+
+object Test extends App {
+  // match is exhaustive
+  (new DoubleValue(1): ISimpleValue) match {
+    case m: IListValue => println("list")
+    case a: IAtomicValue[_] => println("atomic")
+  }
+}