aboutsummaryrefslogtreecommitdiff
path: root/docs/docs/contributing/testing.md
blob: 07aab191847e6db3ab7031a405b7c13db9184a26 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
---
layout: doc-page
title: Testing in Dotty
---

<aside class="warning">
This page should be updated as soon as scala-partest is removed
</aside>

Running all tests in Dotty is as simple as:

```bash
$ sbt test
```

There are currently several forms of tests in Dotty. These can be split into
two categories:

## Unit tests
These tests can be found in `<sub-project>/test` and are used to check
functionality of specific parts of the codebase in isolation e.g: parsing,
scanning and message errors.

Running a single unit test class from sbt is as simple as:

```bash
> testOnly absolute.path.to.TestClass
```

You can further restrict the executed tests to a subset of `TestClass` methods
as follows:

```bash
> testOnly absolute.path.to.TestClass -- *methodName
```

## Integration tests
These tests are Scala source files expected to compile with Dotty (pos tests),
along with their expected output (run tests) or errors (neg tests).

All of these tests are contained in the `./tests/*` directories.

## scala-partest
Historically these tests needed a structure which was generated by running the
unit tests, and then that structure was in turn used by
[scala-partest](http://github.com/scala/scala-partest) to run compilation tests
in parallel.

This test suite can still be used (and is currently a part of the CI to check
that it has the same outcome as the new test suite). It is invoked from sbt by
running one of the following commands:

```bash
> partest-only-no-bootstrap
> partest-only
> partest
```

- `partest-only-no-bootstrap` will only run the integration tests
- `partest-only` will bootstrap the compiler and run the integration tests
- `partest` will bootstrap the compiler, run the unit tests and then the
  integration tests

## dotty parallel test suite
The new test suite will soon become the standard integration test runner. It
has several advantages over the old implementation:

- integrates with JUnit, without the need for setup
- reuses the same VM for compilation
- allows filtering of tests
- runs much faster (almost 2x)

Currently to run these tests you need to invoke from sbt:

```bash
> testOnly dotty.tools.dotc.CompilationTests
```

This might be aliased in the future. It is also possible to run tests filtered
by using:

```bash
> filterTest .*i2147.scala
```

This will run both the test `./tests/pos/i2147.scala` and
`./tests/partest-test/i2147.scala` since both of these match the given regular
expression. This also means that you could run `filterTest .*` to run all
integration tests.