SI-8955 Fix hanging fork-join pool via parallel collections

A recent change [1] to Scala's default fork join thread pool caused
intermittent deadlocks. This is only visible in the development series
of Scala, 2.12.0-SNAPSHOT.

We changed our thread factory to place a hard limit the number of
threads created (equal to the desired parallelism.)

I have extracted a test case [2] that uses jsr166e directly, rather
than using Scala's parallel collections and abstractions on top of FJ.

In the comments of the bug, Viktor suggests this was too aggressive
and instead we ought to increase the limit to parallelism + 256
(with a system property override.)

He explained:

> The number 256 is going to be the default for the max threads for
> FJP in Java9 (down from 32k) so this change will harmonize the
> settings while making it possible to override from the outside.
>
> The cause of the deadlock is twofold:
>
> 1) The test uses ExecutionContext.global, which is not designed
>    for typical ForkJoin workloads since it has async = true
>    (FIFO instead of LIFO)
> 2) And we capped the default max number of threads to be created
>    when doing managed blocking from 32k to number of cores
>    (a tad too aggressive it seems)

Through testing, I found that for this example I could trigger the
hang with:

    parallelismLevel | maxThreads
    -----------------------------
                   2 | <= 4
                   4 | <= 9
                   8 | <= 11
                  16 | <= 15

I have emailed concurrency-interest [3] to help analyse the problem
further, but in the interest of avoiding hangs in the
scalacheck/parallel-collections test, I'm implementing Viktor's
suggestion in the interim.

[1] https://github.com/scala/scala/pull/4042
[2] https://gist.github.com/retronym/2e14cdab6d5612562d95
[3] http://markmail.org/message/czphdyjxpkixeztv

0 files changed, 0 insertions, 0 deletions