diff options
-rw-r--r-- | README.md | 156 | ||||
-rw-r--r-- | src/main/scala/scala/async/AnfTransform.scala | 22 | ||||
-rw-r--r-- | src/main/scala/scala/async/Async.scala | 9 | ||||
-rw-r--r-- | src/main/scala/scala/async/AsyncUtils.scala | 3 | ||||
-rw-r--r-- | src/main/scala/scala/async/ExprBuilder.scala | 3 | ||||
-rw-r--r-- | src/main/scala/scala/async/TransformUtils.scala | 21 | ||||
-rw-r--r-- | src/test/scala/scala/async/run/anf/AnfTransformSpec.scala | 22 |
7 files changed, 207 insertions, 29 deletions
@@ -1,13 +1,156 @@ -Scala Async Project -=================== +# Scala Async Project -Building --------- +## Quickstart + + - Add `scala-async.jar` to your classpath + - Use Scala 2.10.0 + +```scala +import ExecutionContext.Implicits.global +import scala.async.Async.{async, await} + +val future = async { + val f1 = async { ...; true } + val f2 = async { ...; 42 } + if (await(f1)) await(f2) else 0 +} +``` + +## What is `async`? + +`async` marks a block of asynchronous code. Such a block usually contains +one or more `await` calls, which marks a point at which the computation +will be suspended until the awaited `Future` is complete. + +By default, `async` blocks operate on `scala.concurrent.{Future, Promise}`. +The system can be adapted to alternative implementations of the +`Future` pattern. + +Consider the following example: + +```scala +def slowCalcFuture: Future[Int] = ... // 01 +def combined: Future[Int] = async { // 02 + await(slowCalcFuture) + await(slowCalcFuture) // 03 +} +val x: Int = Await.result(combined, 10.seconds) // 05 +``` + +Lines 1 defines an asynchronous method: it returns a `Future`. + +Line 3 begins an `async` block. During compilation, +the contents of this block will be analyzed to identify +the `await` calls, and transformed into non-blocking +code. + +Control flow will immediately pass to line 5, as the +computation in the `async` block is not executed +on the caller's thread. + +Line 4 begins by triggering `slowCalcFuture`, and then +suspending until it has been calculating. Only after it +has finished, we trigger it again, and suspend again. +Finally, we add the results and complete `combined`, which +in turn will release line 5 (unless it had already timed out). + +It is important to note that while we this code is non-blocking, +it is not parallel. If we wanted to parallelize the two computations, +we could rearrange the code as follows. + +```scala +def combined: Future[Int] = async { + val future1 = slowCalcFuture + val future2 = slowCalcFuture + await(future1) + await(future2) +} +``` + +## Comparison with direct use of `Future` API + +This computation could also be expressed by directly using the +higher-order functions of Futures: + +```scala +def slowCalcFuture: Future[Int] = ... +val future1 = slowCalcFuture +val future2 = slowCalcFuture +def combined: Future[Int] = for { + r1 <- future1 + r2 <- future2 +} yield r1 + r2 +``` + +The `async` approach has two advantages over the use of +`map` and `flatMap`. + 1. The code more directly reflects the programmers intent, + and does not require us to name the results `r1` and `r2`. + This advantage is even more pronounces when we mix control + structures in `async` blocks. + 2. `async` blocks are compiled to a single anonymous class, + as opposed to a seperate anonymous class for each closure + required at each generator (`<-`) in the for-comprehension. + This reduces the size of generated code, and can avoid boxing + of intermediate results. + +## Comparison with CPS plugin + +The existing continations (CPS) plugin for Scala can also be used +to provide syntactic layer like `async`. This approach has been +used in Akka's [Dataflow Concurrency](http://doc.akka.io/docs/akka/snapshot/scala/dataflow.html) + +CPS based rewriting of asynchrous code also produces an closure +for each suspension. It can also lead to type errors that are +difficult to understand. + +## How it works + + - The async macro analyses the block of code, looking for control + structures and locations of await calls. It then breaks the code + into 'chunks'. Each chunk contains a linear sequence of statements + that concludes with branching decision, or with the registration + of subsequent state handler as a continuation. + - Before this analysis and transformation, the program is normalized + into a form amenable to this manipulation. This is called the + "A Normal Form" (ANF), and roughly means that: + - `if` and `match` constructs are only used as statements; + they cannot be used as an expression. + - calls to await are not allowed in compound expressions. + - Identify vals, vars and defs that are accessed from multiple + states. These will be lifted out to fields in the state machine + object. + - Synthesize a class that holds: + - an integer representing the current state ID + - the lifted definitions + - an `apply(value: Try[Any]): Unit` method that will be + called on completion of each future. This behaviour of + this method is determined by the current state. It records + the downcast result of the future in a field, and calls the + `resume()` method. + - the `resume(): Unit` method that switches on the current state + and runs the users code for one 'chunk', and either: + a) registers the state machine as the handler for the next future + b) completes the result Promise of the async block, if at the terminal state. + - an `apply(): Unit` method that starts the computation. + +## Troubleshooting + - Logging of the transform can be enabled with `scalac -Dscala.async.debug=true`. + - Tracing of the ANF transform: `scalac -Dscala.async.trace=true` + - Debug the macro expansion by checking out the project and executing the application + [`TreeInterrogation`](https://github.com/phaller/scala-async/blob/master/src/test/scala/scala/async/TreeInterrogation.scala#L59) + +## Limitations + - See the [neg](https://github.com/phaller/scala-async/tree/master/src/test/scala/scala/async/neg) test cases for + for constructs that are not allowed in a async block + - See the [issue list](https://github.com/phaller/scala-async/issues?state=open) for which of these restrictions are planned + to be dropped in the next milestone. + - See [#13](https://github.com/phaller/scala-async/issues/13) for why `await` is not possible in closures, and for suggestions on + ways to structure the code to work around this limitation. + +## Building The async macro and its test suite can be built and run with SBT. -Contributing ------------- +## Contributing If you are interested in contributing code, we ask you to complete and submit to us the Scala Contributor License Agreement, which allows us to ensure that @@ -17,3 +160,4 @@ http://www.scala-lang.org/sites/default/files/contributor_agreement.pdf Before submitting a pull-request, please make sure you have followed the guidelines outlined in our [Pull Request Policy](https://github.com/scala/scala/wiki/Pull-Request-Policy). + diff --git a/src/main/scala/scala/async/AnfTransform.scala b/src/main/scala/scala/async/AnfTransform.scala index 2fa96c9..afcf6bd 100644 --- a/src/main/scala/scala/async/AnfTransform.scala +++ b/src/main/scala/scala/async/AnfTransform.scala @@ -195,18 +195,18 @@ private[async] final case class AnfTransform[C <: Context](c: C) { val isByName: (Int) => Boolean = utils.isByName(fun) val funStats :+ simpleFun = inline.transformToList(fun) def isAwaitRef(name: Name) = name.toString.startsWith(utils.name.await + "$") - val argLists: List[List[Tree]] = args.zipWithIndex map { - case (arg, i) if isByName(i) || isSafeToInline(arg) => List(arg) - case (arg@Ident(name), _) if isAwaitRef(name) => List(arg) // not typed, so it eludes the check in `isSafeToInline` - case (arg, i) => inline.transformToList(arg) match { - case stats :+ expr => - val valDef = defineVal(name.arg(i), expr, arg.pos) - stats ::: List(valDef, Ident(valDef.name)) + val (argStats, argExprs): (List[List[Tree]], List[Tree]) = + mapArguments[List[Tree]](args) { + case (arg, i) if isByName(i) || isSafeToInline(arg) => (Nil, arg) + case (arg@Ident(name), _) if isAwaitRef(name) => (Nil, arg) // not typed, so it eludes the check in `isSafeToInline` + case (arg, i) => + inline.transformToList(arg) match { + case stats :+ expr => + val valDef = defineVal(name.arg(i), expr, arg.pos) + (stats :+ valDef, Ident(valDef.name)) + } } - } - val allArgStats = argLists flatMap (_.init) - val simpleArgs = argLists map (_.last) - funStats ++ allArgStats :+ attachCopy(tree)(Apply(simpleFun, simpleArgs).setSymbol(tree.symbol)) + funStats ++ argStats.flatten :+ attachCopy(tree)(Apply(simpleFun, argExprs).setSymbol(tree.symbol)) case Block(stats, expr) if containsAwait => inline.transformToList(stats :+ expr) diff --git a/src/main/scala/scala/async/Async.scala b/src/main/scala/scala/async/Async.scala index 2ff0f07..6ad1441 100644 --- a/src/main/scala/scala/async/Async.scala +++ b/src/main/scala/scala/async/Async.scala @@ -7,9 +7,6 @@ package scala.async import scala.language.experimental.macros import scala.reflect.macros.Context -/* - * @author Philipp Haller - */ object Async extends AsyncBase { import scala.concurrent.Future @@ -66,12 +63,12 @@ abstract class AsyncBase { def asyncImpl[T: c.WeakTypeTag](c: Context)(body: c.Expr[T]): c.Expr[futureSystem.Fut[T]] = { import c.universe._ - val anaylzer = AsyncAnalysis[c.type](c) + val analyzer = AsyncAnalysis[c.type](c) val utils = TransformUtils[c.type](c) import utils.{name, defn} import builder.futureSystemOps - anaylzer.reportUnsupportedAwaits(body.tree) + analyzer.reportUnsupportedAwaits(body.tree) // Transform to A-normal form: // - no await calls in qualifiers or arguments, @@ -88,7 +85,7 @@ abstract class AsyncBase { // states of our generated state machine, e.g. a value assigned before // an `await` and read afterwards. val renameMap: Map[Symbol, TermName] = { - anaylzer.defTreesUsedInSubsequentStates(anfTree).map { + analyzer.defTreesUsedInSubsequentStates(anfTree).map { vd => (vd.symbol, name.fresh(vd.name.toTermName)) }.toMap diff --git a/src/main/scala/scala/async/AsyncUtils.scala b/src/main/scala/scala/async/AsyncUtils.scala index 0a54d2e..1ade5f0 100644 --- a/src/main/scala/scala/async/AsyncUtils.scala +++ b/src/main/scala/scala/async/AsyncUtils.scala @@ -3,9 +3,6 @@ */ package scala.async -/* - * @author Philipp Haller - */ object AsyncUtils { private def enabled(level: String) = sys.props.getOrElse(s"scala.async.$level", "false").equalsIgnoreCase("true") diff --git a/src/main/scala/scala/async/ExprBuilder.scala b/src/main/scala/scala/async/ExprBuilder.scala index ef27672..180e7b9 100644 --- a/src/main/scala/scala/async/ExprBuilder.scala +++ b/src/main/scala/scala/async/ExprBuilder.scala @@ -8,9 +8,6 @@ import scala.collection.mutable.ListBuffer import collection.mutable import language.existentials -/* - * @author Philipp Haller - */ private[async] final case class ExprBuilder[C <: Context, FS <: FutureSystem](c: C, futureSystem: FS, origTree: C#Tree) { builder => diff --git a/src/main/scala/scala/async/TransformUtils.scala b/src/main/scala/scala/async/TransformUtils.scala index 22c1b90..5a4984f 100644 --- a/src/main/scala/scala/async/TransformUtils.scala +++ b/src/main/scala/scala/async/TransformUtils.scala @@ -293,4 +293,25 @@ private[async] final case class TransformUtils[C <: Context](c: C) { val castTree = tree.asInstanceOf[symtab.Tree] treeInfo.isExprSafeToInline(castTree) } + + /** Map a list of arguments to: + * - A list of argument Trees + * - A list of auxillary results. + * + * The function unwraps and rewraps the `arg :_*` construct. + * + * @param args The original argument trees + * @param f A function from argument (with '_*' unwrapped) and argument index to argument. + * @tparam A The type of the auxillary result + */ + def mapArguments[A](args: List[Tree])(f: (Tree, Int) => (A, Tree)): (List[A], List[Tree]) = { + args match { + case args :+ Typed(tree, Ident(tpnme.WILDCARD_STAR)) => + val (a, argExprs :+ lastArgExpr) = (args :+ tree).zipWithIndex.map(f.tupled).unzip + val exprs = argExprs :+ Typed(lastArgExpr, Ident(tpnme.WILDCARD_STAR)).setPos(lastArgExpr.pos) + (a, exprs) + case args => + args.zipWithIndex.map(f.tupled).unzip + } + } } diff --git a/src/test/scala/scala/async/run/anf/AnfTransformSpec.scala b/src/test/scala/scala/async/run/anf/AnfTransformSpec.scala index 529386b..41c13e0 100644 --- a/src/test/scala/scala/async/run/anf/AnfTransformSpec.scala +++ b/src/test/scala/scala/async/run/anf/AnfTransformSpec.scala @@ -289,4 +289,26 @@ class AnfTransformSpec { foo(a = await(next())) } mustBe ((1, 2)) } + + @Test + def repeatedParams1() { + import scala.async.AsyncId.{async, await} + var i = 0 + def foo(a: Int, b: Int*) = b.toList + def id(i: Int) = i + async { + foo(await(0), id(1), id(2), id(3), await(4)) + } mustBe (List(1, 2, 3, 4)) + } + + @Test + def repeatedParams2() { + import scala.async.AsyncId.{async, await} + var i = 0 + def foo(a: Int, b: Int*) = b.toList + def id(i: Int) = i + async { + foo(await(0), List(id(1), id(2), id(3)): _*) + } mustBe (List(1, 2, 3)) + } } |