Re-wrote docs and some error messages

1 files changed, 357 insertions, 140 deletions

diff --git a/README.md b/README.md
index 34eb57c..e16602d 100644
--- a/README.md
+++ b/README.md
@@ -1,184 +1,361 @@
-Join the chat at https://gitter.im/cvogt/cbt
+(For a tutorial scroll down.)
 
-Welcome to Chris' Build Tool (CBT) for Scala
+Chris' Build Tool (CBT) for Scala
 ============================================
 
-Fun, fast, intuitive, composable and statically checked builds written in Scala.
+Easy to learn and master, lightning fast and backed by a thriving community of enthusiasts and contributors. [Join us on gitter](https://gitter.im/cvogt/cbt). For talks, development roadmap, projects using cbt, etc see the wiki.
+
+What is CBT?
+----------------------------------------------
+CBT is a build tool meaning it helps orchestrating compilation, code and
+documentation generation, packaging, deployment and custom tooling for
+your project. It mainly targets Scala projects but is not exclusive to them.
+
+CBT builds are full programs written using vanilla Scala code.
+Familiar concepts make you feel right at home - build files are classes,
+tasks are defs, you customize using method overrides. You already know these
+things and everything behaves as expected. That way implementing any
+build related requirement becomes as easy as writing any other Scala code.
+
+CBT is simple in the sense that it uses very few concepts.
+A single build uses classes, defs and inheritance.
+Builds and binary dependencies can be composed to model 
+modules depending on each other.
+
+CBT believes good integration with existing tools to be very
+helpful. In that spirit CBT aims for excellent integration with
+the command line and your shell.
+
+CBT considers source files to be an excellent way to distribute code
+and has first class support for source and git dependencies.
+
+How is CBT different from other build tools?
+----------------------------------------------
+Not all build tools allow you to write builds in a full programming language.
+CBT is based on the assumption that builds are complex enough problems to
+warrant this and abstraction and re-use is better handled through libraries
+rather than some restricted, declarative DSL. CBT shares this philosophy with SBT.
+(This also means that integration with external tools such as an IDE better happens
+programmatically through an api rather than a static data representation such as xml.)
+
+Like SBT, CBT chooses Scala as it's language of choice, trying to appeal to
+Scala programmers allowing them to re-use their knowledge and the safety of the language.
+
+Unlike SBT 0.11 and later, CBT maps task execution to JVM method invocations.
+SBT implements it's own self-contained task graph model and interpreter.
+This allows SBT to have it's model exactly fit the requirements.
+CBT instead uses existing JVM concepts for the solution and adds
+custom concepts only when necessary. CBT assumes this to lead to better
+ease of use due to familarity and better integration with existing tools
+such as interactive debuggers or stack traces because CBT's task call stack
+IS the JVM call stack.
+
+SBT 0.7 shared this design decision as many may have forgotten.
+However, CBT is still quite a bit simpler than even SBT 0.7 as
+CBT gets away with fewer concepts. SBT 0.7 had notions of
+main vs test sources, multi-project builds, task-dependencies
+which weren't invocations and other concepts. CBT maps all of
+these to def invocations and build composition instead.
+
+System requirements
+-------------------
+
+CBT is best tested under OSX. People are also using it also under
+Ubuntu and Windows via cygwin. It should be easy to port CBT to
+other systems or drop the cygwin requirement. You will only
+have to touch the launcher bash or .bat scripts.
+Please contribute back if you fixed something :).
+
+You currently need javac and realpath or gcc installed.
+nailgun is optional for speedup. gpg is required only for publishing maven artifacts.
+
+Features
+--------
+
+CBT supports the basic needs for Scala builds right now:
+Compiling, running, testing, packaging, publishing local and to sonatype,
+scaladoc, maven dependencies, source dependencies (e.g. for modularized projects),
+triggering tasks on file changes, cross-compilation, reproducible builds.
+
+There is also a growing number of plugins in `plugins/` and `stage2/plugins/`,
+but some things you'd like may still be missing. Consider writing
+a plugin in that case. It's super easy, just a trait. Share it :).
+
+
+Tutorial
+--------
+
+This section explains how to get started with cbt step-by-step.
+There are also example projects with build files in examples/ and test/.
+
+### Installation
+
+If you haven't cloned cbt yet, clone it now. Cloning is how you install cbt.
+We know that's a bit unusual, but roll with it, there are good reasons :).
+Open a shell, cd to the directory where you want to install cbt and execute:
 
-Currently CBT has been tested in OSX only. Adding support for Unix and Windows
-should not be hard. Please contribute back if you mange :).
+```
+$ git clone git@github.com:cvogt/cbt.git
+```
+
+There are a bash script `cbt` and a `cbt.bat` in the checkout directory.
+Add one to your `$PATH`, e.g. symlink it from `~/bin/cbt`.
+
+Check that it works by calling `cbt`. You should see CBT compiling itself
+and showing a list of built-in tasks.
+
+Great, you're all set up. Now, let's use cbt for an example project.
+
+### Creating your first project
+
+Create a new directory and cd into it. E.g. `my-project`.
+
+```
+$ mkdir my-project
+$ cd my-project
+```
+
+Let's create a tiny sample app. CBT can generate it for you. Just run:
+
+```
+$ cbt tools createMain
+```
+
+### Running your code
+
+Now there should be a file `Main.scala`, which prints `Hello World` when run.
+So let's run it:
+
+```
+$ cbt run
+```
+
+You should see how CBT first compiles your project, then runs it and prints
+`Hello World`. CBT created the file `Main.scala` top-level in your directory.
+You can alternatively place `.scala` or `.java` files in `src/`
+or any of it's subdirectories.
+
+### Creating a build file
+
+Without a build file, CBT just uses some default build settings.
+Let's make the build more concrete by creating a build file.
 
-CBT supports the basic needs for Scala builds right now.
-Composing, compiling, running, testing, packaging, publishing.
-Tasks outside of these, such as building manuals will require
-easy custom code. If you integrate something, consider
-doing it as traits that you make available as a library that
-other builds can depend on and mix in.
+CBT can help you with that. Execute:
 
-Slides and video from CBT talk from NEScala 2016:
-https://github.com/cvogt/talks/raw/master/2016-03-04_NEScala-2016_A-Vision-For-Scala-Builds.pdf
-https://www.youtube.com/watch?v=5HfKw3hgdOM
+```
+$ cbt tools createBuild
+```
 
-Getting started
----------------
+Now there should be a file `build/build.scala` with a sample `Build class.
 
-### Dependencies
-You currently need javac, nailgun, gpg and realpath or gcc installed.
+Btw., a build file can have it's own build and so on recursively like in SBT.
+When you create a file `build/build/build.scala` and change `Build` class in there
+to extend `BuildBuild`, it will be used to build your `build/build.scala`. You can
+add built-time dependencies like plugins this way.
 
-### Installing
-CBT bootstraps from source. To install, just clone the repository.
+### Adding dependencies
 
-### Using
-To use, just call the `cbt` bash script. You will see CBT first
-building up itself, then trying to start your build.
+In the generated `build/build.scala` there are
+several examples for dependencies. We recommend using the constructor syntax
+`ScalaDependency` (for automatically adding the scala version to the artifact id)
+or `MavenDependency` (for leaving the artifact id as is). The SBT-Style `%`-DSL
+syntax is also supported for copy-and-paste convenience, but discouraged.
 
-The easiest CBT build requires no script. It compiles source files from the
-current directory and from `src/` into the target folder. Try calling `cbt compile`.
-If you have a class called Main with a main method, try `cbt run`.
-Put a source file with a Main class into `test/` in order for
-`cbt test` to run it. It will see your main code.
+Alright, let's enable the `override def dependencies`. Make sure to include
+`super.dependencies`, which currently only includes the Scala standard library.
+Add a dependency of your choice, start using it from `Main.scala` and `cbt run` again.
 
-If you need more than this, like dependencies, create a scala file in `build/`
-that describes your build. Here is an example
+As you can see CBT makes choice of the maven repository explicit. It does so for clarity.
 
-```scala
-// build/build.scala
-class Build(val context: cbt.Context) extends PackageJars {
-  override def defaultVersion = "0.6.1"
+### Calling other tasks
 
-  override def name = "play-json-extensions"
+Tasks are just defs. You can call any public zero-arguments method of your
+`Build class or its parents straight from the command line. To see how it works
+let's call the compile task.
 
-  override def groupId = "org.cvogt"
+```
+$ cbt compile
+```
 
-  override def dependencies =
-    super.dependencies ++
-      Resolver(mavenCentral).bind(
-        // encouraged way to declare dependencies
-        ScalaDependency("com.typesafe.play", "play-json", "2.4.4"),
-        MavenDependency("joda-time", "joda-time", "2.9.2")
-      )
+### Creating custom tasks
 
-  override def compile = {
-    println("Compiling...")
-    super.compile
-  }
+In order to create a custom task, simply add a new def to your Build class, e.g.
 
-  def foo = "Hello World"
+```
+class Build...{
+  ...
+  def foo = "asdf"
 }
 ```
 
-Dependencies could also be declared using SBT style.
+Now call the def from the command line:
 
-```scala
-class Build(val context: cbt.Context) extends PackageJars {
-...
-// sbt compatible dependencies definition
-override def dependencies =
-  super.dependencies ++
-    Resolver(mavenCentral).bind(
-      "com.typesafe.play" %% "play-json" % "2.4.4",
-      "joda-time" % "joda-time" % "2.9.2"
-    )
-...
-}      
+```
+$ cbt foo
 ```
 
-Now you can call methods of this class through cbt. Try `cbt foo`.
-You can see how your build is configured via overrides.
+As you can see it prints `asdf`. Adding tasks is that easy.
 
-call `cbt` to see a full list of available commands for this build.
+### Triggering tasks on file-changes
 
-Look into the class PackageBuild (and it's super class BasicBuild) in CBT's source code to see their
-details. The source code is really simple. Don't shy away from
-looking, even as a beginner. No crazy stuff, I promise ;). You
-can find the relevant code in CBT's stage2/BasicBuild.scala
+When you call a task, you can prefix it with `loop`.
+CBT then watches the source files, the build files and even CBT's own
+source code and re-runs the task when anything changes. If necessary,
+this forces CBT to re-build itself, the project's dependencies and the project itself.
 
-I order to keep executing the same command triggered by file changes use `cbt loop <command>`.
+Let's try it. Let's loop the run task. Call this from the shell:
 
-You can find example builds in CBT's own `test/` folder.
-Not all of them have a build file, in which case CBT uses the default
-cbt.BasicBuild.
+```
+$ cbt loop run
+```
 
-A folder `build/` can have its own folder `build/` inside in order
-to add source or maven dependencies to your build. Eventually
-you'll be able to also choose the CBT and Scala versions for
-target builds. Make sure you extend cbt.BuilBuild instead of
-cbt.Build, in order to automatically trigger building of the
-target build.
+Now change `Main.scala and see how cbt picks it up and re-runs it.
+CBT is fast. It may already be done re-compiling and re-running before
+you managed to change windows back from your editor to the shell.
 
-cbt is fast. It uses Nailgun to keep the JVM hot. It uses the Java
-WatchService (respectively a fast OSX port of it) for instant triggering
-re-compilation on file changes. Use `cbt loop compile`.
+Try changing the build file and see how CBT reacts to it as well.
 
-CBT concepts
-------------
+### Adding tests
 
-There two essential primitives available in build scripts for composing
-modular projects:
+The simplest way to add tests is putting a few assertions into the previously
+created Main.scala and be done with it. Alternatively you can add a test
+framework plugin to your build file to use something more sophisticated.
 
-  1. Dynamically compiling and loading Build scripts in other
-     directories and calling methods (aka tasks) on them to compile,
-     get the classpath, ask for version numbers, etc.
+This however means that the class files of your tests will be included in the
+jar should you create one. If that's fine, you are done :). If it is not you
+need to create another project, which depends on your previous project. This
+project will be packaged separately or you can disable packaging there. Let's create
+such a project now.
 
-     This allows to do a lot of things just like that:
-     Multi-project builds, source dependencies, builds of builds and
-     allowing tests simply as dependent projects of the main project, etc.
+Your project containing tests can be anywhere but a recommended location is a
+sub-folder called `test/` in your main project. Let's create it and create a
+Main class and build file:
 
-  2. Maven dependencies
+```
+$ mkdir test
+$ cd test
+$ rm ../Main.scala
+$ cbt tools createMain
+$ cbt tools createBuild
+```
 
-     I wrote my own 50 LOC Maven resolver. It's super quick and I have
-     yet see it not to being able to handle something. I know cases
-     exist, but seem rare.
-     alexarchambault's Coursier can be used as a more complete drop-in.
+We also deleted the main projects Main.scala, because now that we created a new one
+we would have two classes with the same name on the classpath which can be very confusing.
 
-Build scripts also have access to a small unsurprising library for
-- triggering dependencies to build / download and get the classpath
-- compiling Java / Scala code using zinc with given class paths
-- running code
-- packaging jars
-- signing / publishing to sonatype/maven
+Now that we have a Main file in our test project, we can add some assertions to it.
+In order for them to see the main projects code, we still need to do one more thing - 
+add a `DirectoryDependency` to your test project's build file. There is a similar example
+in the generated build.scala. What you need is this:
 
-Scala.js support
-----------------
+```
+override def dependencies = super.dependencies ++ Seq(
+  DirectoryDependency( projectDirectory ++ "/.." )
+)
+```
 
-CBT supports cross-project Scala.js build.
-It preserves same structure as in SBT (https://www.scala-js.org/doc/project/cross-build.html)
+This successfully makes your test project's code see the main projects code.
+Add some class to your main project, e.g. `case class Foo(i: Int = 5)`. Now
+put an assertion into the Main class of your test project, e.g.
+`assert(Foo().i == 5)` and hit `cbt run` inside your test project.
 
- 1. Example for user scalajs project is in: `$CBT_HOME/cbt/examples/build-scalajs`
- 2. `$CBT_HOME/cbt compile`
-    Will compile JVM and JS sources
-    `$CBT_HOME/cbt jsCompile`
-    Will compile JS sources
-    `$CBT_HOME/cbt jvmCompile`
-    Will compile JVM sources
- 3. `$CBT_HOME/cbt fastOptJS` and `$CBT_HOME/cbt fullOptJS`
-    Same as in Scala.js sbt project
+Make sure you deleted your main projects class Main when running your tests.
 
- Note: Scala.js support is under ongoing development.
+Congratulations, you successfully created a dependent project and ran your tests.
 
- Currently missing features:
- * No support for jsDependencies:
-   It means that all 3rd party dependencies should added manually, see scalajs build example
- * No support for test
+### Multi-projects Builds
 
-Missing features in comparison with SBT
----------------------------------------
+A single build only handles a single project in CBT. So there isn't exactly
+such a things as a Multi-project Build. Instead you can simply write multiple
+projects that depend on each other. We have already done that with tests above,
+but you can do the exact same thing to modularize your project into multiple ones.
 
-Not implemented yet, but rather easily possible without API changes or
-major refactors is concurrently building dependencies and
-running tests. Right now it is sequential.
-Maven downloads already happen in parallel as well as some tasks like packaging.
+### Reproducible builds
 
-Another edge case that may need a solution is dynamically overwriting
-tasks. SBT allows that. Classes and traits are static. The only use
-cases I know are debugging, cross builds and the sbt release plugin. A
-solution could be code generating traits at build-time and mixing them
-in ad-hoc. It's a build-tool after all. Build-time code-generation and
-class loading is not rocket science. But there may be simpler solutions
-for the cases at hand. And they are edge cases anyways.
+To achieve reproducible builds, you'll need to tie your build files to a particular
+CBT-version. It doesn't matter what version of CBT you are actually running,
+as long as the `BuildInterface` is compatible (which should be true for a large number
+of versions and we may find a better solution long term. If you see a compile error 
+during compilation of CBT itself that some method in BuildInterface was not
+implemented or incorrectly implemented, you may be running an incompatible CBT
+version. We'll try to fix that later, but for now you might have to checkout
+the required hash of CBT by hand.).
 
-Bash completions
-----------------
+When you specify a particular version, CBT will use that one instead of the installed one.
+
+You can specify one by adding one line right before `class Build`. It looks like this:
+
+```
+// cbt:https://github.com/cvogt/cbt.git#75c32537cd8f29f9d12db37bf06ad942806f02393
+class Build...
+```
+
+The URL points to any git repository containing one of CBT's forks. You currently
+have to use a stable reference - i.e. a hash or tag. (Checkouts are currently not
+updated. If you refer to a branch or tag which is moved on the remote, CBT
+will not realize that and keep using the old version).
+
+### Using CBT like a boss
+
+Do you own your Build Tool or does your Build Tool own you? CBT makes it easy for YOU
+to be in control. We try to work on solid documentation, but even good 
+documentation never tells the whole truth. Documentation can tell how to use
+something and why things are happening, but only the code can tell all the
+details of what exactly is happening. Reading the code can be intimidating for
+many Scala libraries, but not so with CBT. The source code is easy to read
+to the point that even Scala beginners will be able to understand it. So
+don't be afraid to actually look under the hood and check out what's happening.
+
+And guess what, you already have the source code on your disk, because
+you installed CBT by cloning it's git repository. You can even debug CBT and
+your build files in an interactive debugger like IntelliJ after some minor setup.
+
+Finally, you can easily change CBT's code. Then CBT re-builds itself when you try
+to use it the next time. This means any changes you make are instantly reflected.
+This and the simple code make it super easy to fix bugs or add features yourself
+and feed them back into main line CBT.
+
+
+When debugging things, it can help to enable CBT's debug logging by passing
+`-Dlog=all` to CBT (or a logger name instead of `all`).
+
+
+Other design decisions
+--------------------
+
+CBT tries to couple it's code very loosely. OO is used for configuration in build files.
+Interesting logic is in simple supporting library classes/objects, which can be used
+independently. You could even build a different configuration api than OO on top of them.
+
+
+Known limitations
+--------------------------
+- currently CBT supports no generic task scoping. A solution is known, but not implemented.
+  For now manually create intermediate tasks which serve as scoping for
+  known situations and encode the scope in the name, e.g. fastOptScalaJsOptions
+- currently CBT supports no dynamic overrides of tasks. A solution is known, but not implemented.
+  scalaVersion and version are passed through the context instead for dynamic overrides.
+- there is currently no built-in support for resources being added to a jar.
+  Should be simple to add, consider a PR
+- test framework support is currently a bit spotty but actively being worked on
+- concurrent task and build execution is currently disabled
+- CBT uses it's own custom built maven resolver, which is really fast,
+  but likely does not work in some edge cases. Those may or may not be easy to fix.
+  We should add optional coursier integration back for a more complete solution.
+
+Known bugs
+-------------
+- currently there is a bug in CBT where dependent builds may miss changes in the things
+  they depend on. Deleting all target directories and starting from scratch helps.
+- There are some yet unknown bugs which can be solved by killing the nailgun background process
+  and/or re-running the desired cbt task multiple times until it succeeds.
+- if you ever see no output from a command but expect some, make sure you are in the right directory
+  and try to see if any of the above recommendations help
+
+
+Shell completions
+-------------------
+
+### Bash completions
 To auto-complete cbt task names in bash do this:
 
 ```
@@ -193,13 +370,16 @@ for f in ~/.bash_completion.d/*; do
 done
 ```
 
-Fish shell completions
-----------------
-copy this line into your fish configuration, on OSX: /.config/fish/config.fish
+### Fish shell completions
+
+copy this line into your fish configuration, on OSX: `/.config/fish/config.fish`
+
+```
 complete -c cbt -a '(cbt taskNames)'
+```
+
+### Zsh completions
 
-Zsh completions
----------------
 ##### Manual installation
 Add the following to your `.zshrc`
 ```
@@ -215,3 +395,40 @@ Then enable it in your `.zshrc`:
 ```
 plugins=( ... cbt)
 ```
+
+Plugin-author guide
+--------------------
+
+A CBT plugin is a trait that is mixed into a Build class.
+Only use this trait only for wiring things together.
+Don't put logic in there. Instead simply call methods
+on a separate class or object which serves as a library
+for your actual logic. It should be callable and testable
+outside of a Build class. This way the code of your plugin
+will be easier to test and easier to re-use. Feel free
+to make your logic rely on CBT's logger.
+
+ See `plugins/` for examples.
+
+Scala.js support
+----------------
+
+CBT supports cross-project Scala.js builds.
+It preserves same structure as in SBT (https://www.scala-js.org/doc/project/cross-build.html)
+
+ 1. Example for user scalajs project is in: `$CBT_HOME/cbt/examples/build-scalajs`
+ 2. `$CBT_HOME/cbt compile`
+    Will compile JVM and JS sources
+    `$CBT_HOME/cbt jsCompile`
+    Will compile JS sources
+    `$CBT_HOME/cbt jvmCompile`
+    Will compile JVM sources
+ 3. `$CBT_HOME/cbt fastOptJS` and `$CBT_HOME/cbt fullOptJS`
+    Same as in Scala.js sbt project
+
+ Note: Scala.js support is under ongoing development.
+
+ Currently missing features:
+ * No support for jsDependencies:
+   It means that all 3rd party dependencies should added manually, see scalajs build example
+ * No support for test