From 163bbc8f1d78750196b30a46a888e7e65ace5454 Mon Sep 17 00:00:00 2001 From: Christopher Vogt Date: Sat, 11 Mar 2017 16:50:27 -0500 Subject: add developer documentation about cbt version compatibility --- doc/cbt-developer/version-compatibility.md | 50 +++++++++++ doc/design.md | 139 +++++++++++++++++++++++++++++ doc/design.txt | 139 ----------------------------- 3 files changed, 189 insertions(+), 139 deletions(-) create mode 100644 doc/cbt-developer/version-compatibility.md create mode 100644 doc/design.md delete mode 100644 doc/design.txt diff --git a/doc/cbt-developer/version-compatibility.md b/doc/cbt-developer/version-compatibility.md new file mode 100644 index 0000000..986bb37 --- /dev/null +++ b/doc/cbt-developer/version-compatibility.md @@ -0,0 +1,50 @@ +## Why does CBT version compatiblity matter? +CBT allows fixing the version used using the `// cbt: ` annotation. +This means that the CBT version you installed will download the +CBT version referenced there and use that instead. CBT also +allows composing builds, which require different CBT versions. +In order for all of this to work, different CBT versions need to +be able to talk to one another. + +This chapter is about how the current solution works and about +the pitfalls involved which can make a new version of CBT +incompatible with older versions. + +## How can compatibility be broken? +The current solution mostly relies on the Java interfaces in +the `compability/` folder. Changing the Java interface in a +non-backwards-compatible way means making the CBT incompatible with +older versions. Java 8 default methods make this a whole lot easier +and this is the main reason CBT relies on Java 8. But we also +want to keep this interfaces as small as possible in order to +minimize the risk. + +However there are more things that can break compatibility when changed: +- the format of the `// cbt: ` version string +- the name and format of Build classes (or files) that CBT looks for +- communication between versions via reflection in particular + - how the TrapSecurityManager of each CBT version talks to the + installed TrapSecurityManager via reflection + +## How to detect accidental breakages? + +CBT's tests have a few tests with `// cbt: ` annotations and some +reference libraries as Git dependencies that use these annotations. +Many incompatibilities will lead to these tests failing, either with +a compilation error against the `compatiblity/` interfaces or with +a runtime exception or unexpected behavior in case other things are +broken. + +## How an we improve the situation in the long run? + +In the long run we should think about how to reduce the risk +or sometimes even unavoidability of incompatibilities. +The unavoidability mostly stems from limitations of what Java +interfaces can express. However Java 8 interfaces mostly work +fairly well. We should consider using them for the cases that +currently use reflection instead of reflection. + +If Java 8 interfaces still turn out to be a problem in the long run, +we could consider an interface that we control completely, e.g. +an internal serialization formation, which CBT versions use to talk +to each other. diff --git a/doc/design.md b/doc/design.md new file mode 100644 index 0000000..28bbdaa --- /dev/null +++ b/doc/design.md @@ -0,0 +1,139 @@ +# Design + +This chapter explains some of CBT's less obvious design decisions and +the reasons behind them. Hopefully this answers some of the questions +often unanswered when trying to understand a new tool. + + +## Why does CBT use inheritance? + +First of all CBT uses classes, because needs some place to put tasks and +name them. Methods in classes are one way to do it. Methods calling each +other allows to effectively build a graph of dependent tasks. + +Inheritance on top of that allows patching this graph to insert additional +steps, e.g. formatting the code before compiling it. Patching of the task +graph is what you do with sbt andso with CBT via inheritance/overrides. + +This was also discussed in gitter here: https://gitter.im/cvogt/cbt?at=58a95663de50490822e869e5 + +Taking a graph an contrinuously patching it can be confusing, which is +why inheritance is confusing. You can even build non-terminating call +cycles. CBT's logic is not coupled to the inheritance layer. You can +write CBT builds without inheritance. They require a bit more code, but +may end up easier to maintain and understand. + +## Task composition and aborting failed tasks and dependents + +In CBT build tasks are methods. Task can depend on each +other by invoking each other. E.g. `package` is a method that invokes `compile`. +Build tasks can fail. By convention, if a task fails it is expected to throw +an Exception in order to also abort the execution of dependent tasks. When +`compile` finds compile errors it throws an exception and thereby also +drops out of the `package` execution pth. CBT catches and handles the +exception and returns control back to the user. + +This design was chosen for simplicity and because build code lives in a +world with exceptions anyways as a lot of it is directly or indirectly +using java libraries. We might reconsider this design at some point. Or not. +The hope is that Exceptions are without major drawbacks and more +approachable to newcomers than monadic error handling, which would require +wrapper types and value composition via .map and for-expressions. Not using +a Monad however also means that CBT cannot reason about task composition +automatically, which means parallel task execution is not automatic, but +manual and opt-in where wanted. + +## Why do CBT plugins use case classes where methods seem simpler? + +In CBT plugins you may see + +``` +trait SomePlugin{ + def doSomething = SomePluginLib(...context).doSomething( a, b, ... ) +} +case class SomePluginLib(...context...)(implicit ...more context...){ + case class doSomething(a: A, b: B, ...){ + def apply = { + // logic + } + } +} +``` + +Why this? This seems much simpler: + +``` +trait SomePlugin{ + def doSomething = SomePluginLib.doSomething( a, b, ... )(...context) +} +object SomePluginLib + def doSomething(a: A, b: B, ...)(...context...)(implicit ...more context...) = { + // logic + } +} +``` + +The reason is that the former allows easy patching while +the second does not. Let's pretend your build is this: +``` +class Build(val context: Context) extends SomePlugin{ + override def doSomething = super.doSomething.copy( b = someOtherB ) +} +``` +Such a simple replacement of `b` while keeping all other arguments would +not be easily possible if doSomething was a def not a case class. + +## What is newBuild and why do we need it? + +Methods in a class can call each other and thereby effectively form a graph. +Subclasses can patch this graph. Sometimes you want to create several +different variants of a graph. Let's say one building artifacts with a +-SNAPSHOT version and another one building a stable release. Or one with +scalaVersion 2.10 and one with scalaVersion 2.11. Or one optmizing your +code and one not optimizing it. You can subclass your build once for +each alternative. That's fine if you know which class to subclass, but +plugins for your build do not know what your concrete build class is +and cannot produce subclasses for it. Example + +``` +class Build(val context: Context){ + // this works fine + def with210 = new Build(context){ + override def scalaVersion = "2.10" + } + def with211 = new Build(context){ + override def scalaVersion = "2.11" + } +} +``` +Imagine however +``` +// defined by you: +class Build(val context: Context) extends CrossVersionPlugin +// defined by the plugin author: +trait CrossVersionPlugin{ + // this does not compile, Build does not exists when the plugin is compiled + def with210 = new Build(context){ + override def scalaVersion = "2.10" + } + def with211 = new Build(context){ + override def scalaVersion = "2.11" + } +} +``` +Luckily CBT can cheat because it has a compiler available. +``` +// defined by the plugin author: +trait CrossVersionPlugin{ + // method `newBuild` generates a new subclass of Build at runtime and creates an instance. + def with210 = newBuild{""" + override def scalaVersion = "2.10" + """} + def with211 = newBuild{""" + override def scalaVersion = "2.11" + """} +} +``` + +Problem solved. In fact this allows for a very, very flexible way of +creating differents variants of your build. diff --git a/doc/design.txt b/doc/design.txt deleted file mode 100644 index 28bbdaa..0000000 --- a/doc/design.txt +++ /dev/null @@ -1,139 +0,0 @@ -# Design - -This chapter explains some of CBT's less obvious design decisions and -the reasons behind them. Hopefully this answers some of the questions -often unanswered when trying to understand a new tool. - - -## Why does CBT use inheritance? - -First of all CBT uses classes, because needs some place to put tasks and -name them. Methods in classes are one way to do it. Methods calling each -other allows to effectively build a graph of dependent tasks. - -Inheritance on top of that allows patching this graph to insert additional -steps, e.g. formatting the code before compiling it. Patching of the task -graph is what you do with sbt andso with CBT via inheritance/overrides. - -This was also discussed in gitter here: https://gitter.im/cvogt/cbt?at=58a95663de50490822e869e5 - -Taking a graph an contrinuously patching it can be confusing, which is -why inheritance is confusing. You can even build non-terminating call -cycles. CBT's logic is not coupled to the inheritance layer. You can -write CBT builds without inheritance. They require a bit more code, but -may end up easier to maintain and understand. - -## Task composition and aborting failed tasks and dependents - -In CBT build tasks are methods. Task can depend on each -other by invoking each other. E.g. `package` is a method that invokes `compile`. -Build tasks can fail. By convention, if a task fails it is expected to throw -an Exception in order to also abort the execution of dependent tasks. When -`compile` finds compile errors it throws an exception and thereby also -drops out of the `package` execution pth. CBT catches and handles the -exception and returns control back to the user. - -This design was chosen for simplicity and because build code lives in a -world with exceptions anyways as a lot of it is directly or indirectly -using java libraries. We might reconsider this design at some point. Or not. -The hope is that Exceptions are without major drawbacks and more -approachable to newcomers than monadic error handling, which would require -wrapper types and value composition via .map and for-expressions. Not using -a Monad however also means that CBT cannot reason about task composition -automatically, which means parallel task execution is not automatic, but -manual and opt-in where wanted. - -## Why do CBT plugins use case classes where methods seem simpler? - -In CBT plugins you may see - -``` -trait SomePlugin{ - def doSomething = SomePluginLib(...context).doSomething( a, b, ... ) -} -case class SomePluginLib(...context...)(implicit ...more context...){ - case class doSomething(a: A, b: B, ...){ - def apply = { - // logic - } - } -} -``` - -Why this? This seems much simpler: - -``` -trait SomePlugin{ - def doSomething = SomePluginLib.doSomething( a, b, ... )(...context) -} -object SomePluginLib - def doSomething(a: A, b: B, ...)(...context...)(implicit ...more context...) = { - // logic - } -} -``` - -The reason is that the former allows easy patching while -the second does not. Let's pretend your build is this: -``` -class Build(val context: Context) extends SomePlugin{ - override def doSomething = super.doSomething.copy( b = someOtherB ) -} -``` -Such a simple replacement of `b` while keeping all other arguments would -not be easily possible if doSomething was a def not a case class. - -## What is newBuild and why do we need it? - -Methods in a class can call each other and thereby effectively form a graph. -Subclasses can patch this graph. Sometimes you want to create several -different variants of a graph. Let's say one building artifacts with a --SNAPSHOT version and another one building a stable release. Or one with -scalaVersion 2.10 and one with scalaVersion 2.11. Or one optmizing your -code and one not optimizing it. You can subclass your build once for -each alternative. That's fine if you know which class to subclass, but -plugins for your build do not know what your concrete build class is -and cannot produce subclasses for it. Example - -``` -class Build(val context: Context){ - // this works fine - def with210 = new Build(context){ - override def scalaVersion = "2.10" - } - def with211 = new Build(context){ - override def scalaVersion = "2.11" - } -} -``` -Imagine however -``` -// defined by you: -class Build(val context: Context) extends CrossVersionPlugin -// defined by the plugin author: -trait CrossVersionPlugin{ - // this does not compile, Build does not exists when the plugin is compiled - def with210 = new Build(context){ - override def scalaVersion = "2.10" - } - def with211 = new Build(context){ - override def scalaVersion = "2.11" - } -} -``` -Luckily CBT can cheat because it has a compiler available. -``` -// defined by the plugin author: -trait CrossVersionPlugin{ - // method `newBuild` generates a new subclass of Build at runtime and creates an instance. - def with210 = newBuild{""" - override def scalaVersion = "2.10" - """} - def with211 = newBuild{""" - override def scalaVersion = "2.11" - """} -} -``` - -Problem solved. In fact this allows for a very, very flexible way of -creating differents variants of your build. -- cgit v1.2.3