Rewrite README for new layout

3 files changed, 216 insertions, 195 deletions

diff --git a/README.md b/README.md
index a2d3649..c44f820 100644
--- a/README.md
+++ b/README.md
@@ -7,227 +7,94 @@ cherry-picked onto the [1.x
 branch](https://github.com/drivergroup/driver-core/tree/1.x).
 
 ----
+[![Build Status](https://travis-ci.com/drivergroup/driver-core.svg?token=S4oyfBY3YoEdLmckujJx&branch=master)](https://travis-ci.com/drivergroup/driver-core)
 
-# Driver Core Library for Scala Services [![Build Status](https://travis-ci.com/drivergroup/driver-core.svg?token=S4oyfBY3YoEdLmckujJx&branch=master)](https://travis-ci.com/drivergroup/driver-core)
+# Driver Core Library for Scala Services
 
 Multi-cloud utilities and application initialization framework.
 
 This library offers many common utilities for building applications
-that run on multiple environments, including Google Cloud, Ali Cloud,
+that run in multiple environments, including Google Cloud, Ali Cloud,
 and of course on development machines.
 
+## Highlights
 
-# Overview
+- Cloud agnostic.
 
-*This section applies to the 1.x series of core. The current development master branch may include changes not described here.*
+- Sensible defaults. *Applications that use the initialization
+  framework can run out-of-the box on cloud or locally, with minimal
+  config required.*
 
-Core library is used to provide ways to implement practices established in [Driver service template](http://github.com/drivergroup/driver-template) (check its [README.md](https://github.com/drivergroup/driver-template/blob/master/README.md)).
+- Distributed tracing built into all major utilities. *Significant
+  actions are logged and reported automatically to an OpenTrace
+  aggregator.*
 
-## Components
+- Extensible. *Utilities are built to be used standalone. A
+  trait-based initialization framework provides utility instances for
+  common use-cases.*
 
- * `core package` provides `Id` and `Name` implementations (with equal and ordering), utils for ScalaZ `OptionT`, and also `make` and `using` functions and `@@` (tagged) type,
- * `tagging` Utilities for tagging primitive types for extra type safety, as well as some tags that involve extra transformation upon deserializing with spray,
- * `config` Contains method `loadDefaultConfig` with default way of providing config to the application,
- * `domain` Common generic domain objects, e.g., `Email` and `PhoneNumber`,
- * `messages` Localization messages supporting different locales and methods to read from config,
- * `database` Method for database initialization from config, `Id`, `Name`, `Time`, `Date` etc. mapping, schema lifecycle and base classes to implement and test `Repository` (data access objects),
- * `rest` Wrapper over call to external REST API, authorization, context headers, XSS protection, does logging and allows to add Swagger to a service,
- * `auth` Basic entities for authentication and authorization: `User`, `Role` `Permission` `AuthToken`, `AuthCredentials` etc.,
- * `swagger` Contains custom `AbstractModelConverter` to customize Swagger JSON with any Scala  JSON formats created by, for instance, Spray JSON,
- * `json` Json formats for `Id`, `Name`, `Time`, `Revision`, `Email`, `PhoneNumber`, `AuthCredentials` and converters for GADTs, enums and value classes,
- * `file` Interface `FileStorage` and file storage implementations with GCS, S3 and local FS,
- * `app` Base class for Driver service, which initializes swagger, app modules and its routes.
- * `generators` Set of functions to prototype APIs. Combines with `faker` package,
- * `stats` Methods to collect system stats: memory, cpu, gc, file system space usage,
- * `logging` Custom Driver logging layout (not finished yet).
+## Overview
 
-Dependencies of core modules might be found in [Dependencies of the Modules diagram](https://github.com/drivergroup/driver-template/blob/master/Modules%20dependencies.pdf) file in driver-template repository in "Core component dependencies" section.
+### Components
 
-## Examples
+This project is split into multiple submodules, each providing
+specific functionality.
 
-### Functions `make` and `using`
-Those functions are especially useful to make procedural legacy Java APIs more functional and make scope of its usage more explicit. Runnable examples of its usage might be found in [`CoreTest`](https://github.com/drivergroup/driver-core/blob/master/src/test/scala/xyz/driver/core/CoreTest.scala), e.g.,
+Project          | Description
+-----------------|------------
+`core`           | *(deprecated)* Previous application initialization framework.
+`core-database`  | Utilities for working with databases, slick in particular.
+`core-init`      | Mini-framework that offers default instances of many utilities, configured for the current platform.
+`core-messaging` | Library for abstracting over message buses such as Google PubSub.
+`core-reporting` | Combined tracing and logging library.
+`core-rest`      | Abstractions to represent RESTful services, discovery and client implementations.
+`core-storage`   | Object storage utilities.
+`core-types`     | Type definitions that are commonly used by applications.
+`core-util`      | Other utilities that do not belong anywhere else. **Note that this is a staging place for code that does not have its own submodule yet. Do not depend on it externally!**
 
-    useObject(make(new ObjectWithProceduralInitialization) { o =>
-      o.setSetting1(...) // returns Unit
-      o.setSetting2(...) // returns Unit
-      o.setSetting3(...) // returns Unit
-    })
+These components and their internal dependencies can be represented
+with the following graph.
 
-    // instead of
-    val o = new ObjectWithProceduralInitialization
-    o.setSetting1(...)
-    o.setSetting2(...)
-    o.setSetting3(...)
+![Inter-Component Dependencies](documentation/components.svg)
 
-    useObject(o)
+### Dependency
 
-and
-
-    using(... open output stream ...) { os =>
-      os.write(...)
-    }
-
-    // it will be close automatically
-
-### `OptionT` utils
-Before
-
-```
-OptionT.optionT[Future](service.getRecords(id).map(Option.apply))
-OptionT.optionT(service.doSomething(id).map(_ => Option(())))
-
-// Do not want to stop and return `None`, if `produceEffect` returns `None`
-for {
-  x <- service.doSomething(id)
-  _ <- service.produceEffect(id, x).map(_ => ()).orElse(OptionT.some[Future, Unit](())))
-} yield x
-```
-
-after
-
-```
-service.getRecords(id).toOptionT
-service.doSomething(id).toUnitOptionT
-
-// Do not want to stop and return `None` if `produceEffect` returns `None`
-for {
-  x <- service.doSomething(id)
-  _ <- service.produceEffect(id, x).continueIgnoringNone
-} yield x
-```
-
-### `@@` or Tagged types
-
-For type definitions, the only import required is
-
-```scala
-import xyz.driver.core.@@
-```
-
-which provides just the ability to tag types: `val value: String @@ Important`. Two `String`s with different tags will
-be distinguished by the compiler, helping reduce the possibility of mixing values passed into methods with several
-arguments of identical types.
-
-To work with tags in actual values, use the following convenience methods:
-
-```scala
-import xyz.driver.core.tagging._
-
-val str = "abc".tagged[Important]
-```
-
-or go back to plain (say, in case you have an implicit for untagged value)
-
-```scala
-// val trimmedExternalId: String @@ Trimmed = "123"
-
-Trials.filter(_.externalId === trimmedExternalId.untag)
+All components are published together under the same version ([latest
+version](https://github.com/drivergroup/driver-core/releases)).
+```sbt
+libraryDependencies += "xyz.driver" %% "core-<component>" % "<version>"
 ```
 
-### `Time` and `TimeProvider`
-
-**NOTE: The contents of this section has been deprecated - use java.time.Clock instead**
-
-Usage examples for `Time` (also check [TimeTest](https://github.com/drivergroup/driver-core/blob/master/src/test/scala/xyz/driver/core/TimeTest.scala) for more examples).
-
-    Time(234L).isAfter(Time(123L))
-
-    Time(123L).isBefore(Time(234L))
-
-    Time(123L).advanceBy(4 days)
-
-    Seq(Time(321L), Time(123L), Time(231L)).sorted
-
-    startOfMonth(Time(1468937089834L)) should be (Time(1467381889834L))
-
-    textualDate(Time(1468937089834L)) should be ("July 19, 2016")
-
-    textualTime(Time(1468937089834L)) should be ("Jul 19, 2016 10:04:49 AM")
-
-    TimeRange(Time(321L), Time(432L)).duration should be(111.milliseconds)
-
-
-### Generators
-Example of how to generate a case class instance,
-
-    import com.drivergrp.core._
-
-    Consumer(
-         generators.nextId[Consumer],
-         Name[Consumer](faker.Name.name),
-         faker.Lorem.sentence(word_count = 10))
-
-
-For more examples check [project tests](https://github.com/drivergroup/driver-core/tree/master/src/test/scala/xyz/driver/core) or [service template](http://github.com/drivergroup/driver-template) repository.
-
-### App
-
-To start a new application using standard Driver application class, follow this pattern:
-
-    object MyApp extends App {
-
-      new DriverApp(BuildInfo.version,
-                    BuildInfo.gitHeadCommit.getOrElse("None"),
-                    modules = Seq(myModule1, myModule2),
-                    time, log, config,
-                    interface = "::0", baseUrl, scheme, port)
-                   (servicesActorSystem, servicesExecutionContext).run()
-    }
-
-### REST
-With REST utils, for instance, you can use the following directives in [akka-http](https://github.com/akka/akka-http) routes, as follows
-
-    sanitizeRequestEntity { // Prevents XSS
-      serviceContext { implicit ctx => // Extracts context headers from request
-        authorize(CanSeeUser(userId)) { user => // Authorizes and extracts user
-          // Your code using `ctx` and `user`
-        }
-      }
-    }
-
-### Swagger
-Swagger JSON formats built using reflection can be overriden by using `CustomSwaggerJsonConverter` at the start of your application initialization in the following way:
-
-    ModelConverters.getInstance()
-      .addConverter(new CustomSwaggerJsonConverter(Json.mapper(),
-         CustomSwaggerFormats.customProperties, CustomSwaggerFormats.customObjectsExamples))
-
-### Locale
-Locale messages can be initialized and used in the following way,
-
-    val localeConfig = config.getConfig("locale")
-    val log = com.typesafe.scalalogging.Logger(LoggerFactory.getLogger(classOf[MyClass]))
-
-    val messages = xyz.driver.core.messages.Messages.messages(localeConfig, log, Locale.US)
-
-    messages("my.locale.message")
-    messages("my.locale.message.with.param", parameter)
-
-
-### System stats
-Stats it gives access to are,
-
-    xyz.driver.core.stats.SystemStats.memoryUsage
-
-    xyz.driver.core.stats.SystemStats.availableProcessors
-
-    xyz.driver.core.stats.SystemStats.garbageCollectorStats
-
-    xyz.driver.core.stats.SystemStats.fileSystemSpace
+### External dependencies
 
-    xyz.driver.core.stats.SystemStats.operatingSystemStats
+This project has quite a few external dependencies. See
+[build.sbt](build.sbt) for a list of them. Some notable Scala
+libraries used are:
 
+- [akka-http](https://doc.akka.io/docs/akka-http/current/)
+- [akka-stream](https://doc.akka.io/docs/akka/current/stream/)
+- [enumeratum](https://github.com/lloydmeta/enumeratum#enumeratum------)
+- [scala-async](https://github.com/scala/scala-async)
+- [slick](http://slick.lightbend.com/)
+- [spray-json](https://github.com/spray/spray-json)
+- [sttp](https://github.com/softwaremill/sttp)
 
-## Running
+## Example Usage
 
-1. Compile everything and test:
+*TODO*
 
-        $ sbt test
+## Building
 
-2. Publish to local repository, to use changes in depending projects:
+This project uses [sbt](https://www.scala-sbt.org/) as build tool. It
+has grown organically over time, but all new code should follow the
+best practices outlined
+[here](https://style.driver.engineering/scala.html).
 
-        $ sbt publishLocal
+It is set up to automatically build, test, and publish a release when
+a Git tag is pushed that matches the regular expression
+`v[0-9].*`. Hence, to publish a new version "1.2.3", simply create a
+tag "v1.2.3" and `git push --tags` to origin.
 
-3. In order to release a new version of core, merge your PR, tag the HEAD master commit with the next version
-   (don't forget the "v." prefix) and push tags - Travis will release the artifact automatically!
+## Copying
+Copyright 2016-2018 Driver Inc. Released under the Apache License,
+Version 2.0.
diff --git a/documentation/components.dot b/documentation/components.dot
new file mode 100644
index 0000000..f00dff3
--- /dev/null
+++ b/documentation/components.dot
@@ -0,0 +1,21 @@
+digraph core {
+
+	rankdir = BT;
+
+	"core-reporting";
+	"core-storage" -> "core-reporting";
+	"core-messaging" -> "core-reporting";
+	"core-database" -> "core-reporting";
+	"core-database" -> "core-types";
+	"core-rest" -> "core-reporting";
+	"core-rest" -> "core-types";
+	"core-init" -> "core-storage";
+	"core-init" -> "core-messaging";
+	"core-init" -> "core-database";
+	"core-init" -> "core-rest";
+	"core-init" -> "core-reporting";
+
+	core [color=red];
+	core -> "core-init" [color=red];
+
+}
diff --git a/documentation/components.svg b/documentation/components.svg
new file mode 100644
index 0000000..0531399
--- /dev/null
+++ b/documentation/components.svg
@@ -0,0 +1,133 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by graphviz version 2.40.1 (20161225.0304)
+ -->
+<!-- Title: core Pages: 1 -->
+<svg width="542pt" height="260pt"
+ viewBox="0.00 0.00 542.09 260.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 256)">
+<title>core</title>
+<polygon fill="#ffffff" stroke="transparent" points="-4,4 -4,-256 538.09,-256 538.09,4 -4,4"/>
+<!-- core&#45;reporting -->
+<g id="node1" class="node">
+<title>core&#45;reporting</title>
+<ellipse fill="none" stroke="#000000" cx="285.9452" cy="-234" rx="61.1893" ry="18"/>
+<text text-anchor="middle" x="285.9452" y="-230.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;reporting</text>
+</g>
+<!-- core&#45;storage -->
+<g id="node2" class="node">
+<title>core&#45;storage</title>
+<ellipse fill="none" stroke="#000000" cx="53.9452" cy="-162" rx="53.8905" ry="18"/>
+<text text-anchor="middle" x="53.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;storage</text>
+</g>
+<!-- core&#45;storage&#45;&gt;core&#45;reporting -->
+<g id="edge1" class="edge">
+<title>core&#45;storage&#45;&gt;core&#45;reporting</title>
+<path fill="none" stroke="#000000" d="M93.5638,-174.2954C132.345,-186.331 191.6297,-204.7297 234.1337,-217.9206"/>
+<polygon fill="#000000" stroke="#000000" points="233.155,-221.2815 243.7431,-220.9028 235.2299,-214.596 233.155,-221.2815"/>
+</g>
+<!-- core&#45;messaging -->
+<g id="node3" class="node">
+<title>core&#45;messaging</title>
+<ellipse fill="none" stroke="#000000" cx="191.9452" cy="-162" rx="66.0889" ry="18"/>
+<text text-anchor="middle" x="191.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;messaging</text>
+</g>
+<!-- core&#45;messaging&#45;&gt;core&#45;reporting -->
+<g id="edge2" class="edge">
+<title>core&#45;messaging&#45;&gt;core&#45;reporting</title>
+<path fill="none" stroke="#000000" d="M214.2219,-179.063C226.6308,-188.5677 242.254,-200.5344 255.7488,-210.8708"/>
+<polygon fill="#000000" stroke="#000000" points="253.7613,-213.7573 263.8284,-217.0595 258.0179,-208.2001 253.7613,-213.7573"/>
+</g>
+<!-- core&#45;database -->
+<g id="node4" class="node">
+<title>core&#45;database</title>
+<ellipse fill="none" stroke="#000000" cx="474.9452" cy="-162" rx="59.2899" ry="18"/>
+<text text-anchor="middle" x="474.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;database</text>
+</g>
+<!-- core&#45;database&#45;&gt;core&#45;reporting -->
+<g id="edge3" class="edge">
+<title>core&#45;database&#45;&gt;core&#45;reporting</title>
+<path fill="none" stroke="#000000" d="M437.6049,-176.2249C407.6744,-187.627 365.4902,-203.6971 333.2074,-215.9954"/>
+<polygon fill="#000000" stroke="#000000" points="331.6298,-212.8509 323.5309,-219.6816 334.1218,-219.3923 331.6298,-212.8509"/>
+</g>
+<!-- core&#45;types -->
+<g id="node5" class="node">
+<title>core&#45;types</title>
+<ellipse fill="none" stroke="#000000" cx="443.9452" cy="-234" rx="47.3916" ry="18"/>
+<text text-anchor="middle" x="443.9452" y="-230.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;types</text>
+</g>
+<!-- core&#45;database&#45;&gt;core&#45;types -->
+<g id="edge4" class="edge">
+<title>core&#45;database&#45;&gt;core&#45;types</title>
+<path fill="none" stroke="#000000" d="M467.1226,-180.1686C463.6452,-188.2453 459.478,-197.9239 455.642,-206.8332"/>
+<polygon fill="#000000" stroke="#000000" points="452.3979,-205.5176 451.6579,-216.0866 458.8273,-208.2859 452.3979,-205.5176"/>
+</g>
+<!-- core&#45;rest -->
+<g id="node6" class="node">
+<title>core&#45;rest</title>
+<ellipse fill="none" stroke="#000000" cx="355.9452" cy="-162" rx="41.6928" ry="18"/>
+<text text-anchor="middle" x="355.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;rest</text>
+</g>
+<!-- core&#45;rest&#45;&gt;core&#45;reporting -->
+<g id="edge5" class="edge">
+<title>core&#45;rest&#45;&gt;core&#45;reporting</title>
+<path fill="none" stroke="#000000" d="M339.7099,-178.6992C330.9557,-187.7035 319.978,-198.9948 310.2283,-209.0231"/>
+<polygon fill="#000000" stroke="#000000" points="307.5305,-206.7771 303.0691,-216.3868 312.5494,-211.6566 307.5305,-206.7771"/>
+</g>
+<!-- core&#45;rest&#45;&gt;core&#45;types -->
+<g id="edge6" class="edge">
+<title>core&#45;rest&#45;&gt;core&#45;types</title>
+<path fill="none" stroke="#000000" d="M375.4753,-177.9791C387.3842,-187.7228 402.7997,-200.3355 415.9875,-211.1255"/>
+<polygon fill="#000000" stroke="#000000" points="413.9037,-213.9428 423.8597,-217.5664 418.3364,-208.5251 413.9037,-213.9428"/>
+</g>
+<!-- core&#45;init -->
+<g id="node7" class="node">
+<title>core&#45;init</title>
+<ellipse fill="none" stroke="#000000" cx="285.9452" cy="-90" rx="40.0939" ry="18"/>
+<text text-anchor="middle" x="285.9452" y="-86.3" font-family="Times,serif" font-size="14.00" fill="#000000">core&#45;init</text>
+</g>
+<!-- core&#45;init&#45;&gt;core&#45;reporting -->
+<g id="edge11" class="edge">
+<title>core&#45;init&#45;&gt;core&#45;reporting</title>
+<path fill="none" stroke="#000000" d="M285.9452,-108.2377C285.9452,-132.799 285.9452,-176.7526 285.9452,-205.6459"/>
+<polygon fill="#000000" stroke="#000000" points="282.4453,-205.9103 285.9452,-215.9104 289.4453,-205.9104 282.4453,-205.9103"/>
+</g>
+<!-- core&#45;init&#45;&gt;core&#45;storage -->
+<g id="edge7" class="edge">
+<title>core&#45;init&#45;&gt;core&#45;storage</title>
+<path fill="none" stroke="#000000" d="M252.7053,-100.3158C213.6013,-112.4515 148.3163,-132.7124 103.2985,-146.6834"/>
+<polygon fill="#000000" stroke="#000000" points="102.2479,-143.3448 93.7347,-149.6515 104.3228,-150.0302 102.2479,-143.3448"/>
+</g>
+<!-- core&#45;init&#45;&gt;core&#45;messaging -->
+<g id="edge8" class="edge">
+<title>core&#45;init&#45;&gt;core&#45;messaging</title>
+<path fill="none" stroke="#000000" d="M265.5486,-105.6229C252.9101,-115.3035 236.4447,-127.9153 222.2928,-138.7551"/>
+<polygon fill="#000000" stroke="#000000" points="220.0846,-136.0377 214.274,-144.8971 224.3411,-141.5948 220.0846,-136.0377"/>
+</g>
+<!-- core&#45;init&#45;&gt;core&#45;database -->
+<g id="edge9" class="edge">
+<title>core&#45;init&#45;&gt;core&#45;database</title>
+<path fill="none" stroke="#000000" d="M316.732,-101.7283C347.0407,-113.2745 393.7166,-131.0558 428.5269,-144.3169"/>
+<polygon fill="#000000" stroke="#000000" points="427.4162,-147.639 438.0071,-147.9284 429.9083,-141.0976 427.4162,-147.639"/>
+</g>
+<!-- core&#45;init&#45;&gt;core&#45;rest -->
+<g id="edge10" class="edge">
+<title>core&#45;init&#45;&gt;core&#45;rest</title>
+<path fill="none" stroke="#000000" d="M302.1806,-106.6992C311.2805,-116.0592 322.7832,-127.8905 332.8116,-138.2055"/>
+<polygon fill="#000000" stroke="#000000" points="330.3176,-140.6611 339.7979,-145.3913 335.3366,-135.7816 330.3176,-140.6611"/>
+</g>
+<!-- core -->
+<g id="node8" class="node">
+<title>core</title>
+<ellipse fill="none" stroke="#ff0000" cx="285.9452" cy="-18" rx="27" ry="18"/>
+<text text-anchor="middle" x="285.9452" y="-14.3" font-family="Times,serif" font-size="14.00" fill="#000000">core</text>
+</g>
+<!-- core&#45;&gt;core&#45;init -->
+<g id="edge12" class="edge">
+<title>core&#45;&gt;core&#45;init</title>
+<path fill="none" stroke="#ff0000" d="M285.9452,-36.1686C285.9452,-43.869 285.9452,-53.0257 285.9452,-61.5834"/>
+<polygon fill="#ff0000" stroke="#ff0000" points="282.4453,-61.5867 285.9452,-71.5867 289.4453,-61.5868 282.4453,-61.5867"/>
+</g>
+</g>
+</svg>