From 32136fa99a67027a8273b635749b40e93998de0c Mon Sep 17 00:00:00 2001 From: Jakob Odersky Date: Mon, 17 Sep 2018 17:05:23 -0700 Subject: Rewrite README for new layout --- README.md | 257 +++++++++++-------------------------------- documentation/components.dot | 21 ++++ documentation/components.svg | 133 ++++++++++++++++++++++ 3 files changed, 216 insertions(+), 195 deletions(-) create mode 100644 documentation/components.dot create mode 100644 documentation/components.svg diff --git a/README.md b/README.md index a2d3649..c44f820 100644 --- a/README.md +++ b/README.md @@ -7,227 +7,94 @@ cherry-picked onto the [1.x branch](https://github.com/drivergroup/driver-core/tree/1.x). ---- +[![Build Status](https://travis-ci.com/drivergroup/driver-core.svg?token=S4oyfBY3YoEdLmckujJx&branch=master)](https://travis-ci.com/drivergroup/driver-core) -# Driver Core Library for Scala Services [![Build Status](https://travis-ci.com/drivergroup/driver-core.svg?token=S4oyfBY3YoEdLmckujJx&branch=master)](https://travis-ci.com/drivergroup/driver-core) +# Driver Core Library for Scala Services Multi-cloud utilities and application initialization framework. This library offers many common utilities for building applications -that run on multiple environments, including Google Cloud, Ali Cloud, +that run in multiple environments, including Google Cloud, Ali Cloud, and of course on development machines. +## Highlights -# Overview +- Cloud agnostic. -*This section applies to the 1.x series of core. The current development master branch may include changes not described here.* +- Sensible defaults. *Applications that use the initialization + framework can run out-of-the box on cloud or locally, with minimal + config required.* -Core library is used to provide ways to implement practices established in [Driver service template](http://github.com/drivergroup/driver-template) (check its [README.md](https://github.com/drivergroup/driver-template/blob/master/README.md)). +- Distributed tracing built into all major utilities. *Significant + actions are logged and reported automatically to an OpenTrace + aggregator.* -## Components +- Extensible. *Utilities are built to be used standalone. A + trait-based initialization framework provides utility instances for + common use-cases.* - * `core package` provides `Id` and `Name` implementations (with equal and ordering), utils for ScalaZ `OptionT`, and also `make` and `using` functions and `@@` (tagged) type, - * `tagging` Utilities for tagging primitive types for extra type safety, as well as some tags that involve extra transformation upon deserializing with spray, - * `config` Contains method `loadDefaultConfig` with default way of providing config to the application, - * `domain` Common generic domain objects, e.g., `Email` and `PhoneNumber`, - * `messages` Localization messages supporting different locales and methods to read from config, - * `database` Method for database initialization from config, `Id`, `Name`, `Time`, `Date` etc. mapping, schema lifecycle and base classes to implement and test `Repository` (data access objects), - * `rest` Wrapper over call to external REST API, authorization, context headers, XSS protection, does logging and allows to add Swagger to a service, - * `auth` Basic entities for authentication and authorization: `User`, `Role` `Permission` `AuthToken`, `AuthCredentials` etc., - * `swagger` Contains custom `AbstractModelConverter` to customize Swagger JSON with any Scala JSON formats created by, for instance, Spray JSON, - * `json` Json formats for `Id`, `Name`, `Time`, `Revision`, `Email`, `PhoneNumber`, `AuthCredentials` and converters for GADTs, enums and value classes, - * `file` Interface `FileStorage` and file storage implementations with GCS, S3 and local FS, - * `app` Base class for Driver service, which initializes swagger, app modules and its routes. - * `generators` Set of functions to prototype APIs. Combines with `faker` package, - * `stats` Methods to collect system stats: memory, cpu, gc, file system space usage, - * `logging` Custom Driver logging layout (not finished yet). +## Overview -Dependencies of core modules might be found in [Dependencies of the Modules diagram](https://github.com/drivergroup/driver-template/blob/master/Modules%20dependencies.pdf) file in driver-template repository in "Core component dependencies" section. +### Components -## Examples +This project is split into multiple submodules, each providing +specific functionality. -### Functions `make` and `using` -Those functions are especially useful to make procedural legacy Java APIs more functional and make scope of its usage more explicit. Runnable examples of its usage might be found in [`CoreTest`](https://github.com/drivergroup/driver-core/blob/master/src/test/scala/xyz/driver/core/CoreTest.scala), e.g., +Project | Description +-----------------|------------ +`core` | *(deprecated)* Previous application initialization framework. +`core-database` | Utilities for working with databases, slick in particular. +`core-init` | Mini-framework that offers default instances of many utilities, configured for the current platform. +`core-messaging` | Library for abstracting over message buses such as Google PubSub. +`core-reporting` | Combined tracing and logging library. +`core-rest` | Abstractions to represent RESTful services, discovery and client implementations. +`core-storage` | Object storage utilities. +`core-types` | Type definitions that are commonly used by applications. +`core-util` | Other utilities that do not belong anywhere else. **Note that this is a staging place for code that does not have its own submodule yet. Do not depend on it externally!** - useObject(make(new ObjectWithProceduralInitialization) { o => - o.setSetting1(...) // returns Unit - o.setSetting2(...) // returns Unit - o.setSetting3(...) // returns Unit - }) +These components and their internal dependencies can be represented +with the following graph. - // instead of - val o = new ObjectWithProceduralInitialization - o.setSetting1(...) - o.setSetting2(...) - o.setSetting3(...) +![Inter-Component Dependencies](documentation/components.svg) - useObject(o) +### Dependency -and - - using(... open output stream ...) { os => - os.write(...) - } - - // it will be close automatically - -### `OptionT` utils -Before - -``` -OptionT.optionT[Future](service.getRecords(id).map(Option.apply)) -OptionT.optionT(service.doSomething(id).map(_ => Option(()))) - -// Do not want to stop and return `None`, if `produceEffect` returns `None` -for { - x <- service.doSomething(id) - _ <- service.produceEffect(id, x).map(_ => ()).orElse(OptionT.some[Future, Unit](()))) -} yield x -``` - -after - -``` -service.getRecords(id).toOptionT -service.doSomething(id).toUnitOptionT - -// Do not want to stop and return `None` if `produceEffect` returns `None` -for { - x <- service.doSomething(id) - _ <- service.produceEffect(id, x).continueIgnoringNone -} yield x -``` - -### `@@` or Tagged types - -For type definitions, the only import required is - -```scala -import xyz.driver.core.@@ -``` - -which provides just the ability to tag types: `val value: String @@ Important`. Two `String`s with different tags will -be distinguished by the compiler, helping reduce the possibility of mixing values passed into methods with several -arguments of identical types. - -To work with tags in actual values, use the following convenience methods: - -```scala -import xyz.driver.core.tagging._ - -val str = "abc".tagged[Important] -``` - -or go back to plain (say, in case you have an implicit for untagged value) - -```scala -// val trimmedExternalId: String @@ Trimmed = "123" - -Trials.filter(_.externalId === trimmedExternalId.untag) +All components are published together under the same version ([latest +version](https://github.com/drivergroup/driver-core/releases)). +```sbt +libraryDependencies += "xyz.driver" %% "core-" % "" ``` -### `Time` and `TimeProvider` - -**NOTE: The contents of this section has been deprecated - use java.time.Clock instead** - -Usage examples for `Time` (also check [TimeTest](https://github.com/drivergroup/driver-core/blob/master/src/test/scala/xyz/driver/core/TimeTest.scala) for more examples). - - Time(234L).isAfter(Time(123L)) - - Time(123L).isBefore(Time(234L)) - - Time(123L).advanceBy(4 days) - - Seq(Time(321L), Time(123L), Time(231L)).sorted - - startOfMonth(Time(1468937089834L)) should be (Time(1467381889834L)) - - textualDate(Time(1468937089834L)) should be ("July 19, 2016") - - textualTime(Time(1468937089834L)) should be ("Jul 19, 2016 10:04:49 AM") - - TimeRange(Time(321L), Time(432L)).duration should be(111.milliseconds) - - -### Generators -Example of how to generate a case class instance, - - import com.drivergrp.core._ - - Consumer( - generators.nextId[Consumer], - Name[Consumer](faker.Name.name), - faker.Lorem.sentence(word_count = 10)) - - -For more examples check [project tests](https://github.com/drivergroup/driver-core/tree/master/src/test/scala/xyz/driver/core) or [service template](http://github.com/drivergroup/driver-template) repository. - -### App - -To start a new application using standard Driver application class, follow this pattern: - - object MyApp extends App { - - new DriverApp(BuildInfo.version, - BuildInfo.gitHeadCommit.getOrElse("None"), - modules = Seq(myModule1, myModule2), - time, log, config, - interface = "::0", baseUrl, scheme, port) - (servicesActorSystem, servicesExecutionContext).run() - } - -### REST -With REST utils, for instance, you can use the following directives in [akka-http](https://github.com/akka/akka-http) routes, as follows - - sanitizeRequestEntity { // Prevents XSS - serviceContext { implicit ctx => // Extracts context headers from request - authorize(CanSeeUser(userId)) { user => // Authorizes and extracts user - // Your code using `ctx` and `user` - } - } - } - -### Swagger -Swagger JSON formats built using reflection can be overriden by using `CustomSwaggerJsonConverter` at the start of your application initialization in the following way: - - ModelConverters.getInstance() - .addConverter(new CustomSwaggerJsonConverter(Json.mapper(), - CustomSwaggerFormats.customProperties, CustomSwaggerFormats.customObjectsExamples)) - -### Locale -Locale messages can be initialized and used in the following way, - - val localeConfig = config.getConfig("locale") - val log = com.typesafe.scalalogging.Logger(LoggerFactory.getLogger(classOf[MyClass])) - - val messages = xyz.driver.core.messages.Messages.messages(localeConfig, log, Locale.US) - - messages("my.locale.message") - messages("my.locale.message.with.param", parameter) - - -### System stats -Stats it gives access to are, - - xyz.driver.core.stats.SystemStats.memoryUsage - - xyz.driver.core.stats.SystemStats.availableProcessors - - xyz.driver.core.stats.SystemStats.garbageCollectorStats - - xyz.driver.core.stats.SystemStats.fileSystemSpace +### External dependencies - xyz.driver.core.stats.SystemStats.operatingSystemStats +This project has quite a few external dependencies. See +[build.sbt](build.sbt) for a list of them. Some notable Scala +libraries used are: +- [akka-http](https://doc.akka.io/docs/akka-http/current/) +- [akka-stream](https://doc.akka.io/docs/akka/current/stream/) +- [enumeratum](https://github.com/lloydmeta/enumeratum#enumeratum------) +- [scala-async](https://github.com/scala/scala-async) +- [slick](http://slick.lightbend.com/) +- [spray-json](https://github.com/spray/spray-json) +- [sttp](https://github.com/softwaremill/sttp) -## Running +## Example Usage -1. Compile everything and test: +*TODO* - $ sbt test +## Building -2. Publish to local repository, to use changes in depending projects: +This project uses [sbt](https://www.scala-sbt.org/) as build tool. It +has grown organically over time, but all new code should follow the +best practices outlined +[here](https://style.driver.engineering/scala.html). - $ sbt publishLocal +It is set up to automatically build, test, and publish a release when +a Git tag is pushed that matches the regular expression +`v[0-9].*`. Hence, to publish a new version "1.2.3", simply create a +tag "v1.2.3" and `git push --tags` to origin. -3. In order to release a new version of core, merge your PR, tag the HEAD master commit with the next version - (don't forget the "v." prefix) and push tags - Travis will release the artifact automatically! +## Copying +Copyright 2016-2018 Driver Inc. Released under the Apache License, +Version 2.0. diff --git a/documentation/components.dot b/documentation/components.dot new file mode 100644 index 0000000..f00dff3 --- /dev/null +++ b/documentation/components.dot @@ -0,0 +1,21 @@ +digraph core { + + rankdir = BT; + + "core-reporting"; + "core-storage" -> "core-reporting"; + "core-messaging" -> "core-reporting"; + "core-database" -> "core-reporting"; + "core-database" -> "core-types"; + "core-rest" -> "core-reporting"; + "core-rest" -> "core-types"; + "core-init" -> "core-storage"; + "core-init" -> "core-messaging"; + "core-init" -> "core-database"; + "core-init" -> "core-rest"; + "core-init" -> "core-reporting"; + + core [color=red]; + core -> "core-init" [color=red]; + +} diff --git a/documentation/components.svg b/documentation/components.svg new file mode 100644 index 0000000..0531399 --- /dev/null +++ b/documentation/components.svg @@ -0,0 +1,133 @@ + + + + + + +core + + + +core-reporting + +core-reporting + + + +core-storage + +core-storage + + + +core-storage->core-reporting + + + + + +core-messaging + +core-messaging + + + +core-messaging->core-reporting + + + + + +core-database + +core-database + + + +core-database->core-reporting + + + + + +core-types + +core-types + + + +core-database->core-types + + + + + +core-rest + +core-rest + + + +core-rest->core-reporting + + + + + +core-rest->core-types + + + + + +core-init + +core-init + + + +core-init->core-reporting + + + + + +core-init->core-storage + + + + + +core-init->core-messaging + + + + + +core-init->core-database + + + + + +core-init->core-rest + + + + + +core + +core + + + +core->core-init + + + + + -- cgit v1.2.3