diff options
-rw-r--r-- | README.md | 257 | ||||
-rw-r--r-- | documentation/components.dot | 21 | ||||
-rw-r--r-- | documentation/components.svg | 133 |
3 files changed, 216 insertions, 195 deletions
@@ -7,227 +7,94 @@ cherry-picked onto the [1.x branch](https://github.com/drivergroup/driver-core/tree/1.x). ---- +[![Build Status](https://travis-ci.com/drivergroup/driver-core.svg?token=S4oyfBY3YoEdLmckujJx&branch=master)](https://travis-ci.com/drivergroup/driver-core) -# Driver Core Library for Scala Services [![Build Status](https://travis-ci.com/drivergroup/driver-core.svg?token=S4oyfBY3YoEdLmckujJx&branch=master)](https://travis-ci.com/drivergroup/driver-core) +# Driver Core Library for Scala Services Multi-cloud utilities and application initialization framework. This library offers many common utilities for building applications -that run on multiple environments, including Google Cloud, Ali Cloud, +that run in multiple environments, including Google Cloud, Ali Cloud, and of course on development machines. +## Highlights -# Overview +- Cloud agnostic. -*This section applies to the 1.x series of core. The current development master branch may include changes not described here.* +- Sensible defaults. *Applications that use the initialization + framework can run out-of-the box on cloud or locally, with minimal + config required.* -Core library is used to provide ways to implement practices established in [Driver service template](http://github.com/drivergroup/driver-template) (check its [README.md](https://github.com/drivergroup/driver-template/blob/master/README.md)). +- Distributed tracing built into all major utilities. *Significant + actions are logged and reported automatically to an OpenTrace + aggregator.* -## Components +- Extensible. *Utilities are built to be used standalone. A + trait-based initialization framework provides utility instances for + common use-cases.* - * `core package` provides `Id` and `Name` implementations (with equal and ordering), utils for ScalaZ `OptionT`, and also `make` and `using` functions and `@@` (tagged) type, - * `tagging` Utilities for tagging primitive types for extra type safety, as well as some tags that involve extra transformation upon deserializing with spray, - * `config` Contains method `loadDefaultConfig` with default way of providing config to the application, - * `domain` Common generic domain objects, e.g., `Email` and `PhoneNumber`, - * `messages` Localization messages supporting different locales and methods to read from config, - * `database` Method for database initialization from config, `Id`, `Name`, `Time`, `Date` etc. mapping, schema lifecycle and base classes to implement and test `Repository` (data access objects), - * `rest` Wrapper over call to external REST API, authorization, context headers, XSS protection, does logging and allows to add Swagger to a service, - * `auth` Basic entities for authentication and authorization: `User`, `Role` `Permission` `AuthToken`, `AuthCredentials` etc., - * `swagger` Contains custom `AbstractModelConverter` to customize Swagger JSON with any Scala JSON formats created by, for instance, Spray JSON, - * `json` Json formats for `Id`, `Name`, `Time`, `Revision`, `Email`, `PhoneNumber`, `AuthCredentials` and converters for GADTs, enums and value classes, - * `file` Interface `FileStorage` and file storage implementations with GCS, S3 and local FS, - * `app` Base class for Driver service, which initializes swagger, app modules and its routes. - * `generators` Set of functions to prototype APIs. Combines with `faker` package, - * `stats` Methods to collect system stats: memory, cpu, gc, file system space usage, - * `logging` Custom Driver logging layout (not finished yet). +## Overview -Dependencies of core modules might be found in [Dependencies of the Modules diagram](https://github.com/drivergroup/driver-template/blob/master/Modules%20dependencies.pdf) file in driver-template repository in "Core component dependencies" section. +### Components -## Examples +This project is split into multiple submodules, each providing +specific functionality. -### Functions `make` and `using` -Those functions are especially useful to make procedural legacy Java APIs more functional and make scope of its usage more explicit. Runnable examples of its usage might be found in [`CoreTest`](https://github.com/drivergroup/driver-core/blob/master/src/test/scala/xyz/driver/core/CoreTest.scala), e.g., +Project | Description +-----------------|------------ +`core` | *(deprecated)* Previous application initialization framework. +`core-database` | Utilities for working with databases, slick in particular. +`core-init` | Mini-framework that offers default instances of many utilities, configured for the current platform. +`core-messaging` | Library for abstracting over message buses such as Google PubSub. +`core-reporting` | Combined tracing and logging library. +`core-rest` | Abstractions to represent RESTful services, discovery and client implementations. +`core-storage` | Object storage utilities. +`core-types` | Type definitions that are commonly used by applications. +`core-util` | Other utilities that do not belong anywhere else. **Note that this is a staging place for code that does not have its own submodule yet. Do not depend on it externally!** - useObject(make(new ObjectWithProceduralInitialization) { o => - o.setSetting1(...) // returns Unit - o.setSetting2(...) // returns Unit - o.setSetting3(...) // returns Unit - }) +These components and their internal dependencies can be represented +with the following graph. - // instead of - val o = new ObjectWithProceduralInitialization - o.setSetting1(...) - o.setSetting2(...) - o.setSetting3(...) +![Inter-Component Dependencies](documentation/components.svg) - useObject(o) +### Dependency -and - - using(... open output stream ...) { os => - os.write(...) - } - - // it will be close automatically - -### `OptionT` utils -Before - -``` -OptionT.optionT[Future](service.getRecords(id).map(Option.apply)) -OptionT.optionT(service.doSomething(id).map(_ => Option(()))) - -// Do not want to stop and return `None`, if `produceEffect` returns `None` -for { - x <- service.doSomething(id) - _ <- service.produceEffect(id, x).map(_ => ()).orElse(OptionT.some[Future, Unit](()))) -} yield x -``` - -after - -``` -service.getRecords(id).toOptionT -service.doSomething(id).toUnitOptionT - -// Do not want to stop and return `None` if `produceEffect` returns `None` -for { - x <- service.doSomething(id) - _ <- service.produceEffect(id, x).continueIgnoringNone -} yield x -``` - -### `@@` or Tagged types - -For type definitions, the only import required is - -```scala -import xyz.driver.core.@@ -``` - -which provides just the ability to tag types: `val value: String @@ Important`. Two `String`s with different tags will -be distinguished by the compiler, helping reduce the possibility of mixing values passed into methods with several -arguments of identical types. - -To work with tags in actual values, use the following convenience methods: - -```scala -import xyz.driver.core.tagging._ - -val str = "abc".tagged[Important] -``` - -or go back to plain (say, in case you have an implicit for untagged value) - -```scala -// val trimmedExternalId: String @@ Trimmed = "123" - -Trials.filter(_.externalId === trimmedExternalId.untag) +All components are published together under the same version ([latest +version](https://github.com/drivergroup/driver-core/releases)). +```sbt +libraryDependencies += "xyz.driver" %% "core-<component>" % "<version>" ``` -### `Time` and `TimeProvider` - -**NOTE: The contents of this section has been deprecated - use java.time.Clock instead** - -Usage examples for `Time` (also check [TimeTest](https://github.com/drivergroup/driver-core/blob/master/src/test/scala/xyz/driver/core/TimeTest.scala) for more examples). - - Time(234L).isAfter(Time(123L)) - - Time(123L).isBefore(Time(234L)) - - Time(123L).advanceBy(4 days) - - Seq(Time(321L), Time(123L), Time(231L)).sorted - - startOfMonth(Time(1468937089834L)) should be (Time(1467381889834L)) - - textualDate(Time(1468937089834L)) should be ("July 19, 2016") - - textualTime(Time(1468937089834L)) should be ("Jul 19, 2016 10:04:49 AM") - - TimeRange(Time(321L), Time(432L)).duration should be(111.milliseconds) - - -### Generators -Example of how to generate a case class instance, - - import com.drivergrp.core._ - - Consumer( - generators.nextId[Consumer], - Name[Consumer](faker.Name.name), - faker.Lorem.sentence(word_count = 10)) - - -For more examples check [project tests](https://github.com/drivergroup/driver-core/tree/master/src/test/scala/xyz/driver/core) or [service template](http://github.com/drivergroup/driver-template) repository. - -### App - -To start a new application using standard Driver application class, follow this pattern: - - object MyApp extends App { - - new DriverApp(BuildInfo.version, - BuildInfo.gitHeadCommit.getOrElse("None"), - modules = Seq(myModule1, myModule2), - time, log, config, - interface = "::0", baseUrl, scheme, port) - (servicesActorSystem, servicesExecutionContext).run() - } - -### REST -With REST utils, for instance, you can use the following directives in [akka-http](https://github.com/akka/akka-http) routes, as follows - - sanitizeRequestEntity { // Prevents XSS - serviceContext { implicit ctx => // Extracts context headers from request - authorize(CanSeeUser(userId)) { user => // Authorizes and extracts user - // Your code using `ctx` and `user` - } - } - } - -### Swagger -Swagger JSON formats built using reflection can be overriden by using `CustomSwaggerJsonConverter` at the start of your application initialization in the following way: - - ModelConverters.getInstance() - .addConverter(new CustomSwaggerJsonConverter(Json.mapper(), - CustomSwaggerFormats.customProperties, CustomSwaggerFormats.customObjectsExamples)) - -### Locale -Locale messages can be initialized and used in the following way, - - val localeConfig = config.getConfig("locale") - val log = com.typesafe.scalalogging.Logger(LoggerFactory.getLogger(classOf[MyClass])) - - val messages = xyz.driver.core.messages.Messages.messages(localeConfig, log, Locale.US) - - messages("my.locale.message") - messages("my.locale.message.with.param", parameter) - - -### System stats -Stats it gives access to are, - - xyz.driver.core.stats.SystemStats.memoryUsage - - xyz.driver.core.stats.SystemStats.availableProcessors - - xyz.driver.core.stats.SystemStats.garbageCollectorStats - - xyz.driver.core.stats.SystemStats.fileSystemSpace +### External dependencies - xyz.driver.core.stats.SystemStats.operatingSystemStats +This project has quite a few external dependencies. See +[build.sbt](build.sbt) for a list of them. Some notable Scala +libraries used are: +- [akka-http](https://doc.akka.io/docs/akka-http/current/) +- [akka-stream](https://doc.akka.io/docs/akka/current/stream/) +- [enumeratum](https://github.com/lloydmeta/enumeratum#enumeratum------) +- [scala-async](https://github.com/scala/scala-async) +- [slick](http://slick.lightbend.com/) +- [spray-json](https://github.com/spray/spray-json) +- [sttp](https://github.com/softwaremill/sttp) -## Running +## Example Usage -1. Compile everything and test: +*TODO* - $ sbt test +## Building -2. Publish to local repository, to use changes in depending projects: +This project uses [sbt](https://www.scala-sbt.org/) as build tool. It +has grown organically over time, but all new code should follow the +best practices outlined +[here](https://style.driver.engineering/scala.html). - $ sbt publishLocal +It is set up to automatically build, test, and publish a release when +a Git tag is pushed that matches the regular expression +`v[0-9].*`. Hence, to publish a new version "1.2.3", simply create a +tag "v1.2.3" and `git push --tags` to origin. -3. In order to release a new version of core, merge your PR, tag the HEAD master commit with the next version - (don't forget the "v." prefix) and push tags - Travis will release the artifact automatically! +## Copying +Copyright 2016-2018 Driver Inc. Released under the Apache License, +Version 2.0. diff --git a/documentation/components.dot b/documentation/components.dot new file mode 100644 index 0000000..f00dff3 --- /dev/null +++ b/documentation/components.dot @@ -0,0 +1,21 @@ +digraph core { + + rankdir = BT; + + "core-reporting"; + "core-storage" -> "core-reporting"; + "core-messaging" -> "core-reporting"; + "core-database" -> "core-reporting"; + "core-database" -> "core-types"; + "core-rest" -> "core-reporting"; + "core-rest" -> "core-types"; + "core-init" -> "core-storage"; + "core-init" -> "core-messaging"; + "core-init" -> "core-database"; + "core-init" -> "core-rest"; + "core-init" -> "core-reporting"; + + core [color=red]; + core -> "core-init" [color=red]; + +} diff --git a/documentation/components.svg b/documentation/components.svg new file mode 100644 index 0000000..0531399 --- /dev/null +++ b/documentation/components.svg @@ -0,0 +1,133 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" + "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by graphviz version 2.40.1 (20161225.0304) + --> +<!-- Title: core Pages: 1 --> +<svg width="542pt" height="260pt" + viewBox="0.00 0.00 542.09 260.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> +<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 256)"> +<title>core</title> +<polygon fill="#ffffff" stroke="transparent" points="-4,4 -4,-256 538.09,-256 538.09,4 -4,4"/> +<!-- core-reporting --> +<g id="node1" class="node"> +<title>core-reporting</title> +<ellipse fill="none" stroke="#000000" cx="285.9452" cy="-234" rx="61.1893" ry="18"/> +<text text-anchor="middle" x="285.9452" y="-230.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-reporting</text> +</g> +<!-- core-storage --> +<g id="node2" class="node"> +<title>core-storage</title> +<ellipse fill="none" stroke="#000000" cx="53.9452" cy="-162" rx="53.8905" ry="18"/> +<text text-anchor="middle" x="53.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-storage</text> +</g> +<!-- core-storage->core-reporting --> +<g id="edge1" class="edge"> +<title>core-storage->core-reporting</title> +<path fill="none" stroke="#000000" d="M93.5638,-174.2954C132.345,-186.331 191.6297,-204.7297 234.1337,-217.9206"/> +<polygon fill="#000000" stroke="#000000" points="233.155,-221.2815 243.7431,-220.9028 235.2299,-214.596 233.155,-221.2815"/> +</g> +<!-- core-messaging --> +<g id="node3" class="node"> +<title>core-messaging</title> +<ellipse fill="none" stroke="#000000" cx="191.9452" cy="-162" rx="66.0889" ry="18"/> +<text text-anchor="middle" x="191.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-messaging</text> +</g> +<!-- core-messaging->core-reporting --> +<g id="edge2" class="edge"> +<title>core-messaging->core-reporting</title> +<path fill="none" stroke="#000000" d="M214.2219,-179.063C226.6308,-188.5677 242.254,-200.5344 255.7488,-210.8708"/> +<polygon fill="#000000" stroke="#000000" points="253.7613,-213.7573 263.8284,-217.0595 258.0179,-208.2001 253.7613,-213.7573"/> +</g> +<!-- core-database --> +<g id="node4" class="node"> +<title>core-database</title> +<ellipse fill="none" stroke="#000000" cx="474.9452" cy="-162" rx="59.2899" ry="18"/> +<text text-anchor="middle" x="474.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-database</text> +</g> +<!-- core-database->core-reporting --> +<g id="edge3" class="edge"> +<title>core-database->core-reporting</title> +<path fill="none" stroke="#000000" d="M437.6049,-176.2249C407.6744,-187.627 365.4902,-203.6971 333.2074,-215.9954"/> +<polygon fill="#000000" stroke="#000000" points="331.6298,-212.8509 323.5309,-219.6816 334.1218,-219.3923 331.6298,-212.8509"/> +</g> +<!-- core-types --> +<g id="node5" class="node"> +<title>core-types</title> +<ellipse fill="none" stroke="#000000" cx="443.9452" cy="-234" rx="47.3916" ry="18"/> +<text text-anchor="middle" x="443.9452" y="-230.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-types</text> +</g> +<!-- core-database->core-types --> +<g id="edge4" class="edge"> +<title>core-database->core-types</title> +<path fill="none" stroke="#000000" d="M467.1226,-180.1686C463.6452,-188.2453 459.478,-197.9239 455.642,-206.8332"/> +<polygon fill="#000000" stroke="#000000" points="452.3979,-205.5176 451.6579,-216.0866 458.8273,-208.2859 452.3979,-205.5176"/> +</g> +<!-- core-rest --> +<g id="node6" class="node"> +<title>core-rest</title> +<ellipse fill="none" stroke="#000000" cx="355.9452" cy="-162" rx="41.6928" ry="18"/> +<text text-anchor="middle" x="355.9452" y="-158.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-rest</text> +</g> +<!-- core-rest->core-reporting --> +<g id="edge5" class="edge"> +<title>core-rest->core-reporting</title> +<path fill="none" stroke="#000000" d="M339.7099,-178.6992C330.9557,-187.7035 319.978,-198.9948 310.2283,-209.0231"/> +<polygon fill="#000000" stroke="#000000" points="307.5305,-206.7771 303.0691,-216.3868 312.5494,-211.6566 307.5305,-206.7771"/> +</g> +<!-- core-rest->core-types --> +<g id="edge6" class="edge"> +<title>core-rest->core-types</title> +<path fill="none" stroke="#000000" d="M375.4753,-177.9791C387.3842,-187.7228 402.7997,-200.3355 415.9875,-211.1255"/> +<polygon fill="#000000" stroke="#000000" points="413.9037,-213.9428 423.8597,-217.5664 418.3364,-208.5251 413.9037,-213.9428"/> +</g> +<!-- core-init --> +<g id="node7" class="node"> +<title>core-init</title> +<ellipse fill="none" stroke="#000000" cx="285.9452" cy="-90" rx="40.0939" ry="18"/> +<text text-anchor="middle" x="285.9452" y="-86.3" font-family="Times,serif" font-size="14.00" fill="#000000">core-init</text> +</g> +<!-- core-init->core-reporting --> +<g id="edge11" class="edge"> +<title>core-init->core-reporting</title> +<path fill="none" stroke="#000000" d="M285.9452,-108.2377C285.9452,-132.799 285.9452,-176.7526 285.9452,-205.6459"/> +<polygon fill="#000000" stroke="#000000" points="282.4453,-205.9103 285.9452,-215.9104 289.4453,-205.9104 282.4453,-205.9103"/> +</g> +<!-- core-init->core-storage --> +<g id="edge7" class="edge"> +<title>core-init->core-storage</title> +<path fill="none" stroke="#000000" d="M252.7053,-100.3158C213.6013,-112.4515 148.3163,-132.7124 103.2985,-146.6834"/> +<polygon fill="#000000" stroke="#000000" points="102.2479,-143.3448 93.7347,-149.6515 104.3228,-150.0302 102.2479,-143.3448"/> +</g> +<!-- core-init->core-messaging --> +<g id="edge8" class="edge"> +<title>core-init->core-messaging</title> +<path fill="none" stroke="#000000" d="M265.5486,-105.6229C252.9101,-115.3035 236.4447,-127.9153 222.2928,-138.7551"/> +<polygon fill="#000000" stroke="#000000" points="220.0846,-136.0377 214.274,-144.8971 224.3411,-141.5948 220.0846,-136.0377"/> +</g> +<!-- core-init->core-database --> +<g id="edge9" class="edge"> +<title>core-init->core-database</title> +<path fill="none" stroke="#000000" d="M316.732,-101.7283C347.0407,-113.2745 393.7166,-131.0558 428.5269,-144.3169"/> +<polygon fill="#000000" stroke="#000000" points="427.4162,-147.639 438.0071,-147.9284 429.9083,-141.0976 427.4162,-147.639"/> +</g> +<!-- core-init->core-rest --> +<g id="edge10" class="edge"> +<title>core-init->core-rest</title> +<path fill="none" stroke="#000000" d="M302.1806,-106.6992C311.2805,-116.0592 322.7832,-127.8905 332.8116,-138.2055"/> +<polygon fill="#000000" stroke="#000000" points="330.3176,-140.6611 339.7979,-145.3913 335.3366,-135.7816 330.3176,-140.6611"/> +</g> +<!-- core --> +<g id="node8" class="node"> +<title>core</title> +<ellipse fill="none" stroke="#ff0000" cx="285.9452" cy="-18" rx="27" ry="18"/> +<text text-anchor="middle" x="285.9452" y="-14.3" font-family="Times,serif" font-size="14.00" fill="#000000">core</text> +</g> +<!-- core->core-init --> +<g id="edge12" class="edge"> +<title>core->core-init</title> +<path fill="none" stroke="#ff0000" d="M285.9452,-36.1686C285.9452,-43.869 285.9452,-53.0257 285.9452,-61.5834"/> +<polygon fill="#ff0000" stroke="#ff0000" points="282.4453,-61.5867 285.9452,-71.5867 289.4453,-61.5868 282.4453,-61.5867"/> +</g> +</g> +</svg> |