diff options
38 files changed, 927 insertions, 736 deletions
@@ -5,7 +5,9 @@ [](https://maven-badges.herokuapp.com/maven-central/com.softwaremill.sttp/core_2.12) [](https://app.updateimpact.com/latest/634276070333485056/sttp) -The HTTP client for Scala that you always wanted! +The Scala HTTP client that you always wanted! + +sttp is an open-source library which provides a clean, programmer-friendly API to define HTTP requests and execute them using one of the wrapped backends, such as akka-http, async-http-client or OkHttp. ```scala import com.softwaremill.sttp._ @@ -26,588 +28,42 @@ println(response.header("Content-Length")) // response.unsafeBody: by default read into a String println(response.unsafeBody) ``` - -## Goals of the project - -* provide a simple, discoverable, no-surprises, reasonably type-safe API for -making HTTP requests and reading responses -* separate definition of a request from request execution -* provide immutable, easily modifiable data structures for requests and -responses -* support multiple execution backends, both synchronous and asynchronous -* provide support for backend-specific request/response streaming -* minimum dependencies - -See also the [introduction to sttp](https://softwaremill.com/introducing-sttp-the-scala-http-client) -and [sttp streaming & URI interpolators](https://softwaremill.com/sttp-streaming-uri-interpolator) -blogs. - -## Non-goals of the project - -* implement a full HTTP client. Instead, sttp wraps existing HTTP clients, -providing a consistent, programmer-friendly API. All network-related concerns -such as sending the requests, connection pooling, receiving responses are -delegated to the chosen backend -* provide ultimate flexibility in defining the request. While it's possible -to define *most* valid HTTP requests, e.g. some of the less common body -chunking approaches aren't available -## How is sttp different from other libraries? +## Documentation -* immutable request builder which doesn't impose any order in which request -parameters need to be specified. Such an approach allows defining partial -requests with common cookies/headers/options, which can later be specialized -using a specific URI and HTTP method. -* support for multiple backends, both synchronous and asynchronous, with -backend-specific streaming support -* URI interpolator with context-aware escaping, optional parameters support -and parameter collections +sttp documentation is available at [sttp.readthedocs.io](http://sttp.readthedocs.io). ## Quickstart with Ammonite -If you are an [Ammonite](http://ammonite.io) user, you can quickly start -experimenting with sttp by copy-pasting the following: +If you are an [Ammonite](http://ammonite.io) user, you can quickly start experimenting with sttp by copy-pasting the following: ```scala -import $ivy.`com.softwaremill.sttp::core:0.0.20` +import $ivy.`com.softwaremill.sttp::core:1.0.0` import com.softwaremill.sttp._ implicit val backend = HttpURLConnectionBackend() sttp.get(uri"http://httpbin.org/ip").send() ``` -## Adding sttp to your project +## Quickstart with sbt -SBT dependency: +Add the following dependency: ```scala -"com.softwaremill.sttp" %% "core" % "0.0.20" +"com.softwaremill.sttp" %% "core" % "1.0.0" ``` -`sttp` is available for Scala 2.11 and 2.12, and requires Java 7 if using an `OkHttp` based backend, or Java 8 otherwise. The core -module has no transitive dependencies. - -If you'd like to use an alternate backend, [see below](#supported-backends) -for additional instructions. - -## API - -First, import: +Then, import: ```scala import com.softwaremill.sttp._ ``` -This brings into scope `sttp`, the starting request (it's an empty request -with the `Accept-Encoding: gzip, defalte` header added). This request can -be customised, each time yielding a new, immutable request description -(unless a mutable body is set on the request, such as a byte array). - -For example, we can set a cookie, string-body and specify that this should -be a `POST` request to a given URI: - -```scala -val request = sttp - .cookie("login", "me") - .body("This is a test") - .post(uri"http://endpoint.com/secret") -``` - -The request parameters (headers, cookies, body etc.) can be specified in any -order. There's a lot of ways in which you can customize a request: just -explore the API. And [more will be added](#todo)! - -You can create a request description without knowing how it will be sent. -But to send a request, you will need a backend. A default, synchronous backend -based on Java's `HttpURLConnection` is provided out-of-the box. An implicit -value of type `SttpBackend` needs to be in scope to invoke the `send()` on the -request: - -```scala -implicit val backend = HttpURLConnectionBackend() - -val response: Response[String] = request.send() -``` - -By default the response body is read into a utf-8 string. How the response body -is handled is also part of the request description. The body can be ignore -(`.response(ignore)`), read into a sequence of parameters -(`.response(asParams)`), mapped (`.mapResponse`) and more; some backends also -support request & response streaming. - -The default backend doesn't wrap the response into any container, but other -asynchronous backends might do so. The type parameter in the `Response[_]` -type specifies the type of the body. - -## URI interpolator - -Using the URI interpolator it's possible to conveniently create `Uri` -instances, which can then be used to specify request endpoints, for example: - -```scala -import com.softwaremill.sttp._ - -val user = "Mary Smith" -val filter = "programming languages" - -val endpoint: Uri = uri"http://example.com/$user/skills?filter=$filter" -``` - -Any values embedded in the URI will be URL-encoded, taking into account the -context (e.g., the whitespace in `user` will be %-encoded as `%20D`, while the -whitespace in `filter` will be query-encoded as `+`). - -The possibilities of the interpolator don't end here. Other supported features: - -* parameters can have optional values: if the value of a parameter is `None`, -it will be removed -* maps, sequences of tuples and sequences of values can be embedded in the query -part. They will be expanded into query parameters. Maps and sequences of tuples -can also contain optional values, for which mappings will be removed -if `None`. -* optional values in the host part will be expanded to a subdomain if `Some`, -removed if `None` -* sequences in the host part will be expanded to a subdomain sequence -* if a string containing the protocol is embedded *as the very beginning*, it will -not be escaped, allowing to embed entire addresses as prefixes, e.g.: -`uri"$endpoint/login"`, where `val endpoint = "http://example.com/api"`. - -A fully-featured example: - -```scala -import com.softwaremill.sttp._ -val secure = true -val scheme = if (secure) "https" else "http" -val subdomains = List("sub1", "sub2") -val vx = Some("y z") -val params = Map("a" -> 1, "b" -> 2) -val jumpTo = Some("section2") -uri"$scheme://$subdomains.example.com?x=$vx&$params#$jumpTo" - -// generates: -// https://sub1.sub2.example.com?x=y+z&a=1&b=2#section2 -``` - -## Starting & cleaning up - -In case of most backends, you should only instantiate a backend once per -application, as a backend typically allocates resources such as thread or -connection pools. - -When ending the application, make sure to call `backend.close()`, which will -free up resources used by the backend (if any). The close process might be -asynchronous, that is it can complete after the `close()` method returns. - -Note that only resources allocated by the backends are freed. For example, -if you use the `AkkaHttpBackend()` the `close()` method will terminate the -underlying actor system. However, if you have provided an existing actor system -upon backend creation (`AkkaHttpBackend.usingActorSystem`), the `close()` -method will be a no-op. - -## Supported backends - -### Summary - -| Class | Result wrapper | Supported stream type | -| --- | --- | --- | -| `HttpURLConnectionBackend` | None (`Id`) | - | -| `AkkaHttpBackend` | `scala.concurrent.Future` | `akka.stream.scaladsl.Source[ByteString, Any]` | -| `AsyncHttpClientFutureBackend` | `scala.concurrent.Future` | - | -| `AsyncHttpClientScalazBackend` | `scalaz.concurrent.Task` | - | -| `AsyncHttpClientMonixBackend` | `monix.eval.Task` | `monix.reactive.Observable[ByteBuffer]` | -| `AsyncHttpClientCatsBackend` | `F[_]: cats.effect.Async` | - | -| `AsyncHttpClientFs2Backend` | `F[_]: cats.effect.Async` | `fs2.Stream[F, ByteBuffer]` | -| `OkHttpSyncBackend` | None (`Id`) | - | -| `OkHttpFutureBackend` | `scala.concurrent.Future` | - | -| `OkHttpMonixBackend` | `monix.eval.Task` | `monix.reactive.Observable[ByteBuffer]` | - -### `HttpURLConnectionBackend` - -The default **synchronous** backend. Sending a request returns a response wrapped -in the identity type constructor, which is equivalent to no wrapper at all. - -To use, add an implicit value: - -```scala -implicit val sttpBackend = HttpURLConnectionBackend() -``` - -### `AkkaHttpBackend` - -To use, add the following dependency to your project: - -```scala -"com.softwaremill.sttp" %% "akka-http-backend" % "0.0.20" -``` - -This backend depends on [akka-http](http://doc.akka.io/docs/akka-http/current/scala/http/). -A fully **asynchronous** backend. Sending a request returns a response wrapped -in a `Future`. - -Next you'll need to add an implicit value: - -```scala -implicit val sttpBackend = AkkaHttpBackend() - -// or, if you'd like to use an existing actor system: -implicit val sttpBackend = AkkaHttpBackend.usingActorSystem(actorSystem) -``` - -This backend supports sending and receiving -[akka-streams](http://doc.akka.io/docs/akka/current/scala/stream/index.html) -streams of type `akka.stream.scaladsl.Source[ByteString, Any]`. - -To set the request body as a stream: - -```scala -import com.softwaremill.sttp._ -import com.softwaremill.sttp.akkahttp._ - -import akka.stream.scaladsl.Source -import akka.util.ByteString - -val source: Source[ByteString, Any] = ... - -sttp - .streamBody(source) - .post(uri"...") -``` - -To receive the response body as a stream: - -```scala -import com.softwaremill.sttp._ -import com.softwaremill.sttp.akkahttp._ - -import akka.stream.scaladsl.Source -import akka.util.ByteString - -implicit val sttpBackend = AkkaHttpBackend() - -val response: Future[Response[Source[ByteString, Any]]] = - sttp - .post(uri"...") - .response(asStream[Source[ByteString, Any]]) - .send() -``` - -### `AsyncHttpClientBackend` - -To use, add the following dependency to your project: - -```scala -"com.softwaremill.sttp" %% "async-http-client-backend-future" % "0.0.20" -// or -"com.softwaremill.sttp" %% "async-http-client-backend-scalaz" % "0.0.20" -// or -"com.softwaremill.sttp" %% "async-http-client-backend-monix" % "0.0.20" -// or -"com.softwaremill.sttp" %% "async-http-client-backend-cats" % "0.0.20" -``` - -This backend depends on [async-http-client](https://github.com/AsyncHttpClient/async-http-client). -A fully **asynchronous** backend, which uses [Netty](http://netty.io) behind the -scenes. - -The responses are wrapped depending on the dependency chosen in either a: - -* standard Scala `Future` -* [Scalaz](https://github.com/scalaz/scalaz) `Task`. There's a transitive -dependency on `scalaz-concurrent`. -* [Monix](https://monix.io) `Task`. There's a transitive dependency on -`monix-eval`. -* Any type implementing the [Cats Effect](https://github.com/typelevel/cats-effect) `Async` -typeclass, such as `cats.effect.IO`. There's a transitive dependency on `cats-effect`. - -Next you'll need to add an implicit value: - -```scala -implicit val sttpBackend = AsyncHttpClientFutureBackend() - -// or, if you're using the scalaz version: -implicit val sttpBackend = AsyncHttpClientScalazBackend() - -// or, if you're using the monix version: -implicit val sttpBackend = AsyncHttpClientMonixBackend() - -// or, if you're using the cats effect version: -implicit val sttpBackend = AsyncHttpClientCatsBackend[cats.effect.IO]() - -// or, if you'd like to use custom configuration: -implicit val sttpBackend = AsyncHttpClientFutureBackend.usingConfig(asyncHttpClientConfig) - -// or, if you'd like to instantiate the AsyncHttpClient yourself: -implicit val sttpBackend = AsyncHttpClientFutureBackend.usingClient(asyncHttpClient) -``` - -#### Streaming using Monix - -The Monix backend supports streaming (as both Monix and Async Http Client -support reactive streams `Publisher`s out of the box). The type of -supported streams in this case is `Observable[ByteBuffer]`. That is, you can -set such an observable as a request body: - -```scala -import com.softwaremill.sttp._ - -import java.nio.ByteBuffer -import monix.reactive.Observable - -val obs: Observable[ByteBuffer] = ... - -sttp - .streamBody(obs) - .post(uri"...") -``` - -And receive responses as an observable stream: - -```scala -import com.softwaremill.sttp._ -import com.softwaremill.sttp.asynchttpclient.monix._ - -import java.nio.ByteBuffer -import monix.eval.Task -import monix.reactive.Observable - -implicit val sttpBackend = AsyncHttpClientMonixBackend() - -val response: Task[Response[Observable[ByteBuffer]]] = - sttp - .post(uri"...") - .response(asStream[Observable[ByteBuffer]]) - .send() -``` - -It's also possible to use [fs2](https://github.com/functional-streams-for-scala/fs2)s -streams for sending request & receiving responses. - -### `OkHttpClientBackend` - -To use, add the following dependency to your project: - -```scala -"com.softwaremill.sttp" %% "okhttp-backend" % "0.0.20" -// or, for the monix version: -"com.softwaremill.sttp" %% "okhttp-backend-monix" % "0.0.20" -``` - -This backend depends on [OkHttp](http://square.github.io/okhttp/), and offers: - -* a **synchronous** backend: `OkHttpSyncBackend` -* an **asynchronous**, `Future`-based backend: `OkHttpFutureBackend` -* an **asynchronous**, Monix-`Task`-based backend: `OkHttpMonixBackend` - -OkHttp fully supports HTTP/2. - -### Custom backends, logging, metrics - -It is also entirely possible to write your own backend (if so, please consider -contributing!) or wrapping an existing one. You can even write completely -generic wrappers for any delegate backend, as each backend comes equipped -with a monad for the response type. This brings the possibility to `map` and -`flatMap` over responses. - -Possible use-cases for wrapper-backend include: - -* logging -* capturing metrics -* request signing (transforming the request before sending it to the delegate) - -To pass some context to wrapper-backends, requests can be *tagged*. Each -`RequestT` instance contains a `tags: Map[String, Any]` field. This is unused -by http, but can be used e.g. to pass a metric name or logging context. - -## JSON - -JSON encoding of bodies and decoding of responses can be handled using -[Circe](https://circe.github.io/circe/) by the `circe` module. To use -add the following dependency to your project: - -```scala -"com.softwaremill.sttp" %% "circe" % "0.0.20" -``` - -This module adds a method to the request and a function that can be given to -a request to decode the response to a specific object. - -```scala -import com.softwaremill.sttp._ -import com.softwaremill.sttp.circe._ - -implicit val backend = HttpURLConnectionBackend() - -// Assume that there is an implicit circe encoder in scope -// for the request Payload, and a decoder for the Response -val requestPayload: Payload = ??? - -val response: Either[io.circe.Error, Response] = - sttp - .post(uri"...") - .body(requestPayload) - .response(asJson[Response]) - .send() -``` - -## Request type - -All request descriptions have type `RequestT[U, T, S]` (T as in Template). -If this looks a bit complex, don't worry, what the three type parameters stand -for is the only thing you'll hopefully have to remember when using the API! - -Going one-by-one: - -* `U[_]` specifies if the request method and URL are specified. Using the API, -this can be either `type Empty[X] = None`, meaning that the request has neither -a method nor an URI. Or, it can be `type Id[X] = X` (type-level identity), -meaning that the request has both a method and an URI specified. Only requests -with a specified URI & method can be sent. -* `T` specifies the type to which the response will be read. By default, this -is `String`. But it can also be e.g. `Array[Byte]` or `Unit`, if the response -should be ignored. Response body handling can be changed by calling the -`.response` method. With backends which support streaming, this can also be -a supported stream type. -* `S` specifies the stream type that this request uses. Most of the time this -will be `Nothing`, meaning that this request does not send a streaming body -or receive a streaming response. So most of the times you can just ignore -that parameter. But, if you are using a streaming backend and want to -send/receive a stream, the `.streamBody` or `response(asStream[S])` will change -the type parameter. - -There are two type aliases for the request template that are used: - -* `type Request[T, S] = RequestT[Id, T, S]`. A sendable request. -* `type PartialRequest[T, S] = RequestT[Empty, T, S]` - -## Timeouts - -Sttp supports read and connection timeouts: - * Connection timeout - can be set globally (30 seconds by default) - * Read timeout - can be set per request (1 minute by default) - -How to use: -```scala -import com.softwaremill.sttp._ -import scala.concurrent.duration._ - -// all backends provide a constructor that allows users to specify backend options -implicit val backend = HttpURLConnectionBackend( - options = SttpBackendOptions.connectionTimeout(1.minute)) - -sttp - .get(uri"...") - .readTimeout(5.minutes) // or Duration.Inf to turn read timeout off - .send() -``` - -## Proxy - -A proxy can be specified when creating a backend: - -```scala -import com.softwaremill.sttp._ - -implicit val backend = HttpURLConnectionBackend( - options = SttpBackendOptions.httpProxy("some.host", 8080)) - -sttp - .get(uri"...") - .send() // uses the proxy -``` - -## SSL - -SSL handling can be customized (or disabled) when creating a backend and is -backend-specific. - -Depending on the underlying backend's client, you can customize SSL settings -as follows: - -* `HttpUrlConnectionBackend`: when creating the backend, specify the `customizeConnection: HttpURLConnection => Unit` -parameter, and set the hostname verifier & SSL socket factory as required -* akka-http: when creating the backend, specify the `customHttpsContext: Option[HttpsConnectionContext]` -parameter. See [akka-http docs](http://doc.akka.io/docs/akka-http/current/scala/http/server-side/server-https-support.html) -* async-http-client: create a custom client and use the `setSSLContext` method -* OkHttp: create a custom client modifying the SSL settings as described -[on the wiki](https://github.com/square/okhttp/wiki/HTTPS) - -## Testing - -If you need a stub backend for use in tests instead of a "real" backend (you -probably don't want to make HTTP calls during unit tests), you can use the -`SttpBackendStub` class. It allows specifying how the backend should respond -to requests matching given predicates. - -A backend stub can be created using an instance of a "real" backend, or by -explicitly giving the response wrapper monad and supported streams type. - -For example: - -```scala -implicit val testingBackend = SttpBackendStub(HttpURLConnectionBackend()) - .whenRequestMatches(_.uri.path.startsWith(List("a", "b"))) - .thenRespond("Hello there!") - .whenRequestMatches(_.method == Method.POST) - .thenRespondServerError() - -val response1 = sttp.get(uri"http://example.org/a/b/c").send() -// response1.body will be Right("Hello there") - -val response2 = sttp.post(uri"http://example.org/d/e").send() -// response2.code will be 500 -``` - -However, this approach has one caveat: the responses are not type-safe. That -is, the backend cannot match on or verify that the type included in the -response matches the response type requested. - -It is also possible to create a stub backend which delegates calls to another -(possibly "real") backend if none of the specified predicates match a request. -This can be useful during development, to partially stub a yet incomplete -API with which we integrate: - -```scala -implicit val testingBackend = SttpBackendStub.withFallback(HttpURLConnectionBackend()) - .whenRequestMatches(_.uri.path.startsWith(List("a"))) - .thenRespond("I'm a STUB!") - -val response1 = sttp.get(uri"http://api.internal/a").send() -// response1.body will be Right("I'm a STUB") - -val response2 = sttp.post(uri"http://api.internal/b").send() -// response2 will be whatever a "real" network call to api.internal/b returns -``` - -## Notes - -* the encoding for `String`s defaults to `utf-8`. -* unless explicitly specified, the `Content-Type` defaults to: - * `text/plain` for text - * `application/x-www-form-urlencoded` for form data - * `multipart/form-data` for multipart form data - * `application/octet-stream` for everything else (binary) - -## Other Scala HTTP clients - -* [scalaj](https://github.com/scalaj/scalaj-http) -* [akka-http client](http://doc.akka.io/docs/akka-http/current/scala/http/client-side/index.html) -* [dispatch](http://dispatch.databinder.net/Dispatch.html) -* [play ws](https://github.com/playframework/play-ws) -* [fs2-http](https://github.com/Spinoco/fs2-http) -* [http4s](http://http4s.org/v0.17/client/) -* [Gigahorse](http://eed3si9n.com/gigahorse/) -* [RösHTTP](https://github.com/hmil/RosHTTP) +Type `sttp.` and see where your IDE’s auto-complete gets you! ## Contributing -Take a look at the [open issues](https://github.com/softwaremill/sttp/issues) -and pick a task you'd like to work on! +If you have a question, or hit a problem, feel free to ask on our [gitter channel](https://gitter.im/softwaremill/sttp)! -## Credits +Or, if you encounter a bug, something is unclear in the code or documentation, don’t hesitate and open an issue on GitHub. -* [Tomasz Szymański](https://github.com/szimano) -* [Adam Warski](https://github.com/adamw) -* [Omar Alejandro Mainegra Sarduy](https://github.com/omainegra) -* [Bjørn Madsen](https://github.com/aeons) -* [Piotr Buda](https://github.com/pbuda) -* [Piotr Gabara](https://github.com/bhop) -* [Gabriele Petronella](https://github.com/gabro) +We are also always looking for contributions and new ideas, so if you’d like to get into the project, check out the [open issues](https://github.com/softwaremill/sttp/issues), or post your own suggestions! @@ -58,6 +58,7 @@ lazy val rootProject = (project in file(".")) okhttpBackend, okhttpMonixBackend, circe, + json4s, tests ) @@ -151,7 +152,7 @@ lazy val okhttpMonixBackend: Project = (project in file("okhttp-backend/monix")) libraryDependencies ++= Seq(monix) ) dependsOn okhttpBackend -lazy val circe: Project = (project in file("circe")) +lazy val circe: Project = (project in file("json/circe")) .settings(commonSettings: _*) .settings( name := "circe", @@ -162,6 +163,16 @@ lazy val circe: Project = (project in file("circe")) ) ) dependsOn core +lazy val json4s: Project = (project in file("json/json4s")) + .settings(commonSettings: _*) + .settings( + name := "json4s", + libraryDependencies ++= Seq( + "org.json4s" %% "json4s-native" % "3.5.3", + scalaTest % "test" + ) + ) dependsOn core + lazy val tests: Project = (project in file("tests")) .settings(commonSettings: _*) .settings( diff --git a/core/src/main/scala/com/softwaremill/sttp/package.scala b/core/src/main/scala/com/softwaremill/sttp/package.scala index c1178d9..ed685fa 100644 --- a/core/src/main/scala/com/softwaremill/sttp/package.scala +++ b/core/src/main/scala/com/softwaremill/sttp/package.scala @@ -51,6 +51,7 @@ package object sttp { "application/x-www-form-urlencoded" private[sttp] val TextPlainContentType = "text/plain" private[sttp] val MultipartFormDataContentType = "multipart/form-data" + private[sttp] val ApplicationJsonContentType = "application/json" // entry points diff --git a/docs/backends/akkahttp.rst b/docs/backends/akkahttp.rst index 41e3a7c..a30c7b3 100644 --- a/docs/backends/akkahttp.rst +++ b/docs/backends/akkahttp.rst @@ -1,13 +1,13 @@ -``AkkaHttpBackend`` -=================== +.. _akkahttp: + +akka-http backend +================= To use, add the following dependency to your project:: - "com.softwaremill.sttp" %% "akka-http-backend" % "0.0.20" + "com.softwaremill.sttp" %% "akka-http-backend" % "1.0.0" -This backend depends on `akka-http <http://doc.akka.io/docs/akka-http/current/scala/http/>`_. -A fully **asynchronous** backend. Sending a request returns a response wrapped -in a ``Future``. +This backend depends on `akka-http <http://doc.akka.io/docs/akka-http/current/scala/http/>`_. A fully **asynchronous** backend. Sending a request returns a response wrapped in a ``Future``. Next you'll need to add an implicit value:: @@ -16,9 +16,7 @@ Next you'll need to add an implicit value:: // or, if you'd like to use an existing actor system: implicit val sttpBackend = AkkaHttpBackend.usingActorSystem(actorSystem) -This backend supports sending and receiving -`akka-streams <http://doc.akka.io/docs/akka/current/scala/stream/index.html>`_ -streams of type ``akka.stream.scaladsl.Source[ByteString, Any]``. +This backend supports sending and receiving `akka-streams <http://doc.akka.io/docs/akka/current/scala/stream/index.html>`_ streams of type ``akka.stream.scaladsl.Source[ByteString, Any]``. To set the request body as a stream:: diff --git a/docs/backends/asynchttpclient.rst b/docs/backends/asynchttpclient.rst index cd359b1..e6dcf1b 100644 --- a/docs/backends/asynchttpclient.rst +++ b/docs/backends/asynchttpclient.rst @@ -1,15 +1,15 @@ -``AsyncHttpClientBackend`` -========================== +async-http-client backend +========================= To use, add the following dependency to your project:: - "com.softwaremill.sttp" %% "async-http-client-backend-future" % "0.0.20" + "com.softwaremill.sttp" %% "async-http-client-backend-future" % "1.0.0" // or - "com.softwaremill.sttp" %% "async-http-client-backend-scalaz" % "0.0.20" + "com.softwaremill.sttp" %% "async-http-client-backend-scalaz" % "1.0.0" // or - "com.softwaremill.sttp" %% "async-http-client-backend-monix" % "0.0.20" + "com.softwaremill.sttp" %% "async-http-client-backend-monix" % "1.0.0" // or - "com.softwaremill.sttp" %% "async-http-client-backend-cats" % "0.0.20" + "com.softwaremill.sttp" %% "async-http-client-backend-cats" % "1.0.0" This backend depends on `async-http-client <https://github.com/AsyncHttpClient/async-http-client>`_. A fully **asynchronous** backend, which uses `Netty <http://netty.io>`_ behind the @@ -19,7 +19,7 @@ The responses are wrapped depending on the dependency chosen in either a: * standard Scala ``Future`` * `Scalaz <https://github.com/scalaz/scalaz>`_ ``Task``. There's a transitive dependency on ``scalaz-concurrent``. -* `Monix <https://monix.io`_ ``Task``. There's a transitive dependency on ``monix-eval``. +* `Monix <https://monix.io>`_ ``Task``. There's a transitive dependency on ``monix-eval``. * Any type implementing the `Cats Effect <https://github.com/typelevel/cats-effect>`_ ``Async`` typeclass, such as ``cats.effect.IO``. There's a transitive dependency on ``cats-effect``. Next you'll need to add an implicit value:: @@ -44,10 +44,7 @@ Next you'll need to add an implicit value:: Streaming using Monix --------------------- -The Monix backend supports streaming (as both Monix and Async Http Client -support reactive streams ``Publisher``s out of the box). The type of -supported streams in this case is ``Observable[ByteBuffer]``. That is, you can -set such an observable as a request body:: +The Monix backend supports streaming (as both Monix and Async Http Client support reactive streams ``Publisher`` s out of the box). The type of supported streams in this case is ``Observable[ByteBuffer]``. That is, you can set such an observable as a request body:: import com.softwaremill.sttp._ @@ -77,6 +74,5 @@ And receive responses as an observable stream:: .response(asStream[Observable[ByteBuffer]]) .send() -It's also possible to use `fs2 <https://github.com/functional-streams-for-scala/fs2>`_ -streams for sending request & receiving responses. +It's also possible to use `fs2 <https://github.com/functional-streams-for-scala/fs2>`_ streams for sending request & receiving responses. diff --git a/docs/backends/custom.rst b/docs/backends/custom.rst index 86d750a..0820a26 100644 --- a/docs/backends/custom.rst +++ b/docs/backends/custom.rst @@ -1,11 +1,9 @@ +.. _custombackends: + Custom backends, logging, metrics ================================= -It is also entirely possible to write your own backend (if so, please consider -contributing!) or wrapping an existing one. You can even write completely -generic wrappers for any delegate backend, as each backend comes equipped -with a monad for the response type. This brings the possibility to ``map`` and -``flatMap`` over responses. +It is also entirely possible to write custom backends (if doing so, please consider contributing!) or wrap an existing one. One can even write completely generic wrappers for any delegate backend, as each backend comes equipped with a monad for the response type. This brings the possibility to ``map`` and ``flatMap`` over responses. Possible use-cases for wrapper-backend include: @@ -13,7 +11,64 @@ Possible use-cases for wrapper-backend include: * capturing metrics * request signing (transforming the request before sending it to the delegate) -To pass some context to wrapper-backends, requests can be *tagged*. Each -``RequestT`` instance contains a ``tags: Map[String, Any]`` field. This is unused -by http, but can be used e.g. to pass a metric name or logging context. +Request tagging +--------------- + +Each request contains a ``tags: Map[String, Any]`` map. This map can be used to tag the request with any backend-specific information, and isn't used in any way by sttp itself. + +Tags can be added to a request using the ``def tag(k: String, v: Any)`` method, and read using the ``def tag(k: String): Option[Any]`` method. + +Backends, or backend wrappers can use tags e.g. for logging, passing a metric name, using different connection pools, or even different delegate backends. + +Example backend wrapper +----------------------- + +Below is an example on how to implement a backend wrapper, which sends metrics for completed requests and wraps any ``Future``-based backend:: + + // the metrics infrastructure + trait MetricsServer { + def reportDuration(name: String, duration: Long): Unit + } + + class CloudMetricsServer extends MetricsServer { + override def reportDuration(name: String, duration: Long): Unit = ??? + } + + // the backend wrapper + class MetricWrapper[S](delegate: SttpBackend[Future, S], + metrics: MetricsServer) + extends SttpBackend[Future, S] { + + override def send[T](request: Request[T, S]): Future[Response[T]] = { + val start = System.currentTimeMillis() + + def report(metricSuffix: String): Unit = { + val metricPrefix = request.tag("metric").getOrElse("?") + val end = System.currentTimeMillis() + metrics.reportDuration(metricPrefix + "-" + metricSuffix, end - start) + } + + delegate.send(request).andThen { + case Success(response) if response.is200 => report("ok") + case Success(response) => report("notok") + case Failure(t) => report("exception") + } + } + + override def close(): Unit = delegate.close() + + override def responseMonad: MonadError[Future] = delegate.responseMonad + } + + // example usage + implicit val backend = new MetricWrapper( + AkkaHttpBackend(), + new CloudMetricsServer() + ) + + sttp + .get(uri"http://company.com/api/service1") + .tag("metric", "service1") + .send() + diff --git a/docs/backends/httpurlconnection.rst b/docs/backends/httpurlconnection.rst index a196343..e0f6594 100644 --- a/docs/backends/httpurlconnection.rst +++ b/docs/backends/httpurlconnection.rst @@ -1,8 +1,7 @@ -``HttpURLConnectionBackend`` -============================ +HttpURLConnection backend +========================= -The default **synchronous** backend. Sending a request returns a response wrapped -in the identity type constructor, which is equivalent to no wrapper at all. +The default **synchronous** backend. Sending a request returns a response wrapped in the identity type constructor, which is equivalent to no wrapper at all. To use, add an implicit value:: diff --git a/docs/backends/okhttp.rst b/docs/backends/okhttp.rst index 52fbc49..4111e39 100644 --- a/docs/backends/okhttp.rst +++ b/docs/backends/okhttp.rst @@ -1,11 +1,11 @@ -``OkHttpClientBackend`` -======================= +OkHttp backend +============== To use, add the following dependency to your project:: - "com.softwaremill.sttp" %% "okhttp-backend" % "0.0.20" + "com.softwaremill.sttp" %% "okhttp-backend" % "1.0.0" // or, for the monix version: - "com.softwaremill.sttp" %% "okhttp-backend-monix" % "0.0.20" + "com.softwaremill.sttp" %% "okhttp-backend-monix" % "1.0.0" This backend depends on `OkHttp <http://square.github.io/okhttp/>`_, and offers: diff --git a/docs/backends/start_stop.rst b/docs/backends/start_stop.rst index 6cd6f13..ed117a7 100644 --- a/docs/backends/start_stop.rst +++ b/docs/backends/start_stop.rst @@ -1,17 +1,9 @@ Starting & cleaning up ====================== -In case of most backends, you should only instantiate a backend once per -application, as a backend typically allocates resources such as thread or -connection pools. +In case of most backends, you should only instantiate a backend once per application, as a backend typically allocates resources such as thread or connection pools. -When ending the application, make sure to call ``backend.close()``, which will -free up resources used by the backend (if any). The close process might be -asynchronous, that is it can complete after the ``close()`` method returns. +When ending the application, make sure to call ``backend.close()``, which will free up resources used by the backend (if any). The close process might be asynchronous, that is it can complete after the ``close()`` method returns. -Note that only resources allocated by the backends are freed. For example, -if you use the ``AkkaHttpBackend()`` the ``close()`` method will terminate the -underlying actor system. However, if you have provided an existing actor system -upon backend creation (``AkkaHttpBackend.usingActorSystem``), the ``close()`` -method will be a no-op. +Note that only resources allocated by the backends are freed. For example, if you use the ``AkkaHttpBackend()`` the ``close()`` method will terminate the underlying actor system. However, if you have provided an existing actor system upon backend creation (``AkkaHttpBackend.usingActorSystem``), the ``close()`` method will be a no-op. diff --git a/docs/backends/summary.rst b/docs/backends/summary.rst index 6aa0446..d58bb54 100644 --- a/docs/backends/summary.rst +++ b/docs/backends/summary.rst @@ -1,8 +1,21 @@ +.. _backends_summary: + Supported backends ================== +sttp suports a number of synchornous and asynchronous backends. It's the backends that take care of managing connections, sending requests and receiving responses: sttp defines only the API to describe the requests to be send and handle the response data. It's the backends where all the heavy-lifting is done. + +Choosing the right backend depends on a number of factors: if you are using sttp to explore some data, or is it a production system; are you using a synchronous, blocking architecture or an asynchronous one; do you work mostly with Scala's ``Future``, or maybe you use some form of a ``Task`` abstraction; finally, if you want to stream requests/responses, or not. + +Each backend has two type parameters: + +* ``R[_]``, the type constructor in which responses are wrapped. That is, when you invoke ``send()`` on a request description, do you get a ``Response[_]`` directly, or is it wrapped in a ``Future`` or a ``Task``? +* ``S``, the type of supported streams. If ``Nothing``, streaming is not supported. Otherwise, the given type can be used to send request bodies or receive response bodies. + +Below is a summary of all the backends. See the sections on individual backend implementations for more information. + ================================ ============================ ================================================ -Class Result wrapper Supported stream type +Class Response wrapper Supported stream type ================================ ============================ ================================================ ``HttpURLConnectionBackend`` None (``Id``) n/a ``AkkaHttpBackend`` ``scala.concurrent.Future`` ``akka.stream.scaladsl.Source[ByteString, Any]`` diff --git a/docs/testing.rst b/docs/backends/testing.rst index 0ac21c2..c0cfc6f 100644 --- a/docs/testing.rst +++ b/docs/backends/testing.rst @@ -1,13 +1,9 @@ Testing ======= -If you need a stub backend for use in tests instead of a "real" backend (you -probably don't want to make HTTP calls during unit tests), you can use the -``SttpBackendStub`` class. It allows specifying how the backend should respond -to requests matching given predicates. +If you need a stub backend for use in tests instead of a "real" backend (you probably don't want to make HTTP calls during unit tests), you can use the ``SttpBackendStub`` class. It allows specifying how the backend should respond to requests matching given predicates. -A backend stub can be created using an instance of a "real" backend, or by -explicitly giving the response wrapper monad and supported streams type. +A backend stub can be created using an instance of a "real" backend, or by explicitly giving the response wrapper monad and supported streams type. For example:: @@ -23,18 +19,14 @@ For example:: val response2 = sttp.post(uri"http://example.org/d/e").send() // response2.code will be 500 -However, this approach has one caveat: the responses are not type-safe. That -is, the backend cannot match on or verify that the type included in the -response matches the response type requested. +However, this approach has one caveat: the responses are not type-safe. That is, the backend cannot match on or verify that the type included in the response matches the response type requested. -It is also possible to create a stub backend which delegates calls to another -(possibly "real") backend if none of the specified predicates match a request. -This can be useful during development, to partially stub a yet incomplete -API with which we integrate:: +It is also possible to create a stub backend which delegates calls to another (possibly "real") backend if none of the specified predicates match a request. This can be useful during development, to partially stub a yet incomplete API with which we integrate:: - implicit val testingBackend = SttpBackendStub.withFallback(HttpURLConnectionBackend()) - .whenRequestMatches(_.uri.path.startsWith(List("a"))) - .thenRespond("I'm a STUB!") + implicit val testingBackend = + SttpBackendStub.withFallback(HttpURLConnectionBackend()) + .whenRequestMatches(_.uri.path.startsWith(List("a"))) + .thenRespond("I'm a STUB!") val response1 = sttp.get(uri"http://api.internal/a").send() // response1.body will be Right("I'm a STUB") diff --git a/docs/community.rst b/docs/community.rst new file mode 100644 index 0000000..4c68910 --- /dev/null +++ b/docs/community.rst @@ -0,0 +1,8 @@ +Community +========= + +If you have a question, or hit a problem, feel free to ask on our `gitter channel <https://gitter.im/softwaremill/sttp>`_! + +Or, if you encounter a bug, something is unclear in the code or documentation, don't hesitate and `open an issue <https://github.com/softwaremill/sttp/issues>`_ on GitHub. + +We are also always looking for contributions and new ideas, so if you'd like to get into the project, check out the open issues, or post your own suggestions! diff --git a/docs/conf/redirects.rst b/docs/conf/redirects.rst new file mode 100644 index 0000000..e826c7c --- /dev/null +++ b/docs/conf/redirects.rst @@ -0,0 +1,10 @@ +Redirects +========= + +By default, sttp follows redirects. + +If you'd like to disable following redirects, use the ``followRedirects`` method:: + + sttp.followRedirects(false) + +If a request has been redirected, the history of all followed redirects is accessible through the ``response.history`` list. The first response (oldest) comes first. The body of each response will be a ``Left(message)`` (as the status code is non-2xx), where the message is whatever the server returned as the response body. diff --git a/docs/conf/timeouts.rst b/docs/conf/timeouts.rst index 72b2f4a..4c046bc 100644 --- a/docs/conf/timeouts.rst +++ b/docs/conf/timeouts.rst @@ -1,7 +1,7 @@ Timeouts ======== -Sttp supports read and connection timeouts: +sttp supports read and connection timeouts: * Connection timeout - can be set globally (30 seconds by default) * Read timeout - can be set per request (1 minute by default) @@ -11,7 +11,7 @@ How to use:: import com.softwaremill.sttp._ import scala.concurrent.duration._ - // all backends provide a constructor that allows users to specify backend options + // all backends provide a constructor that allows to specify backend options implicit val backend = HttpURLConnectionBackend( options = SttpBackendOptions.connectionTimeout(1.minute)) diff --git a/docs/goals.rst b/docs/goals.rst index 9b59991..20f9537 100644 --- a/docs/goals.rst +++ b/docs/goals.rst @@ -8,9 +8,7 @@ Goals of the project * provide support for backend-specific request/response streaming * minimum dependencies -See also the `introduction to sttp <https://softwaremill.com/introducing-sttp-the-scala-http-client>`_ -and `sttp streaming & URI interpolators <https://softwaremill.com/sttp-streaming-uri-interpolator>`_ -blogs. +See also the `introduction to sttp <https://softwaremill.com/introducing-sttp-the-scala-http-client>`_ and `sttp streaming & URI interpolators <https://softwaremill.com/sttp-streaming-uri-interpolator>`_ blogs. Non-goals of the project ------------------------ diff --git a/docs/index.rst b/docs/index.rst index 0550cd7..e030323 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,12 +1,11 @@ sttp: the Scala HTTP client you always wanted! ============================================== -`sttp <https://github.com/softwaremill/sttp>`_ is an open-source library which provides a clean, programmer-friendly API to define HTTP requests and execute them using one of the wrapped backends, such as `akka-http <https://doc.akka.io/docs/akka-http/current/scala/http/>`_, `async-http-client <https://github.com/AsyncHttpClient/async-http-client>`_ or `OkHttp <http://square.github.io/okhttp/>`_. +Welcome! -First impressions ------------------ +`sttp <https://github.com/softwaremill/sttp>`_ is an open-source library which provides a clean, programmer-friendly API to define HTTP requests and execute them using one of the wrapped backends, such as `akka-http <https://doc.akka.io/docs/akka-http/current/scala/http/>`_, `async-http-client <https://github.com/AsyncHttpClient/async-http-client>`_ or `OkHttp <http://square.github.io/okhttp/>`_. -.. code-block:: scala +Here's a very quick example of sttp in action:: import com.softwaremill.sttp._ @@ -25,38 +24,42 @@ First impressions // response.unsafeBody: by default read into a String println(response.unsafeBody) - - -Quickstart with Ammonite ------------------------- - -If you are an `Ammonite <http://ammonite.io>`_ user, you can quickly start -experimenting with sttp by copy-pasting the following:: - - import $ivy.`com.softwaremill.sttp::core:0.0.20` - import com.softwaremill.sttp._ - implicit val backend = HttpURLConnectionBackend() - sttp.get(uri"http://httpbin.org/ip").send() - -Adding sttp to your project ---------------------------- -The basic dependency which provides the default, synchornous backend is:: +For more details, explore the subjects below! - "com.softwaremill.sttp" %% "core" % "0.0.20" - -``sttp`` is available for Scala 2.11 and 2.12, and requires Java 7 if using an ``OkHttp`` based backend, or Java 8 otherwise. The core module has no transitive dependencies. +.. toctree:: + :maxdepth: 2 + :caption: Getting started -If you'd like to use an alternate backend, [see below](#supported-backends) for additional instructions. + quickstart + goals + community .. toctree:: :maxdepth: 2 + :caption: Request definition - goals requests/basics requests/uri - requests/defaults + requests/headers + requests/cookies + requests/authentication + requests/body + requests/multipart + requests/streaming requests/type + +.. toctree:: + :maxdepth: 2 + :caption: Responses + + responses/basics + responses/body + +.. toctree:: + :maxdepth: 2 + :caption: Backends + backends/summary backends/start_stop backends/httpurlconnection @@ -64,17 +67,22 @@ If you'd like to use an alternate backend, [see below](#supported-backends) for backends/asynchttpclient backends/okhttp backends/custom + backends/testing + +.. toctree:: + :maxdepth: 2 + :caption: Configuration + conf/timeouts conf/ssl - conf/proxy + conf/proxy + conf/redirects + +.. toctree:: + :maxdepth: 2 + :caption: More information + json - testing other credits -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/docs/json.rst b/docs/json.rst index 28cee81..7860e9d 100644 --- a/docs/json.rst +++ b/docs/json.rst @@ -1,14 +1,18 @@ +.. _json: + JSON ==== -JSON encoding of bodies and decoding of responses can be handled using -`Circe <https://circe.github.io/circe/>`_ by the ``circe`` module. To use -add the following dependency to your project:: +Adding support for JSON (or other format) bodies in requests/responses is a matter of providing a :ref:`body serializer <requestbody_custom>` and/or a :ref:`response body specification <responsebodyspec_custom>`. Both are quite straightforward to implement, so integrating with your favorite JSON library shouldn't be a problem. However, there are some integrations available out-of-the-box. + +Circe +----- + +JSON encoding of bodies and decoding of responses can be handled using `Circe <https://circe.github.io/circe/>`_ by the ``circe`` module. To use add the following dependency to your project:: - "com.softwaremill.sttp" %% "circe" % "0.0.20" + "com.softwaremill.sttp" %% "circe" % "1.0.0" -This module adds a method to the request and a function that can be given to -a request to decode the response to a specific object. +This module adds a method to the request and a function that can be given to a request to decode the response to a specific object:: import com.softwaremill.sttp._ import com.softwaremill.sttp.circe._ @@ -26,4 +30,29 @@ a request to decode the response to a specific object. .response(asJson[Response]) .send() - +Json4s +------ + +To encode and decode json using json4s-native, add the following dependency to your project:: + + "com.softwaremill.sttp" %% "json4s" % "1.0.0" + +Using this module it is possible to set request bodies and read response bodies as case classes, using the implicitly available ``org.json4s.Formats`` (which defaults to ``org.json4s.DefaultFormats``). + +Usage example:: + + import com.softwaremill.sttp._ + import com.softwaremill.sttp.json4s._ + + implicit val backend = HttpURLConnectionBackend() + + case class Payload(...) + val requestPayload: Payload = Payload(...) + + val response: Response[Payload] = + sttp + .post(uri"...") + .body(requestPayload) + .response(asJson[Response]) + .send() + diff --git a/docs/quickstart.rst b/docs/quickstart.rst new file mode 100644 index 0000000..eb703d0 --- /dev/null +++ b/docs/quickstart.rst @@ -0,0 +1,37 @@ +.. _quickstart: + +Quickstart +========== + +The main sttp API comes in a single jar without transitive dependencies. This also includes a default, synchronous backend, which is based on Java's ``HttpURLConnection``. For production usages, you'll often want to use an alternate backend (but what's important is that the API remains the same!). See the section on :ref:`backends <backends_summary>` for additional instructions. + +Using sbt +--------- + +The basic dependency which provides the API and the default synchronous backend is:: + + "com.softwaremill.sttp" %% "core" % "1.0.0" + +``sttp`` is available for Scala 2.11 and 2.12, and requires Java 8. The core module has no transitive dependencies. + +Using Ammonite +-------------- + +If you are an `Ammonite <http://ammonite.io>`_ user, you can quickly start experimenting with sttp by copy-pasting the following:: + + import $ivy.`com.softwaremill.sttp::core:1.0.0` + import com.softwaremill.sttp._ + implicit val backend = HttpURLConnectionBackend() + sttp.get(uri"http://httpbin.org/ip").send() + +Imports +------- + +Working with sttp is most convenient if you import the ``sttp`` package entirely:: + + import com.softwaremill.sttp._ + +This brings into scope the starting point for defining requests and some helper methods. All examples in this guide assume that this import is in place. + +And that's all you need to start using sttp! To create and send your first request, import the above, type ``sttp.`` and see where your IDE's auto-complete gets you! Or, read on about the :ref:`basics of defining requests <request_basics>`. + diff --git a/docs/requests/authentication.rst b/docs/requests/authentication.rst new file mode 100644 index 0000000..5c60051 --- /dev/null +++ b/docs/requests/authentication.rst @@ -0,0 +1,15 @@ +.. _authentication: + +Authentication +============== + +sttp supports basic and bearer-token based authentication. In both cases, an ``Authorization`` header is added with the appropriate credentials. + +Basic authentication, using which the username and password are encoded using Base64, can be added as follows:: + + sttp.auth.basic(username, password) + +A bearer token can be added using:: + + sttp.auth.bearer(token) + diff --git a/docs/requests/basics.rst b/docs/requests/basics.rst index 6a6202c..6ed8872 100644 --- a/docs/requests/basics.rst +++ b/docs/requests/basics.rst @@ -1,43 +1,51 @@ -Defining requests: basics +.. _request_basics: + +Request definition basics ========================= -To get easy access to the request definition API, add the following import:: +As mentioned in the :ref:`quickstart <quickstart>`, the following import will be needed:: import com.softwaremill.sttp._ -This brings into scope ``sttp``, the starting request (it's an empty request -with the ``Accept-Encoding: gzip, defalte`` header added). This request can -be customised, each time yielding a new, immutable request description -(unless a mutable body is set on the request, such as a byte array). +This brings into scope ``sttp``, the starting request. This request can be customised, each time yielding a new, immutable request definition (unless a mutable body is set on the request, such as a byte array). As the request definition is immutable, it can be freely stored in values, shared across threads, and customized multiple times in various ways. -For example, we can set a cookie, string-body and specify that this should -be a ``POST`` request to a given URI:: +For example, we can set a cookie, ``String`` -body and specify that this should be a ``POST`` request to a given URI:: val request = sttp .cookie("login", "me") .body("This is a test") .post(uri"http://endpoint.com/secret") + +The request parameters (headers, cookies, body etc.) can be specified **in any order**. It doesn't matter if the request method, the body, the headers or connection options are specified in this sequence or another. This way you can build arbitrary request templates, capturing all that's common among your requests, and customizing as needed. Remember that each time a modifier is applied to a request, you get a new immutable object. + +There's a lot of ways in which you can customize a request, which are covered in this guide. Another option is to just explore the API: most of the methods are self-explanatory and carry scaladocs if needed. -The request parameters (headers, cookies, body etc.) can be specified in any -order. There's a lot of ways in which you can customize a request: just -explore the API. +Using the modifiers, each time we get a new request definition, but it's just a description: a data object; nothing is sent over the network until the ``send()`` method is invoked. -You can create a request description without knowing how it will be sent. -But to send a request, you will need a backend. A default, synchronous backend -based on Java's ``HttpURLConnection`` is provided out-of-the box. An implicit -value of type ``SttpBackend`` needs to be in scope to invoke the ``send()`` on the -request:: +Sending a request +----------------- + +A request definition can be created without knowing how it will be sent. But to send a request, a backend is needed. A default, synchronous backend based on Java's ``HttpURLConnection`` is provided out-of-the box. + +To invoke the ``send()`` method on a request description, an implicit value of type ``SttpBackend`` needs to be in scope:: implicit val backend = HttpURLConnectionBackend() val response: Response[String] = request.send() -By default the response body is read into a utf-8 string. How the response body -is handled is also part of the request description. The body can be ignore -(``.response(ignore)``), read into a sequence of parameters -(``.response(asParams)``), mapped (``.mapResponse``) and more; some backends also -support request & response streaming. +The default backend doesn't wrap the response into any container, but other asynchronous backends might do so. See the section on :ref:`backends <backends_summary>` for more details. + +.. note:: + + Only requests with the request method and uri can be sent. If trying to send a request without these components specified, a compile-time error will be reported. On how this is implemented, see the documentation on the :ref:`type of request definitions <request_type>`. + +Starting requests +----------------- + +sttp provides two starting requests: + +* ``sttp``, which is an empty request with the ``Accept-Encoding: gzip, deflate`` header added. That's the one that is most commonly used. +* ``empty``, a completely empty request, with no headers at all. + +Both of these requests will by default read the response body into a UTF-8 ``String``. How the response body is handled is also part of the request definition. See the section on :ref:`response body specifications <responsebodyspec>` for more details on how to customize that. -The default backend doesn't wrap the response into any container, but other -asynchronous backends might do so. The type parameter in the ``Response[_]`` -type specifies the type of the body. diff --git a/docs/requests/body.rst b/docs/requests/body.rst new file mode 100644 index 0000000..20dd5d6 --- /dev/null +++ b/docs/requests/body.rst @@ -0,0 +1,87 @@ +.. _requestbody: + +Setting the request body +======================== + +Text data +--------- + +In its simplest form, the request's body can be set as a ``String``. By default, this method will: + +* use the UTF-8 encoding to convert the string to a byte array +* if not specified before, set ``Content-Type: text/plain`` +* if not specified before, set ``Content-Length`` to the number of bytes in the array + +A ``String`` body can be set on a request as follows:: + + sttp.body("Hello, world!") + +It is also possible to use a different character encoding:: + + def body(b: String) + def body(b: String, encoding: String) + +Binary data +----------- + +To set a binary-data body, the following methods are available:: + + def body(b: Array[Byte]) + def body(b: ByteBuffer) + def body(b: InputStream) + +If not specified before, these methods will set the content type to ``application/octet-stream``. When using a byte array, additionally the content length will be set to the length of the array (unless specified explicitly). + +.. note:: + + While the object defining a request is immutable, setting a mutable request body will make the whole request definition mutable as well. With ``InputStream``, the request can be moreover sent only once, as input streams can be consumed once. + +Uploading files +--------------- + +To upload a file, simply set the request body as a ``File`` or ``Path``:: + + def body(b: File) + def body(b: Path) + +As with binary body methods, the content type will default to ``application/octet-stream``, and the content length will be set to the length of the file (unless specified explicitly). + +See also :ref:`multi-part <multipart>` and :ref:`streaming <streaming>` support. + +Form data +--------- + +If you set the body as a ``Map[String, String]`` or ``Seq[(String, String)]``, it will be encoded as form-data (as if a web form with the given values was submitted). The content type will default to ``application/x-www-form-urlencoded``; content length will also be set if not specified. + +By default, the ``UTF-8`` encoding is used, but can be also specified explicitly:: + + def body(fs: Map[String, String]) + def body(fs: Map[String, String], encoding: String) + def body(fs: (String, String)*) + def body(fs: Seq[(String, String)], encoding: String) + +.. _requestbody_custom: + +Custom body serializers +----------------------- + +It is also possible to set custom types as request bodies, as long as there's an implicit ``BodySerializer[B]`` value in scope, which is simply an alias for a function:: + + type BodySerializer[B] = B => BasicRequestBody + +A ``BasicRequestBody`` is a wrapper for one of the supported request body types: a ``String``/byte array or an input stream. + +For example, here's how to write a custom serializer for a case class, with serializer-specific default content type:: + + case class Person(name: String, surname: String, age: Int) + + // for this example, assuming names/surnames can't contain commas + implicit val personSerializer: BodySerializer[Person] = { p: Person => + val serialized = s"${p.name},${p.surname},${p.age}" + StringBody(serialized, "UTF-8", Some("application/csv")) + } + + sttp.body(Person("mary", "smith", 67)) + +See the implementations of the ``BasicRequestBody`` trait for more options. + diff --git a/docs/requests/cookies.rst b/docs/requests/cookies.rst new file mode 100644 index 0000000..0645316 --- /dev/null +++ b/docs/requests/cookies.rst @@ -0,0 +1,28 @@ +.. _cookies: + +Cookies +======= + +Cookies sent in requests are key-value pairs contained in the ``Cookie`` header. They can be set on a request in a couple of ways. The first is using the ``.cookie(name: String, value: String)`` method. This will yield a new request definition which, when sent, will contain the given cookie. + +Cookies can also be set using the following methods:: + + def cookie(nv: (String, String)) + def cookie(n: String, v: String) + def cookies(nvs: (String, String)*) + +Cookies from responses +---------------------- + +It is often necessary to copy cookies from a response, e.g. after a login request is sent, and a successful response with the authentication cookie received. Having an object ``response: Response[_]``, cookies on a request can be copied:: + + // Method signature + def cookies(r: Response[_]) + + // Usage + sttp.cookies(response) + +Or, it's also possible to store only the ``com.softwaremill.sttp.Cookie`` objects (a sequence of which can be obtained from a response), and set the on the request:: + + def cookies(cs: Seq[Cookie]) + diff --git a/docs/requests/defaults.rst b/docs/requests/defaults.rst deleted file mode 100644 index f393c04..0000000 --- a/docs/requests/defaults.rst +++ /dev/null @@ -1,11 +0,0 @@ -Defaults -======== - -The encoding for ``String``s defaults to ``utf-8``. - -Unless explicitly specified, the ``Content-Type`` defaults to: - -* ``text/plain`` for text -* ``application/x-www-form-urlencoded`` for form data -* ``multipart/form-data`` for multipart form data -* ``application/octet-stream`` for everything else (binary) diff --git a/docs/requests/headers.rst b/docs/requests/headers.rst new file mode 100644 index 0000000..ad2e695 --- /dev/null +++ b/docs/requests/headers.rst @@ -0,0 +1,30 @@ +Headers +======= + +Arbitrary headers can be set on the request using the ``.header`` method:: + + sttp.header("User-Agent", "myapp") + +As with any other request definition modifier, this method will yield a new request, which has the given header set. The headers can be set at any point when defining the request, arbitrarily interleaved with other modifiers. + +While most headers should be set only once on a request, HTTP allows setting a header multiple times. That's why the ``header`` method has an additional optional boolean parameter, ``replaceExisting``, which defaults to ``true``. This way, if the same header is specified twice, only the last value will be included in the request. If previous values should be preserved, set this parameter to ``false``. + +There are also variants of this method accepting a number of headers:: + + def header(k: String, v: String, replaceExisting: Boolean = false) + def headers(hs: Map[String, String]) + def headers(hs: (String, String)*) + +Both of the ``headers`` append the given headers to the ones currently in the request, without removing duplicates. + +Common headers +-------------- + +For some common headers, dedicated methods are provided:: + + def contentType(ct: String) + def contentType(ct: String, encoding: String) + def contentLength(l: Long) + def acceptEncoding(encoding: String) + +See also documentation on setting :ref:`cookies <cookies>` and :ref:`authentication <authentication>`. diff --git a/docs/requests/multipart.rst b/docs/requests/multipart.rst new file mode 100644 index 0000000..ac921fc --- /dev/null +++ b/docs/requests/multipart.rst @@ -0,0 +1,50 @@ +.. _multipart: + +Multipart requests +================== + +To set a multipart body on a request, the ``multipartBody`` method should be used (instead of ``body``). Each body part is represented as an instance of ``Multipart``, which can be conveniently constructed using ``multipart`` methods coming from the ``com.softwaremill.sttp`` package. + +A single part of a multipart request consist of a mandatory name and a payload of type: + +* ``String`` +* ``Array[Byte]`` +* ``ByteBuffer`` +* ``InputStream`` +* ``File`` +* ``Path`` +* ``Map[String, String]`` +* ``Seq[(String, String)]`` + +The content type of each part is by default the same as when setting simple bodies: ``text/plain`` for parts of type ``String``, ``application/x-www-form-urlencoded`` for parts of key-value pairs (form data) and ``application/octet-stream`` otherwise (for binary data). + +The parts can be specified using either a ``Seq[Multipart]`` or by using multiple arguments:: + + def multipartBody(ps: Seq[Multipart]) + def multipartBody(p1: Multipart, ps: Multipart*) + +For example:: + + sttp.multipartBody( + multipart("text_part", "data1"), + multipart("file_part", someFile), // someFile: File + multipart("form_part", Map("x" -> "10", "y" -> "yes")) + ) + +Customising part meta-data +-------------------------- + +For each part, an optional filename can be specified, as well as a custom content type and additional headers. The following methods are available on ``Multipart`` instances:: + + case class Multipart { + def fileName(v: String): Multipart + def contentType(v: String): Multipart + def header(k: String, v: String): Multipart + } + +For example:: + + sttp.multipartBody( + multipart("logo", logoFile).fileName("logo.jpg").contentType("image/jpg"), + multipart("text", docFile).fileName("text.doc") + ) diff --git a/docs/requests/streaming.rst b/docs/requests/streaming.rst new file mode 100644 index 0000000..85bf2b6 --- /dev/null +++ b/docs/requests/streaming.rst @@ -0,0 +1,28 @@ +.. _streaming: + +Streaming +========= + +Some backends (see :ref:`backends summary <backends_summary>`) support streaming bodies. If that's the case, you can set a stream of the supported type as a request body using the ``streamBody`` method, instead of the usual ``body`` method. + +.. note:: + + Here, streaming refers to (usually) non-blocking, asynchronous streams of data. To send data which is available as an ``InputStream``, or a file from local storage (which is available as a ``File`` or ``Path``), no special backend support is needed. See the documenttation on :ref:`setting the request body <requestbody>`. + +For example, using the :ref:`akka-http backend <akkahttp>`, a request with a streaming body can be defined as follows:: + + import com.softwaremill.sttp._ + import com.softwaremill.sttp.akkahttp._ + + import akka.stream.scaladsl.Source + import akka.util.ByteString + + val source: Source[ByteString, Any] = ... + + sttp + .streamBody(source) + .post(uri"...") + +.. note:: + + A request with the body set as a stream can only be sent using a backend supporting exactly the given type of streams. diff --git a/docs/requests/type.rst b/docs/requests/type.rst index 3cefe97..e246def 100644 --- a/docs/requests/type.rst +++ b/docs/requests/type.rst @@ -1,17 +1,22 @@ -Request type -============ +.. _request_type: -All request descriptions have type ``RequestT[U, T, S]`` (T as in Template). -If this looks a bit complex, don't worry, what the three type parameters stand -for is the only thing you'll hopefully have to remember when using the API! +The type of request definitions +=============================== + +All request definitions have type ``RequestT[U, T, S]`` (RequestT as in Request Template). If this looks a bit complex, don't worry, what the three type parameters stand for is the only thing you'll hopefully have to remember when using the API! Going one-by-one: * ``U[_]`` specifies if the request method and URL are specified. Using the API, this can be either ``type Empty[X] = None``, meaning that the request has neither a method nor an URI. Or, it can be ``type Id[X] = X`` (type-level identity), meaning that the request has both a method and an URI specified. Only requests with a specified URI & method can be sent. -* ``T`` specifies the type to which the response will be read. By default, this is ``String``. But it can also be e.g. ``Array[Byte]`` or ``Unit``, if the response should be ignored. Response body handling can be changed by calling the ``.response`` method. With backends which support streaming, this can also be a supported stream type. -* ``S`` specifies the stream type that this request uses. Most of the time this will be ``Nothing``, meaning that this request does not send a streaming body or receive a streaming response. So most of the times you can just ignore that parameter. But, if you are using a streaming backend and want to send/receive a stream, the ``.streamBody`` or ``response(asStream[S])`` will change the type parameter. +* ``T`` specifies the type to which the response will be read. By default, this is ``String``. But it can also be e.g. ``Array[Byte]`` or ``Unit``, if the response should be ignored. Response body handling can be changed by calling the ``.response`` method. With backends which support streaming, this can also be a supported stream type. See :ref:`response body specifications <responsebodyspec>` for more details. +* ``S`` specifies the stream type that this request uses. Most of the time this will be ``Nothing``, meaning that this request does not send a streaming body or receive a streaming response. So most of the time you can just ignore that parameter. But, if you are using a streaming backend and want to send/receive a stream, the ``.streamBody`` or ``response(asStream[S])`` will change the type parameter. There are two type aliases for the request template that are used: * ``type Request[T, S] = RequestT[Id, T, S]``. A sendable request. * ``type PartialRequest[T, S] = RequestT[Empty, T, S]`` + +As ``sttp``, the starting request, by default reads the body into a ``String``, its type is:: + + sttp: PartialRequest[String, Nothing] + diff --git a/docs/requests/uri.rst b/docs/requests/uri.rst index bcb7c88..93c7b47 100644 --- a/docs/requests/uri.rst +++ b/docs/requests/uri.rst @@ -1,7 +1,16 @@ -Defining requests: URI Interpolator -=================================== +URIs +==== -Using the URI interpolator it's possible to conveniently create ``Uri` instances, which can then be used to specify request endpoints, for example:: +A request can only be sent if the request method & URI are defined. To represent URIs, sttp comes with a ``Uri`` case class, which captures all of the parts of an address. + +To specify the request method and URI, use one of the methods on the request definition corresponding to the name of the desired HTTP method: ``.post``, ``.get``, ``.put`` etc. All of them accept a single parameter, the URI to which the request should be sent (these methods only modify the request definition; they don't send the requests). + +The ``Uri`` class is immutable, and can be constructed by hand, but in many cases the URI interpolator will be easier to use. + +URI interpolator +---------------- + +Using the URI interpolator it's possible to conveniently create ``Uri`` instances, for example:: import com.softwaremill.sttp._ @@ -10,18 +19,55 @@ Using the URI interpolator it's possible to conveniently create ``Uri` instances val endpoint: Uri = uri"http://example.com/$user/skills?filter=$filter" -Any values embedded in the URI will be URL-encoded, taking into account the -context (e.g., the whitespace in ``user`` will be %-encoded as ``%20D``, while the -whitespace in ``filter`` will be query-encoded as ``+``). + assert(endpoint.toString == + "http://example.com/Mary%20Smith/skills?filter=programming+languages") + +Note the ``uri`` prefix before the string and the standard Scala string-embedding syntax (``$user``, ``$filter``). + +Any values embedded in the URI will be URL-encoded, taking into account the context (e.g., the whitespace in ``user`` will be %-encoded as ``%20D``, while the whitespace in ``filter`` will be query-encoded as ``+``). + +All components of the URI can be embedded from values: scheme, username/password, host, port, path, query and fragment. + +Optional values +--------------- + +The URI interpolator supports optional values for hosts (subdomains), query parameters and the fragment. If the value is ``None``, the appropriate URI component will be removed. For example:: + + val v1 = None + val v2 = Some("v2") + + val u1 = uri"http://example.com?p1=$v1&p2=v2" + assert(u1.toString == "http://example.com?p2=v2") + + val u2 = uri"http://$v1.$v2.example.com" + assert(u2.toString == "http://v2.example.com") + + val u3 = uri"http://example.com#$v1" + assert(u3.toString == "http://example.com") + +Maps and sequences +------------------ + +Maps, sequences of tuples and sequences of values can be embedded in the query part. They will be expanded into query parameters. Maps and sequences of tuples can also contain optional values, for which mappings will be removed if ``None``. + +For example:: + + val ps = Map("p1" -> "v1", "p2" -> "v2") + val u4 = uri"http://example.com?$ps&p3=p4" + assert(u4.toString == "http://example.com?p1=v1&p2=v2&p3=p4") + +Sequences in the host part will be expanded to a subdomain sequence. + +Special cases +------------- + +If a string containing the protocol is embedded *as the very beginning*, it will not be escaped, allowing to embed entire addresses as prefixes, e.g.: ``uri"$endpoint/login"``, where ``val endpoint = "http://example.com/api"``. + +This is useful when a base URI is stored in a value, and can then be used as a base for constructing more specific URIs. -The possibilities of the interpolator don't end here. Other supported features: +All features combined +--------------------- -* parameters can have optional values: if the value of a parameter is ``None``, it will be removed -* maps, sequences of tuples and sequences of values can be embedded in the query part. They will be expanded into query parameters. Maps and sequences of tuples can also contain optional values, for which mappings will be removed if ``None``. -* optional values in the host part will be expanded to a subdomain if ``Some``, removed if ``None`` -* sequences in the host part will be expanded to a subdomain sequence -* if a string containing the protocol is embedded *as the very beginning*, it will not be escaped, allowing to embed entire addresses as prefixes, e.g.: ``uri"$endpoint/login"``, where ``val endpoint = "http://example.com/api"``. - A fully-featured example:: import com.softwaremill.sttp._ diff --git a/docs/responses/basics.rst b/docs/responses/basics.rst new file mode 100644 index 0000000..3afc2f2 --- /dev/null +++ b/docs/responses/basics.rst @@ -0,0 +1,50 @@ +Responses +========= + +Responses are represented as instances of the case class ``Response[T]``, where ``T`` is the type of the response body. When sending a request, the response will be returned in a wrapper. For example, for asynchronous backends, we can get a ``Future[Response[T]]``, while for the default synchronous backend, the wrapper will be a no-op, ``Id``, which is the same as no wrapper at all. + +If sending the request fails, either due to client or connection errors, an exception will be thrown (synchronous backends), or an error will be represented in the wrapper (e.g. a failed future). + +.. note:: + + If the request completes, but results in a non-2xx return code, the request is still considered successful, that is, a ``Response[T]`` will be returned. See :ref:`response body specifications <responsebodyspec>` for details on how such cases are handled. + +Response code +------------- + +The response code is available through the ``.code`` property. There are also methods such as ``.isSuccess`` or ``.isServerError`` for checking specific response code ranges. + +Response headers +---------------- + +Response headers are available through the ``.headers`` property, which gives all headers as a sequence (not as a map, as there can be multiple headers with the same name). + +Individual headers can be obtained using the methods:: + + def header(h: String): Option[String] + def headers(h: String): Seq[String] + +There are also helper methods available to read some commonly accessed headers:: + + def contentType: Option[String] + def contentLength: Option[Long] + +Finally, it's possible to parse the response cookies into a sequence of the ``Cookie`` case class:: + + def cookies: Seq[Cookie] + +If the cookies from a response should be set without changes on the request, this can be done directly; see the :ref:`cookies <cookies>` section in the request definition documentation. + +Obtaining the response body +--------------------------- + +The response body can be obtained through the ``.body`` property, which has type ``Either[String, T]``. ``T`` is the body deserialized as specified in the request - see the next section on :ref:`response body specifications <responsebodyspec>`. + +The response body is an either as the body can only be deserialized if the server responded with a success code (2xx). Otherwise, the response body is most probably an error message. + +Hence, the ``response.body`` will be a: + +* ``Left(errorMessage)`` if the request is successful, but response code is not 2xx. +* ``Right(deserializedBody``) if the request is successful and the response code is 2xx. + +You can also forcibly get the deserialized body, regardless of the response code and risking an excepiton being thrown, using the ``response.unsafeBody`` method. diff --git a/docs/responses/body.rst b/docs/responses/body.rst new file mode 100644 index 0000000..10658ed --- /dev/null +++ b/docs/responses/body.rst @@ -0,0 +1,92 @@ +.. _responsebodyspec: + +Response body specification +=========================== + +By default, the received response body will be read as a ``String``, using the ``UTF-8`` encoding. This is of course configurable: response bodies can be ignored, deserialized into custom types, recevied as a stream or saved to a file. + +How the response body will be read is part of the request definition, as already when sending the request, the backend needs to know what to do with the response. The type to which the response body should be deserialized is the second type parameter of ``RequestT``, and stored in the request definition as the ``request.response: ResponseAs[T, S]`` property. + +Basic response specifications +----------------------------- + +To conveniently specify how to deserialize the response body, a number of ``asXxx`` methods are available. They can be used to provide a value for the request definition's ``response`` modifier:: + + sttp.response(asByteArray) + +When the above request is completed and sent, it will result in a ``Response[Array[Byte]]``. Other possible response specifications are:: + + def ignore: ResponseAs[Unit, Nothing] + def asString: ResponseAs[String, Nothing] + def asString(encoding: String): ResponseAs[String, Nothing] + def asByteArray: ResponseAs[Array[Byte], Nothing] + def asParams: ResponseAs[Seq[(String, String)], Nothing] + def asParams(encoding: String): ResponseAs[Seq[(String, String)], Nothing] = + def asFile(file: File, overwrite: Boolean = false): ResponseAs[File, Nothing] + def asPath(path: Path, overwrite: Boolean = false): ResponseAs[Path, Nothing] + +Hence, to discard the response body, simply specify:: + + sttp.response(ignore) + +And to save the response to a file:: + + sttp.response(asFile(someFile)) + +.. note:: + + As the handling of response is specified upfront, there's no need to "consume" the response body. It can be safely discarded if not needed. + +.. _responsebodyspec_custom: + +Custom body deserializers +------------------------- + +It's possible to define custom body deserializers by taking any of the built-in response specifications and mapping over them. Each ``ResponseAs`` instance has a ``map`` method, which can be used to transform it to a specification for another type. Each such value is immutable and can be used multiple times. + +As an example, to read the response body as an int, the following response specification can be defined (warning: this ignores the possibility of exceptions!):: + + val asInt: ResponseAs[Int, Nothing] = asString.map(_.toInt) + + sttp + .response(asInt) + ... + +To integrate with a third-party JSON library:: + + def parseJson(json: String): Either[JsonError, JsonAST] = ... + val asJson: ResponseAs[Either[JsonError, JsonAST], Nothing] = asString.map(parseJson) + + sttp + .response(asJson) + ... + +For some mapped response specifications available out-of-the-box, see :ref:`json support <json>`. + +Streaming +--------- + +If the backend used supports streaming (see :ref:`backends summary <backends_summary>`), it's possible to receive responses as a stream. This can be specified using the following method:: + + def asStream[S]: ResponseAs[S, S] = ResponseAsStream[S, S]() + +For example, when using the :ref:`akka-http backend <akkahttp>`:: + + import com.softwaremill.sttp._ + import com.softwaremill.sttp.akkahttp._ + + import akka.stream.scaladsl.Source + import akka.util.ByteString + + implicit val sttpBackend = AkkaHttpBackend() + + val response: Future[Response[Source[ByteString, Any]]] = + sttp + .post(uri"...") + .response(asStream[Source[ByteString, Any]]) + .send() + +.. note:: + + Unlike with non-streaming response handlers, each streaming response should be entirely consumed by client code. + diff --git a/docs/watch.sh b/docs/watch.sh new file mode 100755 index 0000000..24c4372 --- /dev/null +++ b/docs/watch.sh @@ -0,0 +1,2 @@ +#!/bin/bash +sphinx-autobuild . _build/html diff --git a/circe/src/main/scala/com/softwaremill/sttp/circe/package.scala b/json/circe/src/main/scala/com/softwaremill/sttp/circe/package.scala index 77a926d..ddc9583 100644 --- a/circe/src/main/scala/com/softwaremill/sttp/circe/package.scala +++ b/json/circe/src/main/scala/com/softwaremill/sttp/circe/package.scala @@ -4,7 +4,6 @@ import io.circe.parser._ import io.circe.{Decoder, Encoder} package object circe { - private[sttp] val ApplicationJsonContentType = "application/json" implicit def circeBodySerializer[B]( implicit encoder: Encoder[B]): BodySerializer[B] = diff --git a/circe/src/test/scala/com/softwaremill/sttp/CirceTests.scala b/json/circe/src/test/scala/com/softwaremill/sttp/CirceTests.scala index 1e7dddc..1e7dddc 100644 --- a/circe/src/test/scala/com/softwaremill/sttp/CirceTests.scala +++ b/json/circe/src/test/scala/com/softwaremill/sttp/CirceTests.scala diff --git a/json/json4s/src/main/scala/com/softwaremill/sttp/json4s/package.scala b/json/json4s/src/main/scala/com/softwaremill/sttp/json4s/package.scala new file mode 100644 index 0000000..4c7aa36 --- /dev/null +++ b/json/json4s/src/main/scala/com/softwaremill/sttp/json4s/package.scala @@ -0,0 +1,14 @@ +package com.softwaremill.sttp + +import org.json4s._ +import org.json4s.native.Serialization.{read, write} + +package object json4s { + implicit def json4sBodySerializer[B <: AnyRef]( + implicit formats: Formats = DefaultFormats): BodySerializer[B] = + b => StringBody(write(b), Utf8, Some(ApplicationJsonContentType)) + + def asJson[B: Manifest]( + implicit formats: Formats = DefaultFormats): ResponseAs[B, Nothing] = + asString(Utf8).map(s => read[B](s)) +} diff --git a/json/json4s/src/test/scala/com/softwaremill/sttp/Json4sTests.scala b/json/json4s/src/test/scala/com/softwaremill/sttp/Json4sTests.scala new file mode 100644 index 0000000..bb4a774 --- /dev/null +++ b/json/json4s/src/test/scala/com/softwaremill/sttp/Json4sTests.scala @@ -0,0 +1,75 @@ +package com.softwaremill.sttp + +import org.json4s.ParserUtil.ParseException +import org.scalatest._ + +import scala.language.higherKinds + +class Json4sTests extends FlatSpec with Matchers with EitherValues { + import json4s._ + import Json4sTests._ + + "The json4s module" should "encode arbitrary json bodies" in { + val body = Outer(Inner(42, true, "horses"), "cats") + val expected = """{"foo":{"a":42,"b":true,"c":"horses"},"bar":"cats"}""" + + val req = sttp.body(body) + + extractBody(req) shouldBe expected + } + + it should "decode arbitrary bodies" in { + val body = """{"foo":{"a":42,"b":true,"c":"horses"},"bar":"cats"}""" + val expected = Outer(Inner(42, true, "horses"), "cats") + + val responseAs = asJson[Outer] + + runJsonResponseAs(responseAs)(body) shouldBe expected + } + + it should "fail to decode invalid json" in { + val body = """not valid json""" + + val responseAs = asJson[Outer] + + an[ParseException] should be thrownBy runJsonResponseAs(responseAs)(body) + } + + it should "set the content type" in { + val body = Outer(Inner(42, true, "horses"), "cats") + val req = sttp.body(body) + + val ct = req.headers.toMap.get("Content-Type") + + ct shouldBe Some(contentTypeWithEncoding(ApplicationJsonContentType, Utf8)) + } + + def extractBody[A[_], B, C](request: RequestT[A, B, C]): String = + request.body match { + case StringBody(body, "utf-8", Some(ApplicationJsonContentType)) => + body + case wrongBody => + fail( + s"Request body does not serialize to correct StringBody: $wrongBody") + } + + def runJsonResponseAs[A](responseAs: ResponseAs[A, Nothing]): String => A = + responseAs match { + case responseAs: MappedResponseAs[_, A, Nothing] => + responseAs.raw match { + case ResponseAsString("utf-8") => + responseAs.g + case ResponseAsString(encoding) => + fail( + s"MappedResponseAs wraps a ResponseAsString with wrong encoding: $encoding") + case _ => + fail("MappedResponseAs does not wrap a ResponseAsString") + } + case _ => fail("ResponseAs is not a MappedResponseAs") + } +} + +object Json4sTests { + case class Inner(a: Int, b: Boolean, c: String) + case class Outer(foo: Inner, bar: String) +} diff --git a/notes.txt b/notes.txt new file mode 100644 index 0000000..e93a859 --- /dev/null +++ b/notes.txt @@ -0,0 +1,52 @@ +DONE + - body: bytes, input stream (?), task/future, stream (fs2/akka), form data, file + - set headers + - access uri/method/headers/cookies/body spec + - partial request (no uri + method) / full request + - start with an empty partial request + - make sure response is consumed - only fire request when we know what to do with response? + - reuse connections / connection pooling - in handler + - set cookies (set from response) + - auth + + We want to serialize to: + - string + - byte array + - input stream + - handler-specific stream of bytes/strings + +TODO + - add params: to query, modify URI? + - multi-part uploads + - proxy + - user agent, buffer size + - charset + - zipped encodings + - SSL - mutual? (client side) + - type-safe content/media type as in akk + - stream responses (sendStreamAndReceive?) / strict responses + +IDEAS + - handler restriction? AnyHandler <: Handler Restriction + + Options: + - timeouts (connection/read) + - follow redirect + - ignore SSL + + // + + post: + - data (bytes/is/string - but which encoding?) + - form data (kv pairs - application/x-www-form-urlencoded) + - multipart (files mixed with forms - multipart/form-data) + +Quick start with Ammonite +https://github.com/AsyncHttpClient/async-http-client backend + +https://stackoverflow.com/questions/8659808/how-does-http-file-upload-work +https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type +https://stackoverflow.com/questions/4007969/application-x-www-form-urlencoded-or-multipart-form-data/4073451#4073451 + + +http://www.skorks.com/2010/05/what-every-developer-should-know-about-urls/
\ No newline at end of file diff --git a/project/SttpRelease.scala b/project/SttpRelease.scala index be56c6a..62fc1a6 100644 --- a/project/SttpRelease.scala +++ b/project/SttpRelease.scala @@ -16,7 +16,7 @@ object SttpRelease { runClean, runTest, setReleaseVersion, - updateVersionInReadme, + updateVersionInDocs, commitReleaseVersion, tagRelease, publishArtifacts, @@ -27,7 +27,7 @@ object SttpRelease { ) // based on https://github.com/EECOLOR/sbt-release-custom-steps/blob/master/src/main/scala/org/qirx/sbtrelease/UpdateVersionInFiles.scala - private def updateVersionInReadme: ReleaseStep = { s: State => + private def updateVersionInDocs: ReleaseStep = { s: State => val readmeFile = file("README.md") val readme = IO.read(readmeFile) @@ -36,13 +36,31 @@ object SttpRelease { val releaseVersion = s.get(versions).get._1 - s.log.info(s"Replacing $currentVersionInReadme with $releaseVersion in ${readmeFile.name}") + val settings = Project.extract(s) + val vcs = settings.get(releaseVcs).get - val newReadme = readme.replaceAll(Pattern.quote(currentVersionInReadme), releaseVersion) - IO.write(readmeFile, newReadme) + def replaceInFile(f: File): Unit = { + s.log.info(s"Replacing $currentVersionInReadme with $releaseVersion in ${f.name}") - val settings = Project.extract(s) - settings.get(releaseVcs).get.add(readmeFile.getAbsolutePath) !! s.log + val oldFile = IO.read(f) + val newFile = oldFile.replaceAll(Pattern.quote(currentVersionInReadme), releaseVersion) + IO.write(f, newFile) + + vcs.add(f.getAbsolutePath) !! s.log + } + + def replaceRstInDirectory(d: File) { + Option(d.listFiles()).foreach(_.foreach { f => + if (f.isDirectory) { + replaceRstInDirectory(f) + } else if (f.getName.endsWith(".rst")) { + replaceInFile(f) + } + }) + } + + replaceInFile(readmeFile) + replaceRstInDirectory(file("docs")) s } diff --git a/version.sbt b/version.sbt index 9ee7d50..dc11afe 100644 --- a/version.sbt +++ b/version.sbt @@ -1 +1 @@ -version in ThisBuild := "0.0.21-SNAPSHOT" +version in ThisBuild := "1.0.1-SNAPSHOT" |