Circe Introduction

5 minute read

In the micro service, we always use RESTFul API as communication protocol which pass data using JSON format. So the mapping between JSON and Data Model is an essential work for every system.

In this blog, we will see how Scala use Circe library to do this type of work.

What is Circe?

Circe is a functional JSON library for Scala, its initial name is jfc which means JSON for cats. From this name, we can know it is based on the cats library.

In this Issue, the author talked about why he change the name to Circe.

The motivation of Circe is from Argonaut and make some important changes.

How to install?

The latest version is 0.12.3, add the following code to build.sbt

libraryDependencies ++= Seq(
  "io.circe" %% "circe-core" % "0.12.3",
  "io.circe" %% "circe-generic" % "0.12.3",
  "io.circe" %% "circe-parser" % "0.12.3"
)

circe-core defines the core data type and type classes of Circe
circe-generic use Shapeless to auto-generate Decoder/Encoder for data model(case class).
circe-parser defines some implementation of Parser type class to give an entry of decoding JSON.

What we can find in this library?

The workflow of Circe looks like this

Let’s give a high level overview of this library.

Data Type

To process JSON flexibly and easily, Circe defines a data type called Json to describe any JSON string.

To convert the Json to any other data model, Circe defines a data type Cursor to get the value of given key.

You may notice the Cursor has two methods to get the expected type of value from given key or the current Json

  final def as[A](implicit d: Decoder[A]): Decoder.Result[A] = d.tryDecode(this)
  final def get[A](k: String)(implicit d: Decoder[A]): Decoder.Result[A] = downField(k).as[A]

These are the most popular methods we will use in Circe. And there are lots of other methods which can move the current cursor(use new Json to construct a new cursor) to help us to do operation on expected Json, such as downField, downArray.

Type Classes

Decoder

trait Decoder[A] {
  def apply(c: HCursor): Decoder.Result[A]
}

By constructing a Decoder instance, we can tell Cursor how to convert the current Json to given data model.

Parser

trait Parser {
  def parse(input: String): Either[ParsingFailure, Json]
  def decode[A: Decoder](input: String): Either[Error, A]
}

Parser is used to convert String to Json and defined in circe-core, its implementation is defined in circe-parser package which use jawn to do this work.

Encoder

trait Encoder[A] {
  def apply(a: A): Json
}

implicit class EncoderOps[A](val wrappedEncodeable: A) {
  def asJson(implicit encoder: Encoder[A]): Json = encoder(wrappedEncodeable)
  def asJsonObject(implicit encoder: ObjectEncoder[A]): JsonObject =
    encoder.encodeObject(wrappedEncodeable)
}

To implement Encoder easily, Json supply some methods such as Json.obj, Json.arr to help us.

For the type A which already has an Encoder instance, we can just use a.asJson to convert it to Json

How to decode JSON to data model?

Say we have a raw JSON string like this

{
  "id": "1100101010101",
  "person": {
    "name": "Job",
    "age": 18
  }
}

And we want to convert it to the model IdCard

case class Person(name: String, age: Int)
case class IdCard(id: String, person: Person)

How should we do?

Define Decoder by ourself

To convert Json to IdCard, we need to define a Decoder instance for IdCard

implicit val idCardDecoder: Decoder[IdCard] = new Decoder[IdCard] {
  override def apply(c: HCursor): Decoder.Result[IdCard] = for {
    id <- c.get[String]("id")
    name <- c.downFeild("person").get[String]("name")
    age <- c.downFeild("person").get[Int]("name")
  } yield IdCard(id, Person(name, age))
}

You may notice

We use c.get[A] to get the value of given key under current level
We use c.downFeild(<key>) to go to the next level with given key, and use c.get[A] again to get the value at that level.

Then we can use the Decoder instance like this

import io.circe.parser.decode
import io.circe.Error

val jsonStr: String = ???
val idCard:Either[Error, IdCard] =  decode[IdCard](jsonStr)

Use the existing Decoder in another Decoder

You may notice, when we get the name and age, we need to do c.downFeild("person"). Actually we can remove this operation by define a separated Decoder instance for Person and then reuse it in the idCardDecoder

implicit val personDecoder: Decoder[Person] = new Decoder[Person] {
  override def apply(c: HCursor): Decoder.Result[Person] = for {
    name <- c.get[String]("name")
    age <- c.get[Int]("age")
  } yield Person(name, age)
}

implicit val idCardDecoder: Decoder[IdCard] = new Decoder[IdCard] {
  override def apply(c: HCursor): Decoder.Result[IdCard] = for {
    id <- c.get[String]("id")
    person <- c.get[Person]("person")
  } yield IdCard(id, person)
}

Use auto-generated Decoder

Actually the above definitions of Decoder have lots of boilerplate, we can definitely generate it automatically, just like this

import io.circe.generic.auto._
import io.circe.parser.decode
import io.circe.Error
val jsonStr: String = ???
val idCard:Either[Error, IdCard] =  decode[IdCard](jsonStr)

We just need to import io.circe.generic.auto._, the macro in this package will help us to generate Decoder. But there is also restriction which require the key of attribute in JSON have the same name with the attribute of data model.

Use Decoder generated from other Decoder

The Decoder is a Monad, so we can do map, flatMap on it.

Say we have another data model

case class UniqueIdentity(id: String)

And we want the id to be the join of IdCard.id, Person.name, Person.age, we can decode the JSON directly to this model like this

import io.circe.generic.auto._
import io.circe.parser.decode
import io.circe.Error
val jsonStr: String = ???
implicit val uniqIdentityDecoder: Decoder[UniqueIdentity] = Decoder[IdCard].map(x => s"${x.id}-${x.person.name}-${x.person.age}")
val uniqIdentity:Either[Error, UniqueIdentity] =  decode[UniqueIdentity](jsonStr)

How to encode data model to JSON?

Now we want to do the reverse thing, convert the data model to JSON. First we need to convert the data model to Json, then Json has some method to convert itself to String, such as noSpaces, space2, space4. So we will just focus on how to convert data model to Json here.

Define Encoder by ourself

We can define Encoder instance like this

implicit val idCardEncoder: Encoder[IdCard] = new Encoder[IdCard] {
  override def apply(a: IdCard): Json = {
    Json.obj(
      "id" -> a.id.asJson
      "person" -> Json.obj(
         "name" -> a.person.name.asJson
         "age" -> a.person.age.asJson
       )
    )
  }
}

Then we can use it like this

import io.circe.syntax._
val idCard: IdCard = ???
val idCardJson:Json = idCard.asJson

Please note we are using Json.obj to construct Json.

Use the existing Encoder in another Encoder

Just like the Decoder, we also can define an Encoder instance for Person to make the Encoder instance of IdCard simpler

implicit val personEncoder: Encoder[Person] = new Encoder[Person] {
  override def apply(a: Person): Json = {
    Json.obj(
      "name" -> a.name.asJson
      "age" -> a.age.asJson
    )
  }
}
implicit val idCardEncoder: Encoder[IdCard] = new Encoder[IdCard] {
  override def apply(a: IdCard): Json = {
  Json.obj(
    "id" -> a.id.asJson
    "person" -> a.person.asJson
  )}
}

Use auto-generated Encoder

This is obvious, we can generate Encoder automatically, it has the same restriction as Decoder.

import io.circe.generic.auto._
import io.circe.syntax._
val idCard: IdCard = ???
val idCardJson:Json = idCard.asJson

Use Encoder generated from other Encoder

Encoder has a contrmap method which allow you generate a new Encoder from the existing one.

import io.circe.generic.auto._
import io.circe.syntax._
val uniqIdentity: UniqueIdentity = ???
implicit val uniqIdentityEncoder: Encoder[UniqueIdentity] = Encoder[IdCard].contrmap(x: UniqueIdentity => {
  val info = x.split("-") 
  IdCard(info(0), Person(info(1), info(2).toInt))
})
val uniqIdentityJson:Json = uniqIdentity.asJson 

Twitter Facebook LinkedIn

Jiaming Shang