How to Create a Spray Custom Authenticator

Spray has a few standard authenticators already built in. However, what if the authentication we need is not supported? In this article we will describe how to create a custom authenticator in Spray. In particular, we will implement a custom client id/secret token-based authentication.

Spray Authentication: how it works

Spray has already some built in authentication systems. For example, it supports HTTP Basic Authentication: have a look at their documentation here. What if the authentication logic that we need is not supported? Luckily, Spray allows the implementation of custom authenticators.

The authenticate directive has two signatures:

  def authenticate[T](auth: ⇒ Future[Authentication[T]])(implicit executor: ExecutionContext): Directive1[T]
  def authenticate[T](auth: ContextAuthenticator[T])(implicit executor: ExecutionContext): Directive1[T]

where ContextAuthenticator and Authentication are aliases for the following types:

// from spray.routing.authentication package object
...
  type Authentication[T] = Either[Rejection, T]
  type ContextAuthenticator[T] = RequestContext ⇒ Future[Authentication[T]]
...

In other words, a ContextAuthenticator is a function that takes a RequestContext (i.e.: a wrapper that contains all the information about the received request) and it uses it to either authenticate or reject the request.

Cient Id/Secret Custom Authenticator

Our goal is to create a custom authenticator that looks at the query parameters of a request, it extracts some client_id and client_secret and it authorizes the request if they are correct.

For the purposes of this tutorial we have decided to focusing on the second signature of the authenticate directive, so we will define a ContextAuthenticator to use with the directive.

First, let’s create a data container for our credentials and an auxiliary method to extract the id/secret from the request context:

  case class Credentials(id: String, secret: String)

  private def extractCredentials(ctx: RequestContext): Option[Credentials] = {
    val queryParams = ctx.request.uri.query.toMap
    for {
      id <- queryParams.get("client_id")
      secret <- queryParams.get("client_secret")
    } yield Credentials(id, secret)
  }

We can now define our ContextAuthenticator as following, keeping in mind that Authentication[T] is just a type alias for Either[Rejection, T]:

 val authenticator: ContextAuthenticator[Unit] = { ctx =>
      Future {
        val maybeCredentials = extractCredentials(ctx)
          maybeCredentials.fold[authentication.Authentication[Unit]](
            Left(AuthenticationFailedRejection(CredentialsMissing, List()))
          )( credentials =>
              credentials match {
                case AppCredentials("my-client-id", "my-client-super-secret") => Right()
                case _ => Left(AuthenticationFailedRejection(CredentialsRejected, List()))
              }
          )
      }
  }

If any of the authentication query parameters are missing, the following response is returned:

Status: 401 Unauthorized
Body: The resource requires authentication, which was not supplied with the request

If the credentials are provided but they are not correct, the client will see the following message:

Status: 401 Unauthorized
Body: The supplied authentication is invalid

Finally, if the credentials are correct, the request will be authorized and satisfied if possible.

Note that in a more realistic case scenario, you would probably return a User entity instead of Unit. Moreover, rather than having the correct credentials hard-coded in the code, they should be either configurable or stored in a proper data storage.

We are now ready to use our custom authenticator!

For example, to make all our endpoints use our custom authenticator we could do something similar to the following:

 def routes: Route = 
    sealRoute {
        authenticate(authenticator) { authenticated =>
            firstResourceRoutes ~
              secondResourceRoutes
          }
      }
    }

Summary

Spray allows the implementation of custom authenticators. This article provides an example of how this feature can be used to implement a client id/secret token based authentication.

Static Duck Typing in Scala

Structural Types is a neat feature of Scala that not many developers know or use. This article will provide a brief introduction on what they are and how to use them.

What are Structural Types?

Types can be really powerful in Scala: they can define classes, abstract classes, objects, traits, functions…and a lot more!
What if we don’t really care what our instance is as long as it has a particular structure?

This problem is also called Duck Typing. The duck test by Mr Riley asserts that

When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck.

A Structural Type allows you to specify a desired set of properties and to define a type for them: the compiler will guarantee that the given type is compatible with the defined structure and reflection is used at runtime on the instance to call the method. It is important to mention that the use of reflection is quite expensive, so you should probably be careful when using it if performance is a big concern.

How to use them

Let’s simplify the duck test and let’s see how the problem can be easily solved using structural types.

For the sake of this tutorial we consider a duck any entity that can quack.

This requirement can be translated in the following structural type:

scala> type Duck = { def quack(): String }
defined type alias Duck

Every time we will use the type Duck the compiler will guarantee that the instance associated to it is compatible with the given structure:

scala> class Bird { def quack() = "quack"}
defined class Bird

scala> val d1: Duck = new Bird
d1: Duck = Bird@541a9917
// it works!

scala> class Bro { def quack() = "YOOOO QUAAAACKKK" }
defined class Bro

scala> val d2: Duck = new Bro
d2: Duck = Bro@7db1592b
// it works!

scala> class Person
defined class Person

scala> val d3: Duck = new Person
<console>:9: error: type mismatch;
 found   : Person
 required: Duck
    (which expands to)  AnyRef{def quack(): String}
       val d3: Duck = new Person
// it doesn't work because Person doesn't quack

Summary

In this article we have described how Scala implements Static Duck Typing and how it can be used to validate instances on their structure rather than type.

Performance Comparison between immutable Seq, List, Vector

I have recently attended the Advanced Scala Training Course by TypeSafe during the Scala Days Conference in Amsterdam. During the course we discussed a lot on how to write cleaner and more performant Scala code: one of the parameters that can greatly influence your performance is the type of collections used. Which type of collection shall we use? In this article we try to answer this question by comparing the runtime performance of three immutable Scala collections: Seq, List, Vector.

The Experiment

Our experiment consists in creating increasingly bigger collections of random integers and measure the average execution time of a specific operation. The (quick and dirty!) script used and the generated data can be found here. The script was run on a standard MacBook Pro with a 2.8 GHz Intel Core i5 processor. After each iteration, the number of allocated elements has been increased exponentially with base 2. Each operation has been performed 10 times and the average execution time has been considered for the purposes of this experiment.

My poor Mac managed to analyse up to collections with 2^27 elements before starting screaming the pain of hell — and that is when I decided to stop!

Although, this test cannot be considered valid for any statical significance due to the limited amount of retries and the fact that our collections have been limited to be of type Int, I believe that the results of our experiment are interesting enough to provide some guidance of what type of immutable Scala collection to use according to the feature of our system.

Apply

The analysed operation is the apply operation used to access a random object. By looking at thegraphs, we can see that List didn’t perform well. Lists don’t have randomised element access: every time we need to go through the list until we reach the element at the index we are looking for. Although both Seq and Vector behaved quite well, our winner for this round is Seq. None that the default implementation for Seq for Scala 2.11 is based on LinkedLists, while Vectors are implemented using a bit-map with a factor branching of 32 (a weird structure that can be seen as a tree with up to 32 children nodes).

Seq-applyList-applyVector-applyAll-apply

Append

For the append operation, the clear winner is Vector: because it has a tree structure that make really efficient to append elements. On the other side, List and Seq have a linked structure that makes this operation quite expensive.

Seq-appendList-appendVector-append

All-append

Prepend

List is unbeatable when prepending an element to the collection: all it has to do is add a new pointer and connect the new element with the head of existing list…easy! A Vector has quite a good performance, very similar to the append study case thanks to its tree-ish structure. One the other side, Seq has a disastrous results due to the fact that all the indexes need to be updated when a new element is added at the beginning of the collection.

Seq-prependList-prependVector-prependAll-prepend

Who’s the winner?

The results of our test suggests that unless our system requires an intensive use of specific operations like append or prepend, we should avoid list and sequences in favour of vectors as they have an overall better performance. Note that this is particularly true when performing operations on big collections: when dealing with small ones (i.e. less than 10 elements), there no significant performance difference between one collection type and the other.  This is consistent with the Collection Performance Characteristics described in the Scala Documentation.

Seq List Vector

Summary

Choosing the correct type of Scala collection can have a big impact on the performance of our code. This article has analysed the results of an experiment where the performance of Seq, List, Vector have been compared when accessing, appending and prepending an element. Our experiments are consistent with the Scala Documentation and suggest that Vectors are the collection with the overall better performance.

Pimp My Library

Methods are an efficient way of reducing code duplication and make our code cleaner. What happens if a class that you don’t own (i.e.: any class of the Standard Scala Library) doesn’t have a particular method that could make your life a lot easier — and your code a lot more readable? This article will describe how we can efficiently pimp an existing Scala library and how to seamlessly use it in our code.

How to Extend an Existing Class

Let’s assume that in our application we often need to complete a text with the string “Yo”.
We could write a nice method for it and import it when needed, but that does not make its usage look exactly the same as the other standard methods of the String class. Instead, by “pimping” our class, we will be able to use our method as it was actually part of to the standard implementation of String.

Creating the following class will do the trick:

// in file com/daniela/sfregola/tutorial/package.scala
package com.daniela.sfregola

package object tutorial {

  implicit class ExtendedString(val text: String) extends AnyVal {
    def yofy = s"Yo $text"
  }
}

We can now use our yofy method for any String used in a class in the package com.daniela.sfregola.tutorial:

package com.daniela.sfregola.tutorial

object Main extends App {
	println("bro".yofy)
	//"Yo bro"
}

These are just a few lines of code, but they use quite some interesting and powerful tools of the Scala Language.
First of all, ExtendedString is inside a package object, called tutorial: the class will be automatically imported in all the files that belong to package com.daniela.sfregola.tutorial. For more information on package objects, have a look at this article.
Also, the class is implicit: this allows the compiler to seamlessly wrap an instant of a String inside ExtendedString.
Finally, we can see that our class is a subclass of AnyVal: this is a functionality introduced from Scala 2.10, called Custom Value Class: in practical terms, it makes our code a lot faster following some optimisation from the compiler.

Custom Value Classes

If we play a bit with the :javap command in the scala console we can see how the compiler disassembles our code. If we do this for a Custom Value Classes (i.e. a class that extends AnyVal) we can notice that, instead of allocating an instance for that class type, the compiler will just allocate a java.lang.String: this little trick makes our code a lot more performant as it will avoid the allocation of runtime objects…magic indeed!

Ok, so why don’t we use extends AnyVal everywhere?

The compiler translates our instance into a java.lang.String, so it could struggle if serialising/deserialising it. This approach is usually suggested only when pimping libraries: when extending an existing class, usually we are not actually creating new classes but just adding methods by wrapping an instance of the original class.

Summary

Pimping libraries is a powerful tool to enrich existing libraries. In this article we have described how to efficiently add methods to existing classes. Also, we have briefly described the principle used by the compiler to perform runtime optimisation using Custom Value Classes.

Loading Configurations in Scala

The separation of configuration from code is a good practice that makes our system customisable as we can load different configurations according to the environment we are running it in. In this article we will describe different approaches to load configurations in Scala and how they can be combined together: loading configurations from a file, from command line parameters or from environment variables.

Configurations from a file

Let’s start with the basic case scenario: given a file, we want to read it and parse its values to use them in our code.

First, we need to define our configuration file, let’s call it application.conf.

// application.conf
my {
	secret {
		value = "super-secret"	
	}
}

We can now parse the file and use the obtained configuration in our script:

// config-tutorial.scala
import com.typesafe.config.ConfigFactory

val value = ConfigFactory.load().getString("my.secret.value")
println(s"My secret value is $value")
>> scala config-tutorial.scala 
My secret value is super-secret

By default, the ConfigFactory looks for a configuration file called application.conf. If willing to use a different configuration file (e.g.: another.conf), we just need to indicate a different file name and path to load (e.g.: ConfigFactory.load("another")). The Typesafe Config library provides several methods to make sure that the parsed value is compatible with the expected type: have a look at the Config Typesafe Documentation for methods to parse integers, longs, floats, etc.

Configurations from command line parameters

Another approach is to allow our users to redefine settings through command line parameters rather than changing the configuration file directly. All we have to do is changing our configuration file as following:

// application.conf
my {
	secret {
		value = "super-secret"
		value = ${?VALUE}	
	}
}

The output of our script will now change accordingly to the command line parameters provided.

>> scala config-tutorial.scala 
My secret value is super-secret

>> scala config-tutorial.scala -Dmy.secret.value=another-secret
My secret value is another-secret

Configurations from environment variables

Redefining configurations as part of the command line parameters works in most of the cases, but it can be tedious when we have a lot of parameters to change. Also putting sensitive information, such as passwords or tokens, in clear text in a configuration file or a run script may not be safe enough. Another option to load configurations is to inject our parameters from predefined environment variables.

In order to achieve this, we can just write a simple method that looks for a specific environment variable before loading the configurations in the previously described approach.

import scala.util.Properties

def envOrElseConfig(name: String): String = {
    Properties.envOrElse(
      name.toUpperCase.replaceAll("""\.""", "_"),
      config.getString(name)
    )
}	

Before loading our my.secret.value configuration, this simple method will first check if an environment variable called MY_SECRET_VALUE exists.

We can now put all together and create a script (gist available here) that will inject configurations in the following order:
1) From properly named environment variables
2) From command line parameters
3) From a configuration file

// application.conf
my {
	secret {
		value = "super-secret"
		value = ${?VALUE}	
	}
}
import com.typesafe.config.ConfigFactory
import scala.util.Properties

class MyConfig(fileNameOption: Option[String] = None) {
    
  val config = fileNameOption.fold(
                  ifEmpty = ConfigFactory.load() )(
                  file => ConfigFactory.load(file) )

  def envOrElseConfig(name: String): String = {
    Properties.envOrElse(
      name.toUpperCase.replaceAll("""\.""", "_"),
      config.getString(name)
    )
  }
}

The script can be used as following:

val myConfig = new MyConfig()
val value = myConfig.envOrElseConfig("my.secret.value")
println(s"My secret value is $value")

[/code]

Summary

Having a clear separation between configurations and code allow us to customise its execution to the environment where it runs in. In this article we have described different approaches of defining specific settings. In particular, we have presented a simple script that combines all these approaches in one: the script loads configurations first from environment variables, then from command line parameters and finally from a configuration file.