I recently implemented SCRAM (SHA-256) over SASL for SkySpark v3 so I could use the REST API.

SkySpark v3 sports a new authentication algorithm that must be used to communicate with it. But alas there is not much available that succinctly explains how it works.

What I did find though, was a bucket load of specifications and documentation that were as cryptic as the authentication mechanism itself!

So now I present to you, what would have been extremely helpful to me, a fully worked authentication conversation with SkySpark for SCRAM (SHA-256) over SASL.

Random Stock Photo!

Overview

All the authentication documentation for SkySpark v3 talks of SCRAM SHA-256, or Salted Challenge Response Authentication Mechanism (SCRAM) with SHA-256. SCRAM defines how to encode an authentication message to send to the server. It uses the PBKDF2 algorithm from the Public-Key Cryptography Standards (PKCS).

Simple Authentication and Security Layer (SASL) then defines a protocol of how to send / receive these authentication messages.

These SASL messages are then wrapped up in HTTP headers and the HTTP request is sent as usual.

If you want the low down on all of the above, here are links to the relevant parts of the specifications:

The Project Haystack and SkySpark documentation do have a sample client / server conversation (the SASL part), but they make no attempt to explain how they calculated the values and the given messages (the SCRAM part)!

Steps:

  1. Hello!
  2. First Message
  3. Second Message
  4. Rest
  5. Fantom Code

The following example conversation assumes the following values, which are the same as those used in Project Haystack's authentication documentation:

  • Username: user
  • Password: pencil

Note that all example code listed here is written in Fantom.

1. Hello!

The first request is to initiate the authentication conversation, sending the username we wish to authenticate as.

Note that the URL we authenticate against needs to be a real URL lest we get a 404. It also needs to be one that we're not going to be redirected from. Project specific URLs are fine, but for general purpose usage I find that /ui works well in all scenarios.

Assuming SkySpark is running locally on port 8080 our first HTTP request looks like:

// Client Request: Hello
GET /ui HTTP/1.1
Host: localhost:8080
Authorization: HELLO username=dXNlcg

The username is just our username user Base64 encoded. But note the lack of trailing = characters that are usually used for padding. That's because the Authorization header uses Base64 URIs, which lack the padding.

// Server Response: Hello
HTTP/1.1 401 Unauthorized
WWW-Authenticate: scram handshakeToken=dXNlcg, hash=SHA-256

SkySpark replies with the above, which tells us we're to use SHA-256 for all our encoding. In general Project Haystack mechanisms this could also be SHA-1 or SHA-512.

The handshakeToken is used by the server to keep track of the authentication conversation, similar to a session cookie. So we need to make sure we pass the same value back. (Just ignore the fact it looks like our encoded username - this may change!)

2. First Message

Send an authentication request to the server.

In the request we name the user we wish to authenticate as, and a nonce. The nonce is random sequence of characters and should be of cryptographic strength.

clientNonce    := ... crypto strength random characters ...
// --> fyko+d2lbbFgONRv9qkxdawL

clientFirstMsg := "n=${userName},r=${clientNonce}"
// --> n=user,r=fyko+d2lbbFgONRv9qkxdawL

The first message is then Base64 URI encoded (no trailing = chars) and sent to the server.

// Client Request: 1st Message
GET /ui HTTP/1.1
Host: localhost:8080
Authorization: SCRAM handshakeToken=dXNlcg,
  data=bj11c2VyLHI9ZnlrbytkMmxiYkZnT05Sdjlxa3hkYXdM

SkySpark would then respond with the following response:

// Server Response: 1st Message
HTTP/1.1 401 Unauthorized
WWW-Authenticate: scram handshakeToken=dXNlcg, hash=SHA-256,
  data=cj1meWtvK2QybGJiRmdPTlJ2OXFreGRhd0xIbytWZ2s3cXZVT0tVd3VXTElXZzRsLzlTcmFHTUhFRSxzPXJROVpZM01udEJldVAzRTFURFZDNHc9PSxpPTEwMDAw

If we Base64 decode the data attribute we get:

r=fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE,s=rQ9ZY3MntBeuP3E1TDVC4w==,i=10000

From which we can derive the following variables:

serverFirstMsg   := "r=fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE,s=rQ9ZY3MntBeuP3E1TDVC4w==,i=10000"
serverNonce      := "fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE"
serverSalt       := "rQ9ZY3MntBeuP3E1TDVC4w=="
serverIterations := 10000

3. Second Message

Send an encoded message that proves we have the password. This is hard part.

First we do the cryptographic stuff. Most languages have classes or libraries for doing this. Fantom uses Java's javax.crypto.spec.PBEKeySpec. Because we're using the SHA-256 hashing algorithm, we use PBKDF2WithHmacSHA256.

dkLen           := 32   // the size in bytes of a SHA-256 hash
saltedPassword  := Buf.pbk("PBKDF2WithHmacSHA256", password, Buf.fromBase64(serverSalt), serverIterations, dkLen)
// --> e3 72 cc 2f 65 c6 ac 00 51 e9 e8 de ef 4b ea a3 f0 72 08 a4 6e b2 9f a3 bf 68 ea 76 9e b8 3e e9

dkLen is constant, but is dependent on which hashing algorithm we're using.

// dkLen Values
SHA-1   : 20
SHA-256 : 32
SHA-512 : 64

In the next string, clientFinalNoPf (client final no proof), note that biws is a constant and is the just the string "n,," Base64 encoded. In some literature this is referred to as the GS2 Header.

The string "Client Key" is also constant and is used as a default message to be hashed by the (salted) password.

select all
clientFinalNoPf := "c=biws,r=${serverNonce}"
// --> "c=biws,r=fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE"

authMessage     := "${clientFirstMsg},${serverFirstMsg},${clientFinalNoPf}"
// --> "n=user,r=fyko+d2lbbFgONRv9qkxdawL,r=fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE,s=rQ9ZY3MntBeuP3E1TDVC4w==,i=10000,c=biws,r=fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE"

clientKey       := "Client Key".toBuf.hmac("SHA-256", saltedPassword)
// --> 26 ac fd 4f 40 f9 5e 8e 74 b2 b3 5f 88 cd 8b e4 35 da 89 db 8d ab cc a8 b9 fb de 34 49 f9 93 b8

storedKey       := clientKey.toDigest("SHA-256")
// --> b6 2f 2a 50 c9 9e 42 27 46 85 5e 9a 60 fa 3c 71 39 f8 78 9a 70 60 46 19 4d ae 5c e8 cf 48 e5 37

clientSignature := authMessage.toBuf.hmac("SHA-256", storedKey)
// --> 5b 60 ae 4a 75 d8 da 9c 05 3b 85 ef 36 b6 27 df 2c 0a c9 4c f1 65 8f cf 3d 00 e1 1e ee d6 e1 4c

clientProof     := xor(clientKey, clientSignature)
// --> 7d cc 53 05 35 21 84 12 71 89 36 b0 be 7b ac 3b 19 d0 40 97 7c ce 43 67 84 fb 3f 2a a7 2f 72 f4

clientFinal     := "${clientFinalNoPf},p=${clientProof.toBase64}"
// --> "c=biws,r=fyko+d2lbbFgONRv9qkxdawLHo+Vgk7qvUOKUwuWLIWg4l/9SraGMHEE,p=fcxTBTUhhBJxiTawvnusOxnQQJd8zkNnhPs/KqcvcvQ="

The xor() method is an annoying little thing where each byte of each input have to be xored together. It is not a native Fantom function, but instead a method you have to write yourself. See Fantom Code for more details.

It is the clientFinal string that we send to the server in the next message. Like last time, we Base64 URI encode it and send it as the data attribute of the Authorization header:

// Client Request: 2nd Message
GET /ui HTTP/1.1
Host: localhost:8080
Authorization: SCRAM handshakeToken=dXNlcg,
  data=Yz1iaXdzLHI9ZnlrbytkMmxiYkZnT05Sdjlxa3hkYXdMSG8rVmdrN3F2VU9LVXd1V0xJV2c0bC85U3JhR01IRUUscD1mY3hUQlRVaGhCSnhpVGF3dm51c094blFRSmQ4emtObmhQcy9LcWN2Y3ZRPQ

SkySpark should then respond with:

// Server Response: 2nd Message
HTTP/1.1 200 Auth successful
Authentication-Info: authToken=xxxyyyzzz, hash=SHA-256,
  data=dj1UenFKVlc4bk5uZ1o5ZzFiL1lXaU84cy9abEhxQkwyb3AxYmxSN0txZG1FPQ

It is the authToken attribute that we're after. We can use in all subsequent HTTP requests to communicate with SkySpark.

But before we do; how do we know we're communicating with the correct server? How do we know our requests aren't be re-routed to some hackers server? Well now is our chance to validate the server and check that it also knows the user's password.

Base64 decoding the data attribute gives us:

v=TzqJVW8nNngZ9g1b/YWiO8s/ZlHqBL2op1blR7KqdmE=

If the server truly knows the user's password then we should be able to compute the same value for v.

Note the string "Server Key" is constant and is used as a default message to be hashed by the (salted) password.

serverKey       := "Server Key".toBuf.hmac("SHA-256", saltedPassword)
// --> 5a a1 fd ca 03 cb 46 42 45 ba 1b 94 67 a4 2c 9e 61 47 d6 da 9f cc c9 f2 bf 17 bc 4e ab 2c 1a 75

serverSignature := authMessage.toBuf.hmac("SHA-256", serverKey).toBase64
// --> "TzqJVW8nNngZ9g1b/YWiO8s/ZlHqBL2op1blR7KqdmE="

Because serverSignature equals the value sent by the server, we can trust it and continue communicating with it.

4. Rest

Now we have the authToken we can use it in all our requests to the SkySpark REST API.

// Client Request: 2nd Message
GET /api/demo/about HTTP/1.1
Host: localhost:8080
Authorization: BEARER authToken=xxxyyyzzz

5. Fantom Code

A complete reference implementation of the SCRAM protocol for retreiving an authToken from SkySpark, written in Fantom, is available in this BitBucket Snippet.

The sample code may be used like this:

// get the authToken
authToken := SkySparkAuth().scram(`http://localhost:8080/ui`, "<username>", "<password>")
echo("authToken: ${authToken}")

// call the REST API
zincRes := WebClient(`http://localhost:8080/api/<proj>/about`) {
    it.reqHeaders["Authorization"] = "BEARER authToken=${authToken}"
}.getStr

Note that the haystack-java library also contains a working SCRAM implementation.

Edits

  • 26 December 2016 - Original article.
  • 18 May 2017 - Made the link to the Fantom implemetation much clearer.
  • 26 May 2018 - Added notes on "Client Key" and "Server Key" being constant.
Written by Steve Eynon ( ) on 26th December 2016
Last edited on 26th May 2018
Tags: SkySpark, Security

Discuss