Monday, June 27, 2016

Behavioral testing with Contract 0.0.6

A single stateless contract is never going to go very far in testing a HTTP system, which is why with version 0.0.6 of Contact, we're very pleased to announce 2 game changing new features to our framework.

A quick recap on Contract
Contract is an open source testing tool aimed at replacing a companion system in a HTTP client or server relationship so that shared specifications can be used to unit, integration or acceptance level test to ensure that both systems are on the same page. Server facing tests can be driven using a contract that specifies how the server is to react given a HTTP request. Client facing tests can be driven by an identical contract specifying that when a client makes a specific request, they should receive a specific response. With both sides of a client server relationship available for testing at a significantly earlier stage, we can improve the feedback of our testing cycles, and hopefully deliver value that little bit quicker.

Setup 
Being able to specify a single request response as part of a contract can only test so much of a system. Multiple requests are often required to get the system into a state where assertions can be made. For example, to test read functionality of a system, something must be there to be read. In 0.0.6 we've introduced a setup block to our contract. This allows for specification of any number of setup calls to be made to ensure the system is in the correct state before the contract is actually verified.

The JSON below is an example of the create then read contract applied to our key value example system.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
{
  "info" : {
    "details": "The server should return a key-value pair for a valid get request"
  },
  "setup": [
    {
      "request": {
        "method": "POST",
        "path": "/kv/pair",
        "headers": {
          "content-type": "application/json"
        },
        "body": {
          "key": "weight",
          "value": "220lbs"
        }
      },
      "response": {
        "status": 201,
        "body": {
          "key": "weight",
          "value": "220lbs"
        }
      }
    }
  ],
  "request" : {
    "method" : "GET",
    "path" : "/kv/pair/weight"
  },
  "response" : {
    "status" : 200,
    "body" : {
     "key": "weight",
      "value": "220lbs"
   }
  }
}

This allows us to specify the creation of a key value pair before verifying the read functionality of the system



Variables and state
We introduced wildcards a few releases ago, allowing for specification of values when we didn't know exactly what they were going to be. With this release we're going a step further.

A common use case for testing a HTTP system is creation of a resource and then verification that said resource exists. Up until now, if that resource was assigned a unique id at creation time, we had no way of referring to this id and using it later. We're introducing variables to provide this exact functionality. 

Below is an example similar to the one above, its a contract for the creation of an entity on a server representing an unsecured git repository. The repository will be assigned a unique id at creation time and the id will then be used to verify that the repository information can be read. 



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
{
  "info" : {
    "details": "The server should accept the details of an unsecured repo and store them"
  },
  "setup" : [
    {
      "request": {
        "method": "POST",
        "path": "/services/repositories",
        "body": {
          "name": "kvServerContracts",
          "url": "https://github.com/harmingcola/kvServerContracts.git"
        }
      },
      "response": {
        "status": 201,
        "body": {
          "id": "${contract.var.positiveNumber.repoId}",
          "name": "kvServerContracts",
          "url": "https://github.com/harmingcola/kvServerContracts.git"
        }
      }
    }
  ],
  "request": {
    "method": "GET",
    "path": "/services/repositories/${contract.var.positiveNumber.repoId}"
  },
  "response": {
    "status": 200,
    "body": {
      "id": "${contract.var.positiveNumber.repoId}",
      "name": "kvServerContracts",
      "url": "https://github.com/harmingcola/kvServerContracts.git"
    }
  }
}

For more information on our variables expressions, have a look at our documentation here. Contract is available from Maven Central

Contract source