A Day In The Lyf

…the lyf so short, the craft so longe to lerne

Archive for the ‘Testing’ Category

Stubbing a Mule TCP connector with mountebank

with one comment

My current project involves integration with a Mule service bus and a number of endpoints responding over a TCP connector.  We have limited control over the downstream service, but depend on it deeply, as our REST service is stateless.  

We like to QA our stories using black-box testing of our API, which means sending an HTTP request to our API and expecting it to make the relevant TCP service call.  For some time now, our biggest testing problem has been manufacturing test data for all scenarios.  Were it an HTTP dependency, stubbing would have been easy, but no TCP stubbing solution existed until recently.

Our most versatile approach was also quite kludgy.  We had created Endpoint classes that encapsulated the calls to all downstream dependencies, and had a parallel set of StubEndpoint classes.  When we deployed our API, we had the ability of specifying a different Spring profile that only wired up the StubEndpoint classes. This meant that our stubbing logic was directly housed in our production application.


This is obviously an inelegant solution for our main application, since it now contained a significant amount of code that should never be used in production. Lest you think that having a switch to turn on what should be dead code in production isn’t dangerous, perhaps you should read about Knight Capital, which lost $465 million due to just such a bug.

Perhaps less obviously, this was also a terrible solution for our tests. Let’s say that we were testing an API retrieving travel plans, and we needed to test several different scenarios – plans in the past, cancelled plans, overbooked plans, etc. We might pass a plan number into our API, and then have to code a massive if-else block in our StubEndpoint that switched on the the plan number to manufacture the various scenarios. We might, for example, send plan 12345 for a cancelled flight and plan 23456 for an overbooked plan. The data setup was completely opaque to the tests, and the cognitive distance between the test data setup and the test scenario opened the door for some difficult maintenance and ugly interactions between tests.

We made another attempt at stubbing using Byteman, which allows you to inject Java code into a running agent. We would have to spin up the real service alongside a Byteman agent for this to work:


This had a few big problems. First, it required us to actually stand up the real service, which added some tax to our development time when we wanted to isolation test our API. Second, it was very hard to use. Byteman takes a string written in a special syntax to inject Java code at a certain method in the remote application. Writing the string was fiddly at best. Finally, this approach deeply coupled our code to the remote service code. We had to know intimate details of the scope within which our injected code would run to correctly write the remote Java code. In the end, it proved too difficult to use.

mountebank made remote stubbing a breeze. Each test can now setup the test data specific to the scenario it represents. A class level tearDown or setUp can remove existing stub data before each test runs.


Each test now has full control to specify the result of the RPC call, and, unlike the Byteman approach, they can do so directly in Java code. Once we moved the mountebank interaction into a thin plumbing layer, the test code remained as clean as testing in-process with mockito.

Here’s an example test, which validates what happens through our API when we request a cancelled travel plan:

public void shouldSomethingWithCancelledTravelPlan() {
    // This is a much bigger object graph than represented here
    TravelPlan cancelledTravelPlan = new TravelPlan("1234", "2013-02-15").withStatus(CANCELLED); 

    // Now we tell mountebank to return the specified object when our app connects to the service port
    setupBinaryStub(cancelledTravelPlan, TRAVEL_PLAN_ENDPOINT_PORT);

    HttpResponse response = callOurAPIAt("http://localhost:8080/api/travelPlans/1234&date=2013-02-15");


    // This could be moved to a setUp or tearDown
    // I won't show the code, but it just sends an HTTP DELETE to http://localhost:2525/imposters/{port}

The actual stub setup is in the seupBinaryStub method. Let’s take a look at it:

private void setupBinaryStub(Object stub, int port) {
    HttpPost createStub = new HttpPost("http://localhost:2525/imposters");
    createStub.setHeader("Content-Type", "application/json");
    String json = "{\n" +
                "  \"protocol\": \"tcp\",\n" +
                "  \"port\": " + port + ",\n" +
                "  \"mode\": \"binary\",\n" +
                "  \"stubs\": [\n" +
                "    { \"responses\": [{ \"is\": { \"data\": \"" + encodedString(rawObject) + "\" } }] }\n" +
                "  ]\n" +

    createStub.setEntity(new StringEntity(buildJsonContent(expectation)));
    final HttpResponse response = new DefaultHttpClient().execute(createStub);

    if (response.getStatusLine().getStatusCode() != 201) {
        throw new RuntimeException("expected 201 status but was " + response.getStatusLine().getStatusCode());

The magic sauce is in the encodedString method. Here we have to figure out how our remote service serializes the object graph, and encode it as a base64 string. Figuring out the serialization mechanism can be the trickiest part of all of this. In our case, a short investigation into the code reveals it’s simply Java serialization:

private String encodedString(Object objectToWrite) throws IOException {
    byte[] data = SerializationUtils.serialize(objectToWrite);
    return new Base64().encodeAsString(stream.toByteArray());

Now we can manufacture any test scenario we want, and better yet, we can keep the test data setup scoped within each test without any fear of that data interfering with other tests.

Check it out:

Github repository

Heroku site


Written by Brandon Byars

February 10, 2014 at 5:00 am

Introducing httpmock

leave a comment »

node.js is simply too cool to ignore. I find the trick to learning any new cool framework is to find a new cool project to write, and use the framework to write it in. httpmock is my cool new project.

I tend to work in large’ish programs of work with lots of teams. One recurring pain point in such an environment is dealing with other teams. Sure, your code is fine, but what happens if you have to integrate with their code? These days, that tends to be through web services, and it’s never pretty. Your team writes integration tests, verify they work, and commit the changes. Three days later, for no apparent reason, your build goes red. It turns out that other team broke their code again.

httpmock is a network server stubbing tool. I say “network,” primarily because I’ve got my dreaming hat on. At the moment, it’s an HTTP stubbing tool, but I’d like to add other protocols, especially SMTP. httpmock exposes a RESTful API that lets you spin up HTTP servers that you can control, and that helpully record how they were used so you can make useful test assertions. In other words, httmock expects you to execute your code full-stack, including making a full TCP handshakes across the network. httpmock just lets you control both hands involved in the handshake, and it does so in a way transparent to your code; you just have to point your configuration to the stub servers httpmock creates. I find this quite useful for functional testing.

Don’t get me wrong. Integration tests are important, and you should still have a set of tests that exercise the actual system, but expect some fragility in those tests. If they break, it doesn’t necessarily mean you broke them. Therefore, they should be in a separate build, one that may not have the same “stop the line” implications when it goes red. httpmock should not be used only for builds that are entirely in control of your team..

When I say that httpmock exposes a RESTful API, I mean it’s a truly RESTful API. There’s only one published URL, and hypermedia drives the interaction from there. This makes it easy to evolve the server while maintaining a stable client-side API. At the moment, I have only a Java binding, but I have intentions for more (and I’d love some help on the issue!). The idea of the language bindings is to present to the developer a friendly API that approximates normal unit-testing mocking frameworks without having to write your tests in a language different from your production stack.


public void showVerificationAPI()
    StubServer stub = ControlServer.at("http://localhost:3000").setupPort(3001);
    SystemUnderTest yourCode = new SystemUnderTest();
    assertThat(stub, wasCalled("POST", "/endpoint")
                        .withHeader("Content-Type", "text/plain")
                        .withHeader("Accept", "text/plain")

The code above shows the Java assertions. The first line actually makes two network calls; one to talk to the well-known httpmock URL (in this case, http://localhost:3000) to get the hypermedia. The second one tells httpmock to set up a new server for stubbing purposes, which is then spun down on the last call in the method. For performance reasons, you may want to do some of these administrative tasks once for the test suite, particularly retrieving the initial hypermedia. The wasCalled method also talks to httpmock to perform the verification. The SystemUnderTest should expect to base its web service calls to http://localhost:3001.

public void showStubbingAPI()
    StubServer stub = ControlServer.at("http://localhost:3000").setupPort(3001);
    stub.on("GET", "/").returns(new Stub().withStatus(200).withBody("BODY"));
    SystemUnderTest yourCode = new SystemUnderTest();

Like typical stubbing libraries, httpmock allows you to set up stubs that will be executed if the server receives the expected call.


You need node.js to run the server. You can download httpmock from the incredibly ugly downloads page. Untar the server, and run bin/httpmock start.

This will run the control server on port 3000. You can optionally pass a --port command line argument to change the port and a --pidfile argument to use a non-default pidfile (useful if you’re running multiple instances). If you run the server in a background job, bin/httpmock stop will kill it.

If you’re using npm, you can install httpmock with sudo npm install -g httpmock. Installing it globally allows you to simply say httpmock start on the command line, which feels quite sexy.

The client bindings will simply be a library. For Java, add httpmock.jar to your classpath.


The following are the features I’d like to add to httpmock:

  • Support for multiple protocols, at least for verification if stubbing doesn’t make sense (SMTP, FTP, perhaps even things like printer protocols). It appears at the moment easy enough to support a plugin mechanism where the same REST API will work for multiple protocols, but I won’t know for sure until getting around to a second protocol.
  • Support for HTML. Right now, there is a JSON-based content type used for all communication, which is great for progammatic manipulation. I think it would also be quite nice to allow QA’s to manually set up stubs and perform verifications through a web browser.
  • A more robust stubbing mechanism. It’d be nice, for example, to only conditionally execute stubs if a specific request header was set.
  • More language bindings.

Want to contribute? I’d love the help. Send me a pull request at github.

Written by Brandon Byars

May 30, 2011 at 9:48 pm

Posted in Testing

Tagged with , , ,