A JavaScript Unit Test Trap

Mark Adelsberger JavaScript, Technology Snapshot 3 Comments

You’re a few months into writing a shiny new web app, and the team just settled on some new standards for data structures used for communication with the server.

You’ll need to refactor a few pages, but that’s not a big deal. Your team has been pushing good unit testing practices, and you’ve got great coverage for the affected code. You make the changes, verify that the unit tests still pass, maybe do a little manual testing for good measure, and then get everything committed.

A few days later, the bug reports start rolling in. This column no longer shows the right data. When I run Test Case 5 the Widget Editor doesn’t render correctly. The sort order for data on the home page is wrong.

They’re all for pages you modified, and nobody else has worked on those pages. You know your refactoring is to blame; but the unit tests were still passing! What the heck happened…?

Where Do Mock Objects Come From?

In a strongly-typed language like Java, you probably rely heavily on a mock object framework. Through the magic of gclib the framework dynamically creates subclasses of your various services, repositories, etc. and serves up instances of those subclasses to replace each dependency used by your subject code.

In JavaScript you don’t need all that. In fact, you can’t have all that, because nowhere do you definitively describe a “class” for your objects. So your unit testing tool may provide some mocking support (like the “spy” functionality in Jasmine), but in general your unit tests can just build objects with whatever members they expect the subject code to access.

Do you see the trap yet?

An Example in Two Languages

Your app handles orders. An order has a list of line items (each with a part number, quantity, and unit cost), a shipping address, and a shipping fee. To simplify the example, all costs are in whole dollars.

In Java you define the following classes:

public class Order {
	private List<LineItem> lineItems;
	private Address shipTo;
	private int shippingFee;

	// getters and setters ...

public class LineItem {
	private string partNumber;
	private int quantity;
	private int unitCost;

	// getters and setters ...

You also have an OrderService with the following method:

public int calculateTotalCost(Order order) {
	int result = order.getShippingFee();
	for (LineItem lineItem : order.getLineItems()) {
		result += lineItem.quantity * lineItem.unitCost;

	return result;

To test the OrderService, you’ll want to pass in Order instances with known cost values and check that the correct total is returned. A pure approach would use mock objects with scripted responses for the getters, but for simple data objects with “logic-less” getters and setters, you might elect to create real Order objects instead.

(If the test needed to verify that specific Order methods were used along the way, we’d have to use a mock regardless; but we just want to test for the correct result.)

Order order = new Order();
List<LineItem> lineItems = new ArrayList<LineItem>();
LineItem item1 = new LineItem();


int result = orderService.calculateTotalCost(order);
assertEquals(121, result);

Meanwhile, another team is putting together a proof-of-concept for a more JavaScript-centric architecture. Your Order class gets JSON bindings; the above test Order, for example, would serialize as:

{ "lineItems": [
    "partNumber": "XL123"
  , "quantity": 2
  , "unitCost": 42
, "shipTo": { /*...*/ }
, "shippingFee": 37

They have an orderService.js containing a JavaScript version of calculateTotalCost():

var calculateTotalCost = function (order) {
	var result = order.shippingFee;
	for (var i = 0; i < order.lineItems.length; i++) {
		var lineItem = order.lineItems[i];
		result += lineItem.quantity * lineItem.unitCost;

	return result;

They unit test their JavaScript code with Jasmine. If they need to pass in a simple data object, they build it on the spot. (Alternately, they might include helper functions in their test scripts to build these sorts of object.)

describe("calculateTotalCost()", function () {
	it("gives the total cost including shipping", function () { 
		var order = {
			  orderItems: [{
				  quantity: 2
				, unitCost: 42
			, shippingFee: 37
		var result = calculateTotalCost(order);

Now while the proof-of-concept is being evaluated, this logic has to be maintained in two places; but both teams agree that the test coverage is good (meaning, presumably, that there are more tests than we’ve shown here), so nobody’s too worried.

The Trap Is Sprung

Mid-way through the proof of concept period, you get a new requirement: Shipping costs need to be attributed to specific line items. The JavaScript team is off site, but you figure they can catch up; worst case their unit tests will alert them that something’s broken, right?

The Java team agrees that to avoid potential inconsistency, the shippingFee property will be removed from Order. There was some talk about minimizing interface changes by keeping the Order.getShippingFee() method (rewriting it to sum up the line items’ shipping fee values), but everyone agrees that the order-level shipping fee should be removed from the JSON bindings in any case.

You update the Java classes. The unit tests don’t build since they reference nonexistent methods, so you fix them. Then you notice something weird…

The unit tests are passing. All of them. Even the JavaScript ones.

The problem is that the JavaScript unit tests are making the same wrong assumptions about the Order structure as the code they’re meant to test. In this case, it would likely get caught through even moderately good team communication; but in a bigger system, it’s possible some code would slip through the cracks.

And anyway, isn’t the point of unit tests to alert you to a defect even if you somehow miss it?

So Where Should Mock Objects Come From

There are two ways to improve on this situation. Ideally we’d like to remove assumptions about object structure from the unit test code. If we can’t do that, maybe we can figure out how to validate those assumptions.

In Java, our unit tests validate their assumptions about the structure of an order by trying to compile (and/or run) against the Order and LineItem classes. In JavaScript, though, an object doesn’t have a class.

I noted above that the unit test scripts could contain helper functions for building data objects. We could make it a rule that all data objects must be obtained from helpers, and that the helpers must all be in a central library shared by all test scripts. Then updating that central library should cause all affected tests to fail. This would tend to isolate the assumptions in the test code. It does have its limitations. Ultimately our test codebase still contains unverified assumptions about the object structure, and there’s no guarantee that each assumption can be isolated to a single helper function.

Another option is to create an Order constructor. Of course, when we receive a JSON representation of an Order, it won’t have gone through that constructor; so to make this meaningful, we have to use the received JSON to construct a “real” Order object that our code will operate on, and then throw the JSON representation away.

At a glance, this feels wasteful; but then at some point in history many of the conventions we use to promote testability and maintainability were received with the same criticism. As a side benefit, our constructed object could use closures to conceal the internal state and provide access through getters and setters, providing some of the encapsulation we had in the Java world.

When we first got the new requirement that changed our definition of an Order, we’d update the Order constructor unit tests accordingly, which would lead us to fix the Order constructor itself. Once the constructor is fixed, our unit tests would fail when trying to call setShippingFee() on the order, much like the Java case, and we’d have test failures pointing to each bit of code we need to fix.

Final Thoughts

This is likely not the only solution, and it may not be the best one; let us know how you’ve addressed this issue in the comments below.

Regardless, this should serve as a reminder that languages like JavaScript give us more than enough rope to hang ourselves. As the scale of your JavaScript projects increases, it becomes more important to self-impose some of the discipline that isn’t enforced by the language.

— Mark Adelsberger,

About the Author
Mark Adelsberger

Mark Adelsberger

Mark is a software developer with the Keyhole team living in St. Louis. In addition to expertise surrounding Java and JavaScript technologies, Mark has spent time focused on data/ETL in his 17+ years of experience in both “start-up” and Fortune 500 organizations.

Share this Post

Comments 3

  1. Zach Gardner

    “worst case their unit tests will alert them that something’s broken, right?”

    I’m not sure unit tests are the right place to expect backwards-incompatible API changes to be caught. Unit tests intrinsically should do no AJAX calls, so this change wouldn’t and shouldn’t be caught at this level.

    We’ve handled this same problem with two different mechanisms in projects I’ve been on before. They can be used together or independently, just depends on your budget.

    The first is for the API to use versioning. Any time the API adds a field (non-breaking change), do a minor version increment. Any time the API removes a field (breaking change) do a major version increment. This does mean the API team must do a good job of communicating with consumers of the API, so these teams should have strong leads that are aware of this category of change.

    The second is to implement integration tests. These tests by their nature can and should do AJAX calls. You can get into some neat BDD stuff with integration tests that are incorrect with unit tests.

    I think the idea of centralizing the objects to factories will help some, but I’ve seen API versioning and integration tests as more holistic ways to catch the species of issues you’ve described.

  2. Mark Adelsberger

    Interesting points, but I have to disagree on a couple details.

    It’s true that integration tests have an easier time spotting these problems, but whether this is an integration problem is a matter of interpretation. You could argue that the problem is in the interaction with the server, but I would argue that the client code as a unit unto itself is behaving incorrectly if it makes incorrect assumptions about its inputs. The bug doesn’t really depend on the fact that we happen to know we’ll get the input data via ajax; in fact if we’re writing reusable modules we don’t know if that will always be true.

    Pragmatically bugs caught by unit tests are likely caught more inexpensively.

    Note that none of the tests described above, including the variations that would catch the bug, make ajax calls; that’s the whole point of the mocking discussion. Mocks are supposed to accurately simulate inputs and dependencies, and if they do that then the test certainly can catch the problem without coupling to other parts of the system.

    It definitely is true that using API versions can get you more meaningful diagnostics (presumably during integration testing) instead of random unexpected behaviors if the bug goes uncaught. It’s a good practice, but I see it as separate from the question of early detection during testing.

    Ultimately I think we just disagree on how to classify a bug of this sort. I don’t think it’s a black-and-white issue, but I do believe this type of problem can and should be detected by a well-structured unit test.

Leave a Reply