CommaChamelon

Trailing Commas in JavaScript

by on June 10, 2013 10:00 am

The Scenario

You’re working on a JavaScript project, and your project includes the following object definition:

var deckOfCards = {
    deal : function() {
        // TODO: add dealing logic
    }
};

You go to add another method to your object.

var deckOfCards = {
    deal : function () {
        // TODO: add dealing logic
    }

    shuffle : function () {
        // TODO: add shuffling logic
    }
};

You return to your test page and get mysteriously unexpected behavior. What happened?

You’ve probably recognized the mistake in the above code, but then if you write much JavaScript, there’s a good chance you’ve made the same mistake numerous times, and you’ll probably make it again. Maybe it’s because of our experience with other languages, or maybe it’s just one of those small details, but we often forget to separate JavaScript object members with commas, especially when adding a new method to the end of a class.

So maybe you think to yourself, “You know, most browsers will tolerate a comma after the last member in the object.” You might train yourself to always put a comma after each member in an object, so that you won’t have this problem going forward. Then you would write:

var deckOfCards = {
    deal : function () {
        // TODO: add dealing logic
    },

    shuffle : function () {
        // TODO: add shuffling logic
    },
};

Then you might go on happily building your application, with its ever-growing code base, until one day your client says “Oh, by the way, it’s vitally important that this run in IE8.”

You think back on your observation that most browsers will accept a comma after the last member of a JavaScript object, only to reflect that IE8 is not in that particular majority. Now what?

Now you have a bunch of tiny, hard-to-find bugs scattered throughout a large code base, and you need to find and remove them. To make matters worse, they’re going to blend into the background. They look like valid syntax, especially if you’ve been training yourself to put them in the code on purpose. If I had a worse sense of humor, I’d call them comma chameleons. (Oh, look, I do have a worse sense of humor. Must be a byproduct of the long debugging sessions.)

Going forward, you might adopt a different convention in lieu of trailing commas. You could use the SQL convention of putting commas at the beginning of the new line, on the theory that you edit the end of a list more often than the beginning. Maybe you could always put a dummy element at the end of an object definition so that every “real” element can have a trailing comma with no issues. But right now, you just need a way to fix a mountain of existing code.

Our Solution

When our team encountered this problem, we felt that Perl might hold a solution. Indeed, a quick Google search reveals that others have fallen into the “trailing comma” trap, and some have provided RE strings that will find some of the trailing commas, some of the time. What follows is a somewhat more comprehensive script.

You can download the script here; if you’d like a step-by-step explanation of what the script will do, read on! (Note: the code as it appears below has been modified for readability; only the original version, as found via the previous link, has been tested.)

Before I get into the nuts and bolts, two caveats:

  • First, I said “somewhat more comprehensive.” What we’re looking for here is a good return on investment; it didn’t make sense to write a full-fledged JavaScript parser, which means some evil genius might find a way to hide a trailing comma that this script won’t catch. It should catch the cases that are likely to show up in real code.
  • Second, the error handling in this script is nearly nonexistent. Feel free to beef it up if you’d like.

With those caveats in mind, the script only finds the problems and tells you where they are; it does not attempt to repair the code for you.

Step 1: Find JavaScript files to process

Somehow we have to tell the script what file(s) to process. We could accept a filename on the command line and then rely on OS utilities to traverse our project tree and execute the script for each source file it locates. But directory traversal isn’t very hard in Perl anyway, so let’s allow our command line arguments to include directories to be recursively searched.

sub process
{
    my ($item) = @_;
    print “$item…\n”;

    # directory to be searched recursively?
    search($item) if –d $item;

    # …or file to be scanned?
    scan($item) if –f $item;
}

sub search
{
    my ($item) = @_;
    my $d, @f;

    # Get the list of files/directories in the directory
    opendir($d, $item);
    @f = readdir $d;
    closedir $d;

    # Process each one except the . and .. links
    foreach $f (@f) {
        next if $f eq “.” || $f eq “..”;
        proc(“$item/$f”);
    }
}

sub scan
{
    # TODO
}

# Functions declared; now process each command-line arg
foreach $nxt (@ARGV)
{
    proc($nxt);
}

Now the scan routine will be called for each file named on the command line, or found in a directory named on the command line (or a subdirectory thereof, recursively). So what should the scan function do?

Step 2: Strip Comments

One easy way to get a trailing comma is to comment out the last method in an object. We want our script to handle that case, and we suspect there might be other times when comments get in the way.

We expect to find two kinds of comment: single-line comments (//…) and block comments (/*…*/). Single-line comments alone would be pretty simple. Multi-line comments are somewhat more complex. The possible combinations of the two can get downright ugly. We’ll make some simplifying assumptions for now, so if your code is likely to say things like:

/* single line comments start with // */ ,

…then you may need to add some more robust parsing logic. At a minimum, we’ll handle cases without nested comment markers correctly.

We’ll declare $comment and set it to 0; this will be a boolean indicator of whether we’re in the middle of a multi-line comment; any time a line contains a /* with no matching */, we’ll set $comment to 1. Then we ignore subsequent lines until we find one that does contain a */.

For those lines we’re not ignoring, we just apply a little substring and/or regex logic to remove the comments that are present in the line. We don’t modify $_ itself because we want to preserve the original file contents for our output; so we use the $nocomment variable.

We start the body of the scan function like this:

my ($item) = @_;

my $comment=0;  #is this line part of a multi-line comment?
my $comma=0;  #did the last nonblank line end with a comma?
my $block;      #potential output
my $nocomment;  #input line with comments stripped out
my $f, $i;

open($f, “<$item”);
while(<$f>)
{
    $nocomment = $_;

   if ($comment) {
       #skip lines until we see an end-of-comment marker
       #then remove everything up to the marker (inclusive)
       $i = index $nocomment, '*/';
       next unless $i > -1;
       $nocomment = substr $nocomment, $i+2;
       $comment = 0;
   }

   # remove comments that are entirely on this line
   $nocomment =~ s/\/\/.*//;
   $nocomment =~ s/\/\*.*?\*\///g;

   # check if a comment spills over onto the next line
   $i = index $nocomment, '/*';
   if ($i > -1) {
       $comment = 1;
       $nocomment = substr $nocomment, 0, $i;
   }

   # TODO: detect trailing commas
}

close $f;

Note the importance of order here: We remove all instances of /*…*/ from the line, and after that is done if we still see a /* we know that a comment will carry over to subsequent lines.

Step 3: Detect Trailing Commas

We might think of a trailing comma as the string “,}” or “,]”. Of course, there could be whitespace (likely including newlines) between the comma and the closing brace. (There could be comments, too, but we’ve removed those.)

If we find a comma followed by a closing brace, we’ll output the line number and line contents. (We output the filename when we started processing the file.) If we find a comma at the end of a line, then we’ll scan subsequent lines until we know whether it is or is not a trailing comma. (The $comma variable will serve much the same purpose here as the $comment variable did for multi-line comments.)

So the last #TODO is replaced as follows:

if( $comma )
{
    # We’re resolving a , from the end of a previous line.
    # Add this line to the block of potential output.
    $block .= "  $.: $_";

    # If this isn’t a blank line, we’ll resolve the comma
    if ($nocomment !~ /^\s*$/)
    {
        $comma = 0;
    }
    # If the line starts with ] or }, it’s a trailing comma
    if ($nocomment =~ /^\s*[\]}]/)
    {
        print "$block\n";
    }
}

# look for trailing commas on this line
if( $nocomment =~ /,\s*[\]}]/ )
{
    print "  $.: $_\n" ;
}

# see if this line ends with a ,
if( $nocomment =~ /,\s*$/ )
{
    $comma = 1;
    $block = "  $.: $_";
}

And that’s should do it. Again, there are many ways to improve this script – add error handling, parse the JavaScript more thoroughly, etc. But this is a good “bang for your buck” implementation that shouldn’t produce false positives and will likely catch trailing commas unless someone’s intentionally trying to hide them.

In addition to after-the-fact bug hunting, a tool like this can be used proactively. For example, our team is considering putting a pre-commit hook in place so that code with trailing commas won’t make it into source control in the first place.

Happy Hunting!

— Mark Adelsberger, asktheteam@keyholesoftware.com

  • Share:

Leave a Reply

Things Twitter is Talking About
  • White paper - Why #Nodejs? Pros, cons & an approach for enterprise adoption. Free download: http://t.co/LdhNae94YT
    July 7, 2015 at 2:50 PM
  • There's a new blog from @ChrisShatrov - Becoming a Better Full-Stack Developer: Switching from Back to Front http://t.co/sYvZbWNdI6
    July 7, 2015 at 8:52 AM
  • .@Jinaljay transitioned to #Java from .NET & needed to switch IDEs. Tips that helped her in the switch to #eclipse - http://t.co/wbx9qFUAqp
    July 6, 2015 at 4:05 PM
  • Protip: Code For Maintainability So The Next Developer Doesn’t Hate You -http://t.co/G0gFOD7Pyj
    July 6, 2015 at 3:55 PM
  • Doing #Agile with a Distributed and/or Remote Team? No Problem! Tools and techniques for success - http://t.co/XrYJE8Xict
    July 6, 2015 at 12:35 PM
  • New to #JavaScript prototypal inheritance? Here are some notes to help you along the way - http://t.co/NTIDZS6Uhy
    July 5, 2015 at 7:55 AM
  • ICYMI: we've released a demo version of #GrokOla which is open to the public. Try out its features & capabilities - http://t.co/O4ladowmFU
    July 4, 2015 at 3:05 PM
  • Happy 4th of July from the Keyhole team! We hope that you have a happy and safe holiday with your family and friends.
    July 4, 2015 at 9:55 AM
  • Let's talk testing. Here are common challenges #Agile teams face when writing automated tests & how to overcome them: http://t.co/DrKbNZJcE0
    July 3, 2015 at 11:06 AM
  • #GrokOla users get free educational tutorials. But lucky you, we've released some to the public. #JavaScript primer - http://t.co/nIR9XiWY6O
    July 3, 2015 at 10:55 AM
  • Being able to isolate debugging techniques can help make you a better debugger. Here's Time-Oriented #Debugging http://t.co/UplJgP4VzC
    July 2, 2015 at 10:50 AM
  • RT @zachagardner: @zachagardner has declared it is @ChipotleTweets day at @KeyholeSoftware . You have been warned 🐓🐖🐄
    July 2, 2015 at 10:09 AM
  • Current state of random number generation & the differences in how #Java & #JavaScript approach it - http://t.co/5tBKNXnu8T #security
    July 1, 2015 at 2:45 PM
  • Woohoo - 600 followers! Thanks, everyone. We'd love to ask you - what type of tweets / dev content would you like to see more of from us?
    July 1, 2015 at 10:38 AM
  • We would like to welcome Dallas Monson to the team today! Dallas is a Senior Architect focused on UI/UX and #JavaScript. Welcome, Dallas!
    July 1, 2015 at 8:35 AM
  • Good introduction to TypeScript - http://t.co/0N22fVpAHt Plus, how to approach modularization in #TypeScript - http://t.co/wxRWGBj3Uh
    June 30, 2015 at 3:25 PM
  • .@mrbristopher just delivered a new S911 Night Drone to James Hayes, winner of our #kcdc15 giveaway! Congrats, James! http://t.co/RriJIxubH2
    June 30, 2015 at 11:35 AM
  • It feels like primitives could have been left out of the initial implementation of #Java. See why - http://t.co/A8ChCBHXJO
    June 29, 2015 at 4:05 PM
  • Developers in a bounce house! I repeat, developers in a bounce house! We had a blast at our 1st company picnic. Pics: http://t.co/XIqs7ECUst
    June 29, 2015 at 1:40 PM
  • New #SpringBatch tutorial from @jhackett01: Spring Batch – Replacing XML Job Configuration With JavaConfig http://t.co/PmdXnriKQu #java
    June 29, 2015 at 11:46 AM