Source Maps in Node.js

Of the 21,717 respondents to the 2019 State of JavaScript Survey, ~60% said that they spend time working in an alternate flavor of JavaScript, this is up from ~21% in 2016. Increasingly, when someone writes JavaScript, they're actually writing an abstraction that compiles to JavaScript.

These abstractions offer a variety of benefits: for instance the type safety introduced by Flow and TypeScript, or the functional programming paradigm introduced by ClojureScript. However, we are also faced with a challenge. Node.js, developed in 2009, didn't anticipate the modern world of transpilers. And so alternate flavors of JavaScript present a disadvantage in terms of observability. Take the following TypeScript code:

enum HttpStatusCode {
  NOT_FOUND = 404,
  ERROR = 500
}
class MyHttpError extends Error {
  code: HttpStatusCode;
  constructor(msg: string, code: HttpStatusCode) {
    super(msg);
    Error.captureStackTrace(this, MyHttpError);
    this.code = code;
  }
}
throw new MyHttpError('not found', HttpStatusCode.NOT_FOUND);

When compiled to JavaScript, the above code ends up looking like this:

var __extends = (this && this.__extends) || (function () {
    var extendStatics = function (d, b) {
        extendStatics = Object.setPrototypeOf ||
            ({ __proto__: [] } instanceof Array && function (d, b) { d.__proto__ = b; }) ||
            function (d, b) { for (var p in b) if (b.hasOwnProperty(p)) d[p] = b[p]; };
        return extendStatics(d, b);
    };
    return function (d, b) {
        extendStatics(d, b);
        function __() { this.constructor = d; }
        d.prototype = b === null ? Object.create(b) : (__.prototype = b.prototype, new __());
    };
})();
var HttpStatusCode;
(function (HttpStatusCode) {
    HttpStatusCode[HttpStatusCode["NOT_FOUND"] = 404] = "NOT_FOUND";
    HttpStatusCode[HttpStatusCode["ERROR"] = 500] = "ERROR";
})(HttpStatusCode || (HttpStatusCode = {}));
var MyHttpError = /** @class */ (function (_super) {
    __extends(MyHttpError, _super);
    function MyHttpError(msg, code) {
        var _this = _super.call(this, msg) || this;
        Error.captureStackTrace(_this, MyHttpError);
        _this.code = code;
        return _this;
    }
    return MyHttpError;
}(Error));
throw new MyHttpError('not found', HttpStatusCode.NOT_FOUND);
//# sourceMappingURL=test.js.map

The generated source looks a lot different than the original source. Take the statement throw new MyHttpError. In the original source this error is thrown on line 13, but in the generated source it's thrown on line 29. These differences make debugging difficult — the stack trace displayed in the code being run doesn't match the code being written.

The increasing popularity of alternate flavors of JavaScript and the observability challenges described above are the motivation for the recent work in Node.js related to source maps (node#28960, node#31143, node#31132).

The Source Map V3 specification

Source maps provide a method for translating from generated source back to the original, via meta-information attached to the generated source; In JavaScript and CSS Source Map Revision 3 has become the de facto standard. Here are some key facts about the specification:

  • V3 of the spec was created as a joint effort by engineers John Lenz at Google and Nick Fitzgerald at Mozilla.
  • The format trades simplicity and flexibility for a reduced overall size, making it more practical to use source maps in real-world environments, e.g., loading them over a network when DevTools are opened.
  • The format allows for 1 to 1 mappings, e.g., a .ts file mapping to its compiled .js version, and for 1 to many mappings, e.g., many sources being minified into a single source file.
  • Source maps are embedded in the generated source using a special comment. These comments may contain the entire source map, using a Data URI, or may reference an external URL or file.

Looking back at the TypeScript example earlier in this article, note the line at the end of the generated source:

//# sourceMappingURL=test.js.map

This special comment indicates that the source map can be found in the file test.js.map, which is in the same folder as the generated source code. Most tools that transpile JavaScript, such as TypeScript, provide an option to generate source maps.

Adding support for the Source Map V3 format to Node.js was an important step towards better supporting the alternate flavors of JavaScript that are being written today.

Caching source maps for code coverage

In v12.11.0, the behavior of the environment variable NODE_V8_COVERAGE was updated, such that when a program is run with this variable set, and a require or import observes a special source map comment, the corresponding source map is loaded and cached.

As the variable name suggests, source map support was initially added to address a problem with native V8 code coverage; tools like ts-node and nyc insert source maps during run time, and these source maps were not available after a program finished execution. Due to this, accurate coverage reports could not be provided. By caching source maps during runtime, and outputting the information with coverage reports, Node.js' native code coverage could be made to provide accurate reports for tools like ts-node.

Along with addressing bugs with code coverage, this work created a foundation for adding further source map support to Node.js.

Applying source maps to stack traces

In v12.12.0, Node.js introduced the flag --enable-source-maps. When a program is run with this flag set, source maps will be cached, and used to provide an actionable stack trace when an exception occurs. Let's look at what happens when we execute the TypeScript example in the first section of this article: