Improved Error Handling in DoneJS

DoneJS now features improved error handling and messaging for those on donejs @2. In recent weeks we have added:

A formatted error page highlighting what went wrong thanks to donejs-error-format.
The ability for done-serve to recover from just about any error while preserving hot module replacement.
Code snippets in error messages that show where the problem occured.
Links to documentation explaining each error in greater detail.

This video shows off the incredible developer experience we are striving for in DoneJS (make fullscreen to get a better view):

The DoneJS homepage talks about usability, performance, and maintainability as being the cornerstones of the framework. In version 1 we put a strong focus on performance; in version 2, we are now looking towards maintainability.

With CanJS 4.0, we started to place more emphasis on the debugging experience. can.queues.logStack() and can-debug combined provide insight into how data and events cause changes throughout an application. Recently we have extended this emphasis to other tools in our suite: steal (module loading and bundling), done-ssr (server-side rendering), and done-serve (development server). Because DoneJS is a full-stack framework, we can provide a holistic development experience.

With the Improved Error Messages and Handling proposal we set out to fix 2 things:

Make done-serve more resilient to errors and bad application state.
Revisit how we structure error messages for greater clarity.

In this article I'll go over what we did and where we want to take things from here.

The Importance of Good Errors

Writing good error messages can be difficult to do for a variety of reasons, including that errors are often unanticipated events. (hopefully). If a library doesn't have good error messages it might be because they are focused on steering you in the right direction before errors occur.

In DoneJS we have traditionally put an emphasis on guides as a way of teaching developers how to use our libraries. When you start developing your own application you tend to run into things that guides don't teach you, and that's where API documentation comes in.

Error messages should be a bridge between a developer's use of a library and deeper understanding of advanced concepts. The idea being that error messages should not only help you fix your immediate problem but also guide you towards a better understanding of the framework.

A good error message should follow the Five Ws (with how replacing when). Here's an example of a new error message in steal-less:

It tells you:

What occurred - A stylesheet could not be fetched.
Who was involved - not-exists.less in the code frame.
Where the error occurs - The stack trace points to main.less line 2.
Why the error might have occurred - Perhaps the file hasn't been saved yet, and therefore isn't present on disk, or perhaps the path is incorrect in the parent stylesheet.
How this error can be avoided - by linking to documentation on stealjs.com that explains how paths are resolved in steal.

All of these parts are important, but I especially like the why. Errors are helpful for teaching, but we all will encounter errors from time to time and providing suggestions on how to fix the problem helps speed up development workflow. DoneJS is in a unique position to answer the why because it is involved from the moment files are created through testing out their functionality.

For this cycle we worked on almost a dozen different error scenarios to improve the messaging. Most of these came in Steal, as loading modules is common source of errors. Here are a few of the highlights:

Missing Files (404s)

This was the first error I worked on and it's an error that everyone encounters. The error looks like this now:

When I started working on it, it looked like this:

So to improve this I wanted to:

Remove the steal.js stack trace, replacing it with the line in the parent module that was importing the missing module.
Create a page on stealjs.com that documents this particular error message and gives some hints at what could be the problem.
Show a snippet of the code where the error takes place. For small issues like typos this becomes quickly apparent, making for a quick fix.

This error message works with plugins that provide the right metadata. Here's steal-stache providing an error that points to the can-import that imports the missing file:

Syntax Errors

Since this is such a common category of mistake we wanted to clean these up as much as possible. In addition to CommonJS and ES modules, we also improved JSON syntax errors. Here's a case where a property isn't wrapped in double quotes:

The stack trace at the bottom shows that the error occurs in config.json, line 5.

Error Handling in done-serve

In development, done-ssr is connected to hot module swapping just as the browser is. In the browser you can reset things very easily by refreshing. But this is more cumbersome to do with the server; you have to kill it and restart. This meant we needed to make done-ssr more resilient to errors so that it could fix itself once code was corrected. We felt the best workflow would be:

You run donejs develop and the server starts, connecting to a HMR (hot module replacement) server.
An error occurs (maybe one of the ones listed in the section above) and the server remembers this error and renders HTML for it.
Once an error occurs, the server will continue to render the error page until fixed.
Once fixed, the server's state is restored; HMR refreshes the browser to a working state.

To achieve this, we had to add some new APIs to steal to provide errors that occur during HMR, and create a library for formatting error messages. The video at the top of this article shows off the workflow and how everything comes together.

If you are not using done-serve, you can still get the nicely formatted error messages by using donejs-error-format. If you are using express, your code most likely will look like:

const express = require("express");
const errorFormat = require("donejs-error-format");
const ssr = require("done-ssr-middleware");

const app = express();

app.use(express.static(__dirname + "/public"));
app.use(ssr({ config: __dirname + "/package.json!npm" }));

// The last middleware should be the error handler

app.use(function(error, request, response, next) {
  let parts = errorFormat.extract(error);
  let html = errorFormat.html(parts);

  console.error(error);

  response.type("html").end(html);
});

Next Steps

Now that the infrastructure is in place, we'll continue to improve error messages and provide more consistency across projects. We're also looking into ways to provide tighter integration with DoneJS projects so you could do things like write error messages with inferred links instead of hardcoded URLs. We're continuing to explore these types of integration points. We want feedback on these sorts of things, so please file an issue with your ideas.