Working with node servers is part of the job. While iterating on such servers, it’s common to go for a solution involving automatic restarts using something like nodemon.

npx nodemon --exec "npm start" --watch src

This will automatically restart the server whenever there are changes to the src folder. But I think we can do better than that!

The restart can be expensive, depending on the kinds of initialization that’s involved. What if we just make node reevaluate only the parts that changed?

Forget that module

If we mess a little with the module (require) cache of node, we can break the normal singleton handling of modules, and run it again, with any changes we might have added in the meantime.

// forgetModule.js
// const { flatMap } = require('lodash/fp');
const flatMap = fn => xs => xs.reduce((agg, x) => agg.concat(fn(x)), []);

module.exports = function forgetModule(filename) {
	// Clear the module cache, load any updates from filesystem:
	const cached = require.cache[require.resolve(filename)] || {};
	const getModules = flatMap(m => [m.id].concat(getModules(m.children)));
	getModules([cached])
		.filter(s => s && !s.match(/\/node_modules\//)) // but leave node modules
		.forEach(fn => {
			delete require.cache[fn];
		});
};

With that, we will get the first-load behavior (evaluation) on any following require:

require('./log'); // console.log('hi');
require('./log'); // - cached, nothing happens.
forgetModule('./log');
require('./log'); // console.log('hi');

The module will be read from disk and evaluated fresh. Success!

Swappable routing

Forgetting modules is all well and good, but how do we make something like express swap out it’s routes? There is no API to interfere with existing routes in the middleware stack.

We can instead add a route function (which will be locked in the middleware stack) that is dynamic, and all we have to do then is swap out the handler function in there.

// index.js
const express = require('express');
const forgetModule = require('./forgetModule');

const app = express();

// A non-swappable router:
app.use(require('./fixed-routes'));

// Swappable route handler
app.use((req, res, next) => {
	// To keep it simple, reload the routes on every request:
	if (process.env.NODE_ENV !== 'production') {
		forgetModule('./routes');
	}
	// Anywhere we want to support hot-swapping, we need to require in place
	require('./routes')(req, res, next);
});

app.listen(3000);

// routes.js
const Router = require('express').Router;
const router = Router();

router.get('/', (req, res) => {
	res.send('Hello world');
});

module.exports = router;

Watch and forget. Reload on change

If the simple always reload solution seems a little too coarse, we can always combine it with a file watcher, still without having to restart the whole process:

// watch.js
const chokidar = require('chokidar');

chokidar.watch('./src').on('all', (event, path) => {
	forgetModule(path);
});

If the inline requires are set up, we can use require('./watch'), and reload on change instead of on every request.

Caveat: A watcher like this will only work if you have an inline require to the file that is being removed. I.e. changing routes.js from above will work, but changing routes/auth.js, required from routes.js will not since routes is still cached. One possible solution is to use the watcher to invalidate, but still invalidate the root, i.e. forgetModule('./routes').

Another solution could be to traverse upwards in the module tree when invalidating, until reaching a “stop” file, such as the routes file. This would give a more fine grained invalidation, in cases where invalidating all our routes is causes problems.

Posted in Programming with :