Thoughts were had

All posts

Notes on returning to Angular 1

This post is written for the purpose of covering some gotchas and observations when going back to using Angular 1.x. For reference, we’ll use react components and lifecycles.

This might seem backwards, but has come up when returning to a large legacy codebase.

Read more

Posted in Programming Angular 0 Comments

Express HMR with webpack

Finally got around to setting up a working express hot module reloading config.

When, a year ago, I started replacing a custom Module implementation in our express servers with plain node modules in the name of simplicity, my colleagues started complaining about no longer having server side reloading. But with this, developing on the server is back to its former glory!

The purpose is to have only part of an express server work in this way, so that a module being required can update itself in place.

We’ll focus on getting a single module, and all it’s subdependencies to reload during development. Specifically, how to dynamically discover express.Router modules and mount them with reloading on an existing “normal” express server.

If you’re looking for a way to run your entire server through webpack, take a look at Hot reload all the things!

Webpack config

Let’s start out with a minimal webpack.config.server.js, that we’ll use as an example. We want to run everything through babel, to get that awesome es2018 feature support into node.

// webpack/webpack.config.server.js

const webpack = require('webpack');
const path = require('path');
const nodeExternals = require('webpack-node-externals');

module.exports = {
	entry: ['webpack/hot/poll?1000', './src/mount.webpack'],
	target: 'node',
	output: {
		path: path.join(__dirname, '.build'),
		filename: 'server.js'
		// expose main method as:
		library: 'mount',
    // Build it as a commonjs library so we can include it
		libraryTarget: 'commonjs'
  },
	// Don't bundle node_modules, they'll be available at runtime
	externals: [
		nodeExternals({
			// Include the hot reload polling code in the bundle though
			whitelist: ['webpack/hot/poll?1000']
		})
	],
	module: {
		rules: [
			{
				test: /\.js?$/,
				use: 'babel-loader',
				exclude: /node_modules/
			},
			{
				test: /\.json$/,
				use: 'json-loader'
			}
		]
	},
	plugins: [
		new webpack.NamedModulesPlugin(),
		new webpack.HotModuleReplacementPlugin(),
		new webpack.NoEmitOnErrorsPlugin()
	]
};

Autodiscovery with require contexts

The auto discovery loader could look something like this, before we add webpack:

// src/mount.js

const glob = require('glob');
const path = require('path');

module.exports = app => {
	// Automatically mount all *.express.js files:
	glob('src/routes/*/*.express.js', function(err, files) {
		files.forEach(file => {
			const entrypoint = require(file);
			// Handle ES6 default exports:
			const entrypointRouter = entrypoint.default || entrypoint;
			const mountAt = path.basename(path.dirname(file));
			app.use(`/${mountAt}`, entrypointRouter);
		});
	});
};

To be included in our express server like so:

// index.js

const express = require('express');
const app = express();

require('./.build/server').mount(app);

app.listen(3000);

But how can we keep this auto discovery, when we add webpack? require(file) can’t very well be statically analyzed, and trying to build it will generate errors instead.

That’s where require.context comes in. require.context is like webpacks own globber, with the added knowledge of modules and bundling. It looks like this:

require.context(directory, (useSubdirectories = false), (regExp = /^\.\//));
// src/mount.webpack.js

const glob = require('glob');
const path = require('path');

module.exports = app => {
	// Automatically mount all *.express.js files:
	const contextRequire = require.context('src/routes', true, /\.express\.js$/);
	const files = contextRequire.keys();

	files.forEach(file => {
		// Use the new context aware require to include the module:
		const entrypoint = contextRequire(file);
		// Handle ES6 default exports:
		const entrypointRouter = entrypoint.default || entrypoint;
		const mountAt = path.basename(path.dirname(file));
		app.use(`/${mountAt}`, entrypointRouter);
	});
};

HMR, require.context and express

In order to reload the dynamic context, we must hot.accept it by id, and then re-evaluate the context. Then we can swap out the route handler in express, and see our changes.

let context = require.context('src/routes', true, /\.express\.js$/);
if (module.hot) {
	module.hot.accept(context.id, function() {
		// Re-evaluate context to get updates:
		context = require.context('src/routes', true, /\.express\.js$/);
		// Do something with the updated library module...
	});
}

In our server, we can use the callback to re-evaluate our routes, and mount a new Router instance into express:

// src/mount.webpack.js
const { Router } = require('express');

module.exports = app => {
	let expressRouter;
	let buildPromise;
	let contextRequire;

	const getRouter = () => {
		contextRequire = require.context('src/routes', true, /\.express\.js$/);
		const files = contextRequire.keys();
		const newRouter = Router();

		files.forEach(file => {
			// Use the new context aware require to include the module:
			const entrypoint = contextRequire(file);
			// Handle ES6 default exports:
			const entrypointRouter = entrypoint.default || entrypoint;
			const mountAt = path.basename(path.dirname(file));
			app.use(`/${mountAt}`, entrypointRouter);
		});
	};
	// Initial load:
	getRouter();

	// Hot reload the context
	if (module.hot) {
		module.hot.accept(contextRequire.id, getRouter);
	}

	app.use((req, res, next) => {
		// A swappable route handler for HMR updates
		expressRouter(req, res, next);
	});
};

Now, if we try and make any changes to a matching router, we should see something like

[HMR] Updated modules:
[HMR]  - ./src/routes/auth/login.express.js
[HMR]  - ./src/routes recursive \.express\.js$
[HMR] Update applied.

And the router should have been automatically swapped out.

A simple dev-server middleware

Having been spoiled by webpack-dev-server and webpack-dev-middleware, running webpack --watch on the side doesn’t feel quite right. Luckily, we can use the Node.js API of webpack to do something similar.

That means adding an automatic wait for build, and compiling during startup.

// index.js
// "webpack-dev-server" minimal example

const express = require('express');
const app = express();

const isDev = process.env.NODE_ENV !== 'production';

if (isDev) {
	const webpack = require('webpack');
	const webpackConfig = require('./webpack/webpack.config.server.js');

	const buildPromise = new Promise(resolve => {
		const compiler = webpack(webpackConfig);
		compiler.watch({ filename: paths.appSrc }, function(err, stats) {
			if (err) {
				console.error(err);
			}
			resolve();
		});
	}).then(() => {
		// Mount the routes when ready (and only once via promise semantics):
		require('./.build/server').mount(app);
	});

	// waitForBuild middleware to avoid confusing 404s
	app.use((req, res, next) => {
		buildPromise.then(next, next);
	});
} else {
	// In production mode, load it as before:
	require('./.build/server').mount(app);
}

app.listen(3000);

Source-maps for sanity and production

To keep sane with transpiled code, we want source-maps in the node process itself. This is easily accomplished by making sure webpack config is set to devtool: 'inline-source-map', and the following snippet injected before importing the bundled code:

require('source-map-support').install({
	// hookRequire for inline-sourcemaps support:
	hookRequire: true
});

Conclusion

We had to jump through a few hoops, but the server is happily compiling as needed and hot reloading during development.

Before starting in production, use webpack CLI to generate the mount file:

$ webpack --config webpack/webpack.config.server.js

Have fun iterating even faster on your express modules!

Read more

Posted in Programming Webpack 0 Comments

Railway Oriented Programming in Javascript

A mental model for promise chains and error handling, adapted to javascript promises. This post is entirely based on the post and talk Railway Oriented Programming by Scott Wlaschin.

I am quite fascinated with our tendency to focus on the happy code path, which often leaves error handling as an afterthought. It always bothered me, and with the recent popularization of functional programming techniques I think I have finally found a mental model for error handling that I like. It’s the either monad!

As it turns out, promises in Javascript are quite analogous to the Either monad. If you have no idea what that means, I hope the Railway analogy will help you as it did me.

Most functions that are doing external calls like HTTP requests, file system operations or runtime validation will have at least two different outputs: a success case and an error case.

Read more

Posted in Programming FP, Promises 0 Comments

Abstractions are mental overhead

It’s incredible how quickly everything moves in our field. Nothing makes this more clear than looking through our own old code, or in this example old posts right here on the blog.

The quest for not repeating anything makes us do crazy hacks and abstractions, just to avoid having those very error prone background-color: red; “duplicated” between two files.

Most abstractions seem beneign at conception, but tend to mutate with time. The more abstractions it has, the more cracks will appear when you need to move or change an interface between classes/modules. When a feature needs adding, and falls into the void, who knows where it ends up? Sometimes math.random() might as well decide, since “anywhere” will seem perfect when deadlines loom.

Even when you perfectly understand the entire codebase, figuring out which classes to change can be hard. Now pass this code on to the next guy, who has none of the context. How many concepts will they have to learn in order to work in the framework and vocabulary that you have created without breaking it?

Avoid leading your colleagues deeper into the rabbit hole of random abstractions by keeping the APIs simple. Use simple or common datatypes where possible.

Avoid over engineering your APIs to please the OOP gods.

Don’t try to be clever with indirections

Tunnel vision comes in many forms. Trying to decouple your code is another example. In this example, a post from MSDN called Active events. Basically, this article describes a pattern to avoid having any explicit dependencies in your code, in the attempt to make it easier to replace parts of it with another DLL.

This hits a nerve with me, as implicit dependencies are at the core of many bugs I’ve seen lately. You might be able to replace the AuthenticationProvider from any file in the project now, but is this a fox in sheeps clothes? I think it is, since you have most certainly just confused your IDE, and possibly also your colleagues. If the code that runs when invoking dispatch("auth-please") can live anywhere, you might as well have just done address space randomization on you project files. You have decoupled the code, your IDE and any possibility of doing explorative discovery to learn how it works.

Working in a system of implicit dependencies can be extremely draining, since the effects of change are hidden behind layers of indirection and abstraction. You get zero help from standard tooling. The less your tools can help you, the more mental capacity is needed to juggle the parts and their relationships. Even more so, to start changing these relationships when requirements inevitably change.

Read more

Posted in Programming 0 Comments

Extending $q promises in Angular

Why another post about this? Most implementations seem to overlook the fact that the changes applied to delegate.defer only affect the first promise in the chain, since by design the defer function used internally in Angular cannot be modified.

Since we can affect the first returned promise we have our way in. Now to make sure we stay “in”.

Read more

Posted in Programming Angular, Promises 0 Comments