Scott Addie

Last Updated: 8/16/2015

This past week, I had the pleasure of attending the fourth annual That Conference in Wisconsin Dells. Something I’ve always admired about this particular developers’ conference is its Open Spaces concept. The abundance of bacon-wrapped cuisine comes in a close second place.

What is an Open Space, you ask? In a nutshell, attendees are given the opportunity to create hour-long, ad hoc sessions on topics of their choosing to engage with other attendees. Since the volatile package management landscape is a personal topic of interest, I proposed a “Battle of the Package Managers” session; and, there was one takeaway that resonated afterwards. There is a great deal of uncertainty amongst NPM users on Windows who have encountered the 260-character path length limitation. This limitation stems from the Windows API’s MAX_PATH constraint. My goal with this blog post is to advocate practical solutions to the problem, but let’s first set the stage with a bit of background.

The Problem

Node.js relies on NPM to install the required modules. Said modules are installed into a node_modules directory in the project’s root directory. If you were to install Node.js on your machine at the time of this writing, NPM 2.x would be installed along with it. Execute the following command to see the version being used:

npm -v

For this blog post, version 2.11.3 is being used. This version was installed when upgrading to Node.js version 0.12.7. To effectively portray the problem we’re about to explore, execute the following command in your project’s root directory:

npm list

A quick look at the screenshots below proves that the maximum file path limitation is likely to be exceeded, due to NPM’s nesting of dependencies. There are 1,039 folders total within node_modules. Keep in mind that NPM 2.x is being used in this example.

As a Windows user, what are your options to mitigate this problem? Below are 3 possibilities.

1. Visual Studio Extension

If you’re a long-time Visual Studio user like myself, then you’ve been spoiled. The IDE abstracts away the intricacies of several development tools and technologies, thus maximizing productivity and allowing developers to focus on tackling the problems surfacing in their business domain. You want to wrangle this issue with the least amount of friction possible; and by that, I mean not using the command line. Luckily, Mads Kristensen of the ASP.NET team authored a “Flatten Packages” Visual Studio extension which is compatible with 2012 – 2015.

In Visual Studio 2015, you see a nice Dependencies node in Solution Explorer. It’s under here that the NPM modules will be listed. From the Solution Explorer tab, click the “Show All Files” icon for the web project containing the node_modules directory. Unfortunately, you cannot run Mads’ extension against the Dependencies/NPM folder; instead, right-click the now visible node_modules directory, and select “Flatten Packages…”. Alternatively, you can right-click the package.json file and will see the same context menu option.

The first time running this extension, the dialog below will appear. Don’t be alarmed! It’s indicating that there’s a dependent NPM  module called “flatten-packages” which must be installed at the system level. Click OK.

After the flattening process has completed, only 346 folders exist within node_modules. That’s a 66% decrease, and Windows is delighted to host this directory structure. There is no dependency nesting whatsoever.

2. flatten-packages

If you’re using an IDE or editor other than Visual Studio, then the extension described in the previous section isn’t applicable. It’s possible to replicate what the extension did; however, the command line must be used. Open a command prompt, and issue the following command to install the “flatten-packages” NPM module:

npm install -g flatten-packages

Next, navigate to your project directory, at the same level as the node_modules directory. Issue the following command:

flatten-packages

A quick look at the resulting folder size and dependency tree proves that the Visual Studio extension yields the same results as this manual approach. In other words, the extension takes the same actions behind the scenes.

3. Upgrade to npm@3

Version 3 of NPM aims to ease the pain for Windows users; however, it will remain in a beta phase until the team has deemed it stable enough for production. To install this version of NPM, issue the following command:

npm install -g npm@3.0-latest

Now issue the npm -v command, and notice the npm@2 is still being used. Why is that?!? It turns out that some configuration is needed. Open your Environment Variables modal (on Windows 10, that’s Control Panel –> System and Security –> System –> Advanced system settings), and modify the “Path” system variable. Order is important here. Amongst a plethora of other entries, you’ll have the path for Node.js:

C:\Program Files\nodejs\;

Immediately before that path, include the NPM path, so that you have something like this:

%APPDATA%\npm;C:\Program Files\nodejs\;

After changing your “Path” variable, close the command prompt, open it again, and check your NPM version number again with the npm -v command. This time around, you should see that version 3.x is being used.

After running an npm install command to install the necessary dependencies, it’s clear that the folder size is a drastic improvement over npm@2. There is a 56% decrease in the number of folders. Although that’s 10% worse than using the “flatten-packages” option, I’d much prefer this route. It is native NPM functionality rather than a “plugin”.

Conclusion

This is far from an exhaustive list of possible solutions. These are merely 3 solutions which I’ve used on past project work. It is my opinion that Windows NPM users are in a tough spot at this time; however, the situation is widely recognized across the industry and is slowly improving.

The “flatten-packages” option seems to be the most aggressive, and I’d even venture to say it’s experimental based solely on my mixed success with it. For example, I’ve witnessed entire Grunt build tasks break after the dependency tree flattening process. Comparatively, NPM@3 is a slightly more conservative beta product with some known issues. The prospect of a trouble-free experience with it is nothing short of a pipe dream. Thankfully, the NPM team is openly admitting that with this excerpt from their GitHub repository:

We need your help to find and fix its remaining bugs. It’s a significant rewrite, so we are sure there still significant bugs remaining. So do us a solid and deploy it in non-critical CI environments and for day-to-day use, but maybe don’t use it for production maintenance or frontline continuous deployment just yet.

There isn’t a bulletproof solution available. As with anything else, the moral of the story is to proceed with caution. What are some other solutions that have worked for you? I’m anxious to hear feedback on this topic.