others-How to solve 'node_modules directory missing or not created' issue when using npm install command ?

Jul 14, 2021

1. Purpose

In this post, I would demo how to solve the node_modules directory not creating issue when using npm install command .

Suppose we have created a new directory named nodejs1

$ mkdir nodejs1
$ cd nodejs1

And then we call the npm install command to install a module, say, express:

$ npm install express
added 49 packages, and removed 72 packages in 4s
npm notice
npm notice New minor version of npm available! 7.11.2 -> 7.19.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v7.19.1
npm notice Run npm install -g [email protected] to update!
npm notice

Then we list the directory including the hidden files:

$ ls -la
total 0
drwxr-xr-x  2 bswen  staff   64 Jul 14 20:06 .
drwxr-xr-x  8 bswen  staff  256 Jul 14 20:06 ..

Where is the node_modules directory? It should be created automatically after executing the command npm install xxx, but it’s missing in current working directory! Where is it?

2. Environment

Node.js v12.14.0
npm 7.11.2

3. The solution and reason

3.1 The solution

Suppose your home directory is ~/

npm has already created the node_modules directory in your home path, e.g. ~/node_modules

➜  nodejs1 ls ~/node_modules
@types              depd                merge-descriptors   send
@webassemblyjs      destroy             methods             serve-static
@xtuc               ee-first            mime                setprototypeof
accepts             express             mime-db             socket.io
array-flatten       engine.io           mime-types          socket.io-adapter
base64-arraybuffer  engine.io-parser    ms                  socket.io-parser

You can see that our modules are installed in that directory by default, what if we want to install them into current directory?

We can do as follows to create node_modules directory in our current working directory:

$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help init` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
package name: (nodejs1)
version: (1.0.0)
description:
entry point: (index.js)
test command:
git repository:
keywords:
author:
license: (ISC)
About to write to /Users/bswen/js-projects/nodejs1/package.json:

{
  "name": "nodejs1",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}


Is this OK? (yes) yes

Now ,after executing the npm init command, let’s see the results:

$ ls -la
drwxr-xr-x  3 bswen  staff    96B Jul 14 20:25 .
drwxr-xr-x  8 bswen  staff   256B Jul 14 20:06 ..
-rw-r--r--  1 bswen  staff   203B Jul 14 20:25 package.json

The package.json content is:

➜  nodejs1 cat package.json
{
  "name": "nodejs1",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

Then we can install the package again:

➜  nodejs1 npm install express
added 50 packages in 2s

➜  nodejs1 ls -l
total 80
drwxr-xr-x  53 bswen  staff   1.7K Jul 14 20:26 node_modules
-rw-r--r--   1 bswen  staff    32K Jul 14 20:26 package-lock.json
-rw-r--r--   1 bswen  staff   253B Jul 14 20:26 package.json

You can see that the package.json and node_modules are both created correctly now in current working directory.

We can check the content of package.json again:

➜  nodejs1 cat package.json
{
  "name": "nodejs1",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.1"
  }
}

You can see that our package express is added into the dependencies of package.json.

3.2 The reason

According to the npm official document, the npm init is:

npm init <initializer> can be used to set up a new or existing npm package.

initializer in this case is an npm package named create-<initializer>, which will be installed by npm-exec, and then have its main bin executed – presumably creating or updating package.json and running any other initialization-related operations.

The npm init command would try to initialize the package.json in current directory. And according to this article , the npm install xxx is:

npm install :

Install the package in the directory as a symlink in the current project. Its dependencies will be installed before it’s linked. If sits inside the root of your project, its dependencies may be hoisted to the top-level node_modules as they would for other types of dependencies.

That is to say, npm install would try to install the dependency in the top-level node_modules directory, and then link the files to your working directory.

When you require a package , say bar.js, in your code, node.js would try to find the dependency in following order:

/home/bswen/<current_working_dir>/node_modules/bar.js
/home/bswen/node_modules/bar.js
/home/node_modules/bar.js
/node_modules/bar.js

The key point is:

Read the dependencies of the nearest node_modules first
Recursively look up node_modules dependencies

4. Summary

In this post, I tried to demo how to solve the node_modules initialization problem when trying to create it using npm install xxx command. The key point is that you should first call npm init to create the package.json, then call npm install xxx to install the dependency, since node.js would only create node_modules in current working directory when there exists a package.json file.