Yarn's bundled node-gyp broken

[wyattjoh]

The current version of yarn still has a pre-existing issue with the pre-bundled version of node-gyp related to yarnpkg/yarn#2266

A temporary fix relates to adding:

RUN yarn global add node-gyp

To our Dockerfile.

[wyattjoh]

You can refer to: https://circleci.com/gh/coralproject/talk/1921 with details of why the build failed.

[LaurentGoderre]

To me it failed only because the build was trying to override the included yarn. I don't see any node-gyp error.

[LaurentGoderre]

As you can see, several other build because of trying to install yarn via npm: #337 (comment)

[wyattjoh]

I linked the wrong build logs :( below is the relevant error:

Step 9/11 : RUN yarn install --production
 ---> Running in c936d65bcc45
yarn install v0.21.3
[1/4] Resolving packages...
[2/4] Fetching packages...
warning fsevents@1.0.17: The platform "linux" is incompatible with this module.
info "fsevents@1.0.17" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error /usr/src/app/node_modules/bcrypt: Command failed.
Exit code: 127
Command: sh
Arguments: -c node-gyp rebuild
Directory: /usr/src/app/node_modules/bcrypt
Output:
sh: 1: node-gyp: not found
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command '/bin/sh -c yarn install --production' returned a non-zero code: 1

Which was ran with the repo here https://github.com/coralproject/talk/tree/2c98aa20977155d09b08e927c574be1221d2d707 where I had removed my patch from the Dockerfile to make it the following:

FROM node:7.6

# Add node-gyp for bcrypt build support
# RUN yarn global add node-gyp

# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Setup the environment
ENV NODE_ENV production
ENV PATH /usr/src/app/bin:$PATH
ENV TALK_PORT 5000
EXPOSE 5000

# Install app dependencies
COPY package.json yarn.lock /usr/src/app/
RUN yarn install --production

# Bundle app source
COPY . /usr/src/app

CMD [ "yarn", "start" ]

[LaurentGoderre]

Node-gyp is not included in node though so you would always need to install it no?

[wyattjoh]

Previously, using npm install it successfully installs.

[LaurentGoderre]

It doesn't install with yarn?

[wyattjoh]

That's correct. With the following Dockerfile:

FROM node:7.6

# Add node-gyp for bcrypt build support
# RUN yarn global add node-gyp

# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Setup the environment
ENV NODE_ENV production
ENV PATH /usr/src/app/bin:$PATH
ENV TALK_PORT 5000
EXPOSE 5000

# Install app dependencies
COPY package.json yarn.lock /usr/src/app/
# RUN yarn install --production
RUN npm install --production

# Bundle app source
COPY . /usr/src/app

CMD [ "npm", "start" ]

I got no errors during install.

[LaurentGoderre]

That's weird...I ran the same thing and it worked:

Sending build context to Docker daemon 2.048 kB
Step 1/2 : FROM node:7.6
7.6: Pulling from library/node
693502eb7dfb: Pull complete 
081cd4bfd521: Pull complete 
5d2dc01312f3: Pull complete 
54a5f7da9a4f: Pull complete 
f6ebc0704397: Pull complete 
cd487f72ac9f: Pull complete 
d3e88bf53e70: Pull complete 
88dbf525e469: Pull complete 
Digest: sha256:34d21bab49105c1ab384e50432aa6ebd6dac69c2637378e7ecb6f40cda97118b
Status: Downloaded newer image for node:7.6
 ---> 7b74673e3e78
Step 2/2 : RUN yarn global add node-gyp
 ---> Running in 58721d12a3bd
yarn global v0.21.3
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed "node-gyp@3.5.0" with binaries:
      - node-gyp
Done in 5.15s.
 ---> d383b79b9c88
Removing intermediate container 58721d12a3bd
Successfully built d383b79b9c88

[LaurentGoderre]

Is it possible that node-gyp installs but isn't working?

[wyattjoh]

This is my issue:

When installing via yarn, I need to run:

yarn global add node-gyp
yarn install --production

While with npm I can run:

npm install --production

I don't need to explicitly install node-gyp, hence why it seems that the upstream issue with yarn (yarnpkg/yarn#2266) might be causing the issue now where when installed via yarn (specifically, the bundled version), that node-gyp fails.