Pfthb

I'm a very experienced Java dev, and I've worked on build automation for Java builds a great deal. I haven't done much with front-end builds, so I'm not familiar with the problem space in most of the front-end build tools.

In one situation, my build from a plain pipeline script in Jenkins is failing with the following:

[xxx-feature%2F...] Running shell script
+ yarn test:coverage -u
yarn run v1.3.2
warning package.json: No license field
$ jest --env=jsdom --coverage -u
/bin/sh: 1: jest: not found

This is from a build of a pull request branch. The same pipeline script is used for the build of the master branch, although it performs some additional steps. When the master build gets to the same point in the script, it doesn't fail.

This failure isn't specific to a particular pull request branch. I've seen this same error for many of the pull request builds. I doubt any of them have passed.

I've compared the "package.json" files, and they are identical.

What else should I be looking at to diagnose this problem?

Update:

I've made no real progress with this. I've since added additional diagnostics that are run in both the "master" build and the "pull request" build. I thought the best chance of getting a clue was running "env" right before it. Unfortunately, I saw nothing useful in those minor differences. PATH isn't even set for either.

Update:

Just to emphasize the difference, here is the output from the same pipeline script, on the "master" build:

[...] Running shell script
+ yarn --verbose test:coverage -u
yarn run v1.3.2
warning package.json: No license field
$ jest --env=jsdom --coverage -u
[BABEL] Note: The code generator has deoptimised the styling ...

The unit tests work fine in the master build, although it fails to find "jest" in the PR build.

Update:

A commenter thought that perhaps "yarn install" might be important here. The build doesn't run "yarn install" directly, but it does run "yarn", which runs "yarn install" under the covers. I also noted that that wasn't the first "yarn" command it was running. I just changed the script to do that first, but it didn't fix the problem. I'm going to include here a heavily edited output showing the "yarn install" up to the failing call. Perhaps there will be some relevant clue in this.

+ yarn
yarn install v1.3.2
warning package.json: No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@1.2.4: The platform "linux" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > react-router@4.3.1" has unmet peer dependency "react@>=15".
warning " > enzyme-adapter-react-16@1.7.0" has unmet peer dependency "react@^16.0.0-0".
warning " > enzyme-adapter-react-16@1.7.0" has unmet peer dependency "react-dom@^16.0.0-0".
warning "enzyme-adapter-react-16 > enzyme-adapter-utils@1.9.0" has unmet peer dependency "react@0.13.x || 0.14.x || ^15.0.0-0 || ^16.0.0-0".
warning "enzyme-adapter-react-16 > react-test-renderer@16.6.3" has unmet peer dependency "react@^16.6.3".
warning "jest > jest-cli > jest-environment-jsdom > jsdom > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".
warning "jest > jest-cli > jest-environment-jsdom > jsdom > request-promise-native > request-promise-core@1.1.1" has unmet peer dependency "request@^2.34".
[4/4] Building fresh packages...
Done in 29.01s.
+ yarn cache clean
yarn cache v1.3.2
warning package.json: No license field
success Cleared cache.
Done in 2.08s.
+ yarn clean:all
yarn run v1.3.2
warning package.json: No license field
$ yarn clean:lib && yarn clean:temp
warning package.json: No license field
$ rimraf ./build
/bin/sh: 1: rimraf: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
+ yarn upgrade --scope @idp
yarn upgrade v1.3.2
warning package.json: No license field
[1/4] Resolving packages...
warning css-loader > cssnano > postcss-merge-rules > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning css-loader > cssnano > autoprefixer > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning css-loader > cssnano > postcss-merge-rules > caniuse-api > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning enzyme > rst-selector-parser > nearley > nomnom@1.6.2: Package no longer supported. Contact support@npmjs.com for more info.
[2/4] Fetching packages...
info fsevents@1.2.4: The platform "linux" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > react-router@4.3.1" has unmet peer dependency "react@>=15".
warning " > enzyme-adapter-react-16@1.7.0" has unmet peer dependency "react@^16.0.0-0".
warning " > enzyme-adapter-react-16@1.7.0" has unmet peer dependency "react-dom@^16.0.0-0".
warning "enzyme-adapter-react-16 > enzyme-adapter-utils@1.9.0" has unmet peer dependency "react@0.13.x || 0.14.x || ^15.0.0-0 || ^16.0.0-0".
warning "enzyme-adapter-react-16 > react-test-renderer@16.6.3" has unmet peer dependency "react@^16.6.3".
warning "jest > jest-cli > jest-environment-jsdom > jsdom > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".
warning "jest > jest-cli > jest-environment-jsdom > jsdom > request-promise-native > request-promise-core@1.1.1" has unmet peer dependency "request@^2.34".
[4/4] Rebuilding all packages...
success Saved lockfile.
success Saved 1294 new dependencies.
├─ @types/jest@21.1.10
├─ babel-jest@22.4.4
├─ babel-plugin-jest-hoist@22.4.4
├─ babel-preset-jest@22.4.4
├─ jest-changed-files@22.4.3
├─ jest-cli@22.4.4
├─ jest-config@22.4.4
├─ jest-diff@22.4.3
├─ jest-docblock@22.4.3
├─ jest-environment-jsdom@22.4.3
├─ jest-environment-node@22.4.3
├─ jest-get-type@22.4.3
├─ jest-haste-map@22.4.3
├─ jest-jasmine2@22.4.4
├─ jest-leak-detector@22.4.3
├─ jest-matcher-utils@22.4.3
├─ jest-message-util@22.4.3
├─ jest-mock@22.4.3
├─ jest-regex-util@22.4.3
├─ jest-resolve-dependencies@22.4.3
├─ jest-resolve@22.4.3
├─ jest-runner@22.4.4
├─ jest-runtime@22.4.4
├─ jest-serializer@22.4.3
├─ jest-snapshot@22.4.3
├─ jest-util@22.4.3
├─ jest-validate@22.4.4
├─ jest-worker@22.4.3
├─ jest@22.4.4
├─ ts-jest@22.4.6
Done in 49.89s.
+ yarn upgrade --scope @idse
yarn upgrade v1.3.2
warning package.json: No license field
[1/4] Resolving packages...
warning css-loader > cssnano > postcss-merge-rules > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning css-loader > cssnano > postcss-merge-rules > caniuse-api > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning css-loader > cssnano > autoprefixer > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning enzyme > rst-selector-parser > nearley > nomnom@1.6.2: Package no longer supported. Contact support@npmjs.com for more info.
[2/4] Fetching packages...
info fsevents@1.2.4: The platform "linux" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > react-router@4.3.1" has unmet peer dependency "react@>=15".
warning " > enzyme-adapter-react-16@1.7.0" has unmet peer dependency "react@^16.0.0-0".
warning " > enzyme-adapter-react-16@1.7.0" has unmet peer dependency "react-dom@^16.0.0-0".
warning "enzyme-adapter-react-16 > enzyme-adapter-utils@1.9.0" has unmet peer dependency "react@0.13.x || 0.14.x || ^15.0.0-0 || ^16.0.0-0".
warning "enzyme-adapter-react-16 > react-test-renderer@16.6.3" has unmet peer dependency "react@^16.6.3".
warning "jest > jest-cli > jest-environment-jsdom > jsdom > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".
warning "jest > jest-cli > jest-environment-jsdom > jsdom > request-promise-native > request-promise-core@1.1.1" has unmet peer dependency "request@^2.34".
[4/4] Rebuilding all packages...
success Saved lockfile.
success Saved 1294 new dependencies.
├─ @types/jest@21.1.10
├─ babel-jest@22.4.4
├─ babel-plugin-jest-hoist@22.4.4
├─ babel-preset-jest@22.4.4
├─ jest-changed-files@22.4.3
├─ jest-cli@22.4.4
├─ jest-config@22.4.4
├─ jest-diff@22.4.3
├─ jest-docblock@22.4.3
├─ jest-environment-jsdom@22.4.3
├─ jest-environment-node@22.4.3
├─ jest-get-type@22.4.3
├─ jest-haste-map@22.4.3
├─ jest-jasmine2@22.4.4
├─ jest-leak-detector@22.4.3
├─ jest-matcher-utils@22.4.3
├─ jest-message-util@22.4.3
├─ jest-mock@22.4.3
├─ jest-regex-util@22.4.3
├─ jest-resolve-dependencies@22.4.3
├─ jest-resolve@22.4.3
├─ jest-runner@22.4.4
├─ jest-runtime@22.4.4
├─ jest-serializer@22.4.3
├─ jest-snapshot@22.4.3
├─ jest-util@22.4.3
├─ jest-validate@22.4.4
├─ jest-worker@22.4.3
├─ jest@22.4.4
├─ ts-jest@22.4.6
Done in 34.56s.
+ yarn --verbose test:coverage -u
yarn run v1.3.2
$ jest --env=jsdom --coverage -u
/bin/sh: 1: jest: not found
verbose 0.383 Error: Command failed with exit code 127.
 at /usr/share/yarn/lib/cli.js:35620:15
 at Generator.throw (<anonymous>)
 at step (/usr/share/yarn/lib/cli.js:92:30)
 at /usr/share/yarn/lib/cli.js:105:13
 at <anonymous>
 at process._tickCallback (internal/process/next_tick.js:188:7)
error Command failed with exit code 127.

Update:

A commenter asked me to show the contents of "node_modules/.bin". I added that, and also the contents of another directory referenced in that listing.

+ ls -lt node_modules/.bin
total 0
lrwxrwxrwx 1 81050 20059 18 Dec 3 17:25 which -> ../which/bin/which
lrwxrwxrwx 1 81050 20059 47 Dec 3 17:25 webpack-dev-server -> ../webpack-dev-server/bin/webpack-dev-server.js
...
lrwxrwxrwx 1 81050 20059 35 Dec 3 17:25 jest-runtime -> ../jest-runtime/bin/jest-runtime.js
lrwxrwxrwx 1 81050 20059 23 Dec 3 17:25 jest -> ../jest-cli/bin/jest.js
...
+ ls -lt node_modules/jest-cli/bin
total 4
-rwxr-xr-x 1 81050 20059 416 Dec 3 17:25 jest.js

I don't see anything wrong here.

Update:

I just found a fix for this, but I still don't understand what the underlying cause is.

I've been struggling with this for almost a month now, with no resolution, and then on Friday I noticed that two builds of the same pull request had gotten past this. Builds that ran after the builds for that PR continued to fail. I reran one of the builds that passed, and it passed again. There was nothing significant in the changes in the PR. The only thing that was at all different from all the other PRs was the form of the branch name.

Almost all of the other branches in this project were of the form "feature/..." or "bugfix/...", or "release/...". The one that passed used a name without "/" in it. I have a mechanism in the build script that ensures that workspaces for PR builds are unique, by appending the branch name to the workspace directory, but since I didn't want "feature/" to represent a directory level, I first URL encode the branch name, resulting in "feature%2F...". I do this in our Java-based builds, and this works perfectly fine.

Since I was pretty certain by this point that I knew this was somehow causing the problem, I changed the script to instead simply replace "/" with "-". This fixed the problem. I have no clue why.