Cannot read properties of undefined (reading 'isServer')

[fragglebob]

Current behavior

Since upgrading to Cypress 10, we are getting crashes in our CI with the following error:

Cannot read properties of undefined (reading 'isServer')
TypeError: Cannot read properties of undefined (reading 'isServer')
    at TLSWrap.onerror (node:_tls_wrap:411:27)
  cypress:network:agent addRequest called { isHttps: true, href: 'https://api.cypress.io/exceptions' } +49ms
  cypress:network:agent got family { family: 4, href: 'https://api.cypress.io/exceptions' } +0ms
  cypress:server:browsers killing browser process +9s
  cypress:server:browsers:chrome closing remote interface client +9s
  cypress:server:browsers:chrome closing chrome +0ms
  cypress:cli child event fired { event: 'exit', code: 1, signal: null } +2m
  cypress:cli Stopping Xvfb +2m

This is happening almost ever run we do. Hover we are running in 30 machines parallel, and the most I've seen is two machines fail in the same test run.

I found an issue nodejs/node#41501 and PR nodejs/node#41523 that relate to this error above, which seems to have been fixed in Node v16.15.0.

I set up for our tests to run in a custom container image today that is using Node v16.15.1, so I could see if that helps. However I've now realised that cypress has an internal version of node which is being used, v16.13.2 which still has this bug in it from Node.

Desired behavior

Ideal if should not being crashing when this error happens.

As this is an issue with how node is happening an error internally, there is probably some other error happening, but it's breaking before we can see what that is.

Test code to reproduce

Not sure how to reproduce this as I can't work out what is causing the issue.

In nodejs/node#41501 they weren't able to workout what the exact cause was, other than something wrong with the network somewhere.

Cypress Version

10.1.0

Other

No response

[AtofStryker]

Hey @fragglebob. Thank you for opening an issue. Have you found a way to reproduce this locally in a reproduction repo or does this only happen in CI with parallel jobs?

Not ideal, but I am wondering if you can verify if this works by creating a custom image similar to this and bumping the node version to 16.15.1, and building a browser image to consume and run your tests in CI.

[fragglebob]

Unfortunately I've not seen it locally or anyone else.

I had already tried that suggestion of a custom images with v16.15.1 and the same browsers.

I think the version being used by cypress's server is the version bundled in electron. I can see this from a couple of logs that report the node version. However it does start sub processes on the system version.

Is there a way to skip the whole electron start up?

I see electron v20 will have node v16.15.0, but it's still in alpha.

We have some verbose logging on, and I've been trying to see if there is some pattern to when this happens, but not found anything yet.

[darrinmn9]

I am having a similar issue. For me, It occurs when upgrading from 9.6.1 -> 9.7.0 (electron 100). For my test suite, I can consistently reproduce this issue locally on my mac and in aws codebuild CI docker image - both of which use node v16.14.0. I am not parallelizing tests. What is interesting is that it always fails on the 2nd test in my test suite (the first test passes successfully), but it isn't specific to the actual 2nd test spec. I can move test specs around, comment/delete specs, and it still consistently fails on the 2nd spec that runs.

TypeError: Cannot read properties of undefined (reading 'isServer')
    at TLSWrap.onerror (node:_tls_wrap:411:27)

[AtofStryker]

I think the version being used by cypress's server is the version bundled in electron. I can see this from a couple of logs that report the node version. However it does start sub processes on the system version.

You are right about this. The method I proposed won't work 😞 .

Is there a way to skip the whole electron start up?

I don't think so. Even in run mode, the electron app still starts and runs a few shared processes.

@flotwig @tbiethman do you all have any ideas on this one? It almost seems like we would need to wait until electron releases v20 to update the bundled node version.

[devhid]

I'm having the same issue:

Cannot read properties of undefined (reading 'isServer')
TypeError: Cannot read properties of undefined (reading 'isServer')
    at TLSWrap.onerror (node:_tls_wrap:411:27)

POST /logging_client_events 200 95.593 ms - -
The Test Runner unexpectedly exited via a exit event with signal SIGABRT

Please search Cypress documentation for possible solutions:

https://on.cypress.io

Check if there is a GitHub issue describing this crash:

https://github.com/cypress-io/cypress/issues

Consider opening a new issue.

----------

Platform: darwin-x64 (21.4.0)
Cypress Version: 10.1.0

This is a (fairly) minimal reproducible example: https://github.com/devhid/cypress-v10-issue

In my case, I'm trying to run Lighthouse on some sample websites for both mobile and desktop viewports, but I get the mentioned error at the end of the test run.

My Lighthouse tests run fine on Cypress v9.6.0:

❯ npx cypress --version    
Cypress package version: 9.6.0
Cypress binary version: 9.6.0
Electron version: 15.5.1
Bundled Node version: 16.5.0

But not on Cypress v10.1.0:

❯ npx cypress --version
Cypress package version: 10.1.0
Cypress binary version: 10.1.0
Electron version: 18.0.4
Bundled Node version: 16.13.2

I'm using the following Node / NPM versions:

❯ node --version  
v16.14.2

❯ npm --version   
8.10.0

Steps:

1. npm i
2. npx cypress open
3. Run lighthouse.cy.js

[fragglebob]

I have been able to reproduce the error in the repo @devhid has shared locally. It's doesn't happen every time for me, but it happens often enough.

[fragglebob]

I started Cypress electron app directly with an debugger attached and have some extra info on the underlying error, that node is struggling to handle.

code: "ERR_SSL_SSLV3_ALERT_CLOSE_NOTIFY"
function: "OPENSSL_internal"
library: "SSL routines"
reason: "SSLV3_ALERT_CLOSE_NOTIFY"
message: "140252445026280:error:100003e8:SSL routines:OPENSSL_internal:SSLV3_ALERT_CLOSE_NOTIFY:../../third_party/boringssl/src/ssl/tls_record.cc:594:SSL alert number 0\n"
stack: "Error: 140252445026280:error:100003e8:SSL routines:OPENSSL_internal:SSLV3_ALERT_CLOSE_NOTIFY:../../third_party/boringssl/src/ssl/tls_record.cc:594:SSL alert number 0\n\n\n"

[fragglebob]

Ok, I've done some more research into this error.

It seems the owner of TLSWrap was being set to something that wasn't a TLSSocket by a change in Node v16.7.0 nodejs/node#38468 which was later reverted in version v16.14.0.

This is happening because the http/s Agent is reusing an existing socket, and this internal change the owner, so when an error does happen it breaks. The only way I can think to avoid a state in these node versions is to not use keepAlive and avoid connection reuse. Which would probably degrade performance.

[flotwig]

@fragglebob I'm not surprised to find out it's "something wacky to do with Keep-Alive", the interplay of Keep-Alive and Agents is a really tricky area.

Sounds like the solution is to bump Cypress's internal Node.js version. Looks like Electron 19.0.0 has Node 16.14.2, so it should avoid this.

Would still be nice to have a cleaner reproduction, but at least we have @devhid's sorta flaky repro to test if Electron 19 seems to resolve the issue.

The only way I can think to avoid a state in these node versions is to not use keepAlive and avoid connection reuse. Which would probably degrade performance.

Yeah, removing Keep-Alive would really hinder proxy performance.

[fragglebob]

@flotwig yep, all that async stuff is complicated.

Yep, a bump to Electron 19 should be good. Which is nicer than having to wait for version 20.

I kinda came up a little script based on some unit tests in node, where I could quickly see what was going on in the owner in different versions.