URL parse fails

[pt-rick]

• Version: 11.6.0
• Platform: FreeBSD 11.2-RELEASE-p4
• Subsystem: url

Simple parse of a URL fails...

test.js
const url = require('url');

const test = new URL("https://www.test.com/test");

error
internal/url.js:243
throw error;
^

TypeError [ERR_INVALID_URL]: Invalid URL: https://www.test.com/test
at onParseError (internal/url.js:241:17)
at new URL (internal/url.js:319:5)
at Object. (/shared/rick/pushy/test.js:3:14)
at Module._compile (internal/modules/cjs/loader.js:721:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:732:10)
at Module.load (internal/modules/cjs/loader.js:620:32)
at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
at Function.Module._load (internal/modules/cjs/loader.js:552:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:774:12)
at executeUserCode (internal/bootstrap/node.js:342:17)

When using the deprecated url.parse, this is the result:

test.js
const url = require('url');

const test = url.parse("https://www.test.com/test");

error
url.js:385
this.hostname = toASCII(this.hostname, true);
^

Error: Cannot convert name to ASCII
at Url.parse (url.js:385:23)
at Object.urlParse [as parse] (url.js:149:13)
at Object. (/shared/rick/pushy/test.js:3:18)
at Module._compile (internal/modules/cjs/loader.js:721:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:732:10)
at Module.load (internal/modules/cjs/loader.js:620:32)
at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
at Function.Module._load (internal/modules/cjs/loader.js:552:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:774:12)
at executeUserCode (internal/bootstrap/node.js:342:17)

[mscdex]

Both work for me on node master and node v10.15.0.

[devsnek]

new URL(...) throws for me, as it should, since test.com/test is treated like a relative url, and URL objects are absolute.

url.parse(...) returns an object with pathname = "test.com/test", which is expected.

[bnoordhuis]

I'll close this out, can reopen if necessary.

[pt-rick]

This is too simple to not work so was thinking it perhaps has something to do with my environment. But even when removing everything from node_modules, I get this error. So perhaps is there some configuration or environment setting or missing package that I should look for that might be causing this problem? Node on Free BSD perhaps?

To @mscdex comment, if I make the url a relative url (without 'https://') it doesn't throw for me if I use the deprecated url.parse but does, as you say, throw on new URL. However, both throw for me on an absolute URL as shown in the code above.

[sam-github]

% node -p 'require("url").parse("https://www.test.com/test")'; node --version
Url {
  protocol: 'https:',
  slashes: true,
  auth: null,
  host: 'www.test.com',
  port: null,
  hostname: 'www.test.com',
  hash: null,
  search: null,
  query: null,
  pathname: '/test',
  path: '/test',
  href: 'https://www.test.com/test' }
v11.6.0

Works for me. Could be something about your system, but its hard to know. Try to strip back to something as simple as possible, like above. Maybe even build node from source if you are using a package. Its hard to know what else to suggest.

[pt-rick]

Thanks @sam-github for some ideas. I'm actually using a library that is calling url.parse so I managed to strip this down to as far as I can - 2 lines and no node_modules. Doing a cut and paste of your command, I get this:

node -p 'require("url").parse("https://www.test.com/test")'; node --version

url.js:385
this.hostname = toASCII(this.hostname, true);
^

Error: Cannot convert name to ASCII
at Url.parse (url.js:385:23)
at Object.urlParse [as parse] (url.js:149:13)
at [eval]:1:16
at Script.runInThisContext (vm.js:123:20)
at Object.runInThisContext (vm.js:312:38)
at Object. ([eval]-wrapper:6:22)
at Module._compile (internal/modules/cjs/loader.js:721:30)
at evalScript (internal/bootstrap/node.js:720:27)
at executeUserCode (internal/bootstrap/node.js:313:7)
at startExecution (internal/bootstrap/node.js:276:5)
v11.6.0

In stepping through the debugger, I now can see that toASCII doesn't seem to be getting resolved correctly and this is causing it to fail. The definition on line 24 or url.js is as follows:

const { toASCII } = internalBinding('config').hasIntl ? internalBinding('icu') : require('punycode');

internalBinding('config').hasIntl is undefined so it is trying to destruct to punycode but that seems to be where the failure is occurring as 'toASCII' is defined as ƒ toASCII() { [native code] }. I see that punycode has been deprecated so not sure if that is the issue.

Likewise when I change my code to new URL("https://www.test.com/test"), it fails on the call to 'parse' on line 319 of internal/url.js. 'parse' is defined as ƒ parse() { [native code] } and that throws the error there.

[ChALkeR]

@bnoordhuis perhaps this needs reopening?

[sam-github]

That's more interesting. Could you give your node -p process.versions output? I wonder if what you are running into is because of ICU? Perhaps with the right configure options it would be reproducable on other sytems, there are a number of ICU build/configure options.

Also, where did you get node from? We don't distribute pre-built binaries for FreeBSD. If its a pre-built package, it might be linked against an ICU library that isn't compatible with node. I would suggest building node from source yourself and seeing if you can repro with your own node, that might allow you to isolate at what point this problem got introduced.

[sam-github]

If this is a problem with how someone built node, it might not be a node issue, but I'll reopen while @pt-rick gathers a bit more information, since it looks like he can reproduce with just node and no external deps.

[pt-rick]

@sam-github here's the answer to the process versions. I'll have to get back to you about the FreeBSD package and source later.
{ node: '11.6.0',
v8: '7.0.276.38-node.13',
uv: '1.24.1',
zlib: '1.2.11',
ares: '1.14.0',
modules: '67',
nghttp2: '1.35.1',
napi: '3',
llhttp: '1.0.1',
http_parser: '2.8.0',
openssl: '1.1.0j',
icu: '63.1',
unicode: '11.0',
cldr: undefined,
tz: undefined }