Skip to content

Z_BUF_ERROR on GET of https://intljs.rmtag.com/115557.ct.js through proxy #1631

@johnrichardrinehart

Description

@johnrichardrinehart

Describe the bug

  • Node.js version: 15.3.0
  • OS & version: Ubuntu 20.04

Fetching with got using a proxy Agent (hpagent) results in a Z_BUF_ERROR (unexpected content length). But, fetching with curl -x <agent> https://intljs.rmtag.com/115557.ct.js --compressed works just fine.

Actual behavior

Stack trace:

RequestError: unexpected end of file
    at PassThrough.<anonymous> (/home/john/Downloads/repos/puppeteer_api/.yarn/unplugged/got-npm-11.8.1-7775de79cb/node_modules/got/dist/source/core/index.js:797:31)
    at Object.onceWrapper (node:events:483:26)
    at PassThrough.emit (node:events:388:22)
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:80:21)
    at Zlib.zlibOnError [as onerror] (node:zlib:182:17) {
  name: 'ReadError',
  code: 'Z_BUF_ERROR',
  timings: {
    start: 1613893520933,
    socket: 1613893532290,
    lookup: 1613893532290,
    connect: 1613893532290,
    secureConnect: 1613893533013,
    upload: 1613893533014,
    response: 1613893534192,
    end: 1613893537248,
    error: undefined,
    abort: 1613893537257,
    phases: {
      wait: 11357,
      dns: 0,
      tcp: 0,
      tls: 723,
      request: 1,
      firstByte: 1178,
      download: 3056,
      total: 16324
    }
  }
}

Expected behavior

Fetching the asset wouldn't produce a Z_BUF_ERROR.

Code to reproduce

Here is a console.log of the arguments passed to got(url, options) (console.log({url: url, options: options})).

{
url: "https://intljs.rmtag.com/115557.ct.js",
options: {
  cookieJar: CookieJar {
    rejectPublicSuffixes: true,
    enableLooseMode: false,
    allowSpecialUseDomain: false,
    store: { idx: {} },
    prefixSecurity: 'silent',
    _cloneSync: [Function (anonymous)],
    _importCookiesSync: [Function (anonymous)],
    getCookiesSync: [Function (anonymous)],
    getCookieStringSync: [Function (anonymous)],
    getSetCookieStringsSync: [Function (anonymous)],
    removeAllCookiesSync: [Function (anonymous)],
    setCookieSync: [Function (anonymous)],
    serializeSync: [Function (anonymous)]
  },
  method: 'GET',
  body: undefined,
  headers: {
    'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"',
    'sec-ch-ua-mobile': '?0',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4403.0 Safari/537.36',
    accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'accept-encoding': 'gzip, deflate, br',
    host: 'intljs.rmtag.com',
    'sec-fetch-mode': 'navigate',
    'sec-fetch-site': 'none',
    'sec-fetch-user': '?1'
  },
  agent: {
    https: HttpsProxyAgent {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      defaultPort: 443,
      protocol: 'https:',
      options: [Object],
      requests: {},
      sockets: {},
      freeSockets: {},
      keepAliveMsecs: 1000,
      keepAlive: true,
      maxSockets: 256,
      maxFreeSockets: 256,
      scheduling: 'lifo',
      maxTotalSockets: Infinity,
      totalSocketCount: 0,
      maxCachedSessions: 100,
      _sessionCache: [Object],
      proxy: [URL],
      [Symbol(kCapture)]: false
    }
  },
  responseType: 'buffer',
  maxRedirects: 15,
  throwHttpErrors: false,
  ignoreInvalidCookies: true,
  followRedirect: false,
  timeout: 30000,
  decompress: true
}
}

Checklist

  • I have read the documentation. **I can "fix" this with decompress: false, but that's undesirable for obvious reasons.
  • I have tried my code with the latest version of Node.js and Got. ** I'm using got v11.8.1. **

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions