Failed to execute 'postMessage' on 'Worker' when worker:true and using transformHeader function

[UlTriX]

Using Papa v5.0.0-beta.0 trying to use transformHeader config as follow:

Papa.parse(file, { header: true, preview: 1000, worker: true, transformHeader: function (header) { return header; } });

and getting error:

Error: Failed to execute 'postMessage' on 'Worker': function (header) { return header; } could not be cloned.

What am I missing?

Thank you

[nbrysondev]

I have the same issue for the transform callback. Is this feature fully implemented?

[fabricedesre]

Functions are not transferable so in worker mode a different strategy should be used. I can think of 2 options:

• postMessage to the main thread to run the transformation functions (likely slow).
• define the transform functions in their own file and import it in the worker with importScripts('...');

[rbrundritt]

It is possible to pass a simple function to a web worker, although a bit hackish. You would convert the function to a string then use eval in the work to re-inflate it. For example:

From the UI thread.

var headerTransform = function(h) { return h.trim(); };

worker.postMessage({
      transformHeader: headerTransform.toString()
});

In the worker:

self.onmessage = function (e) {
    if (e.data.transformHeader) {
        eval('var transformHeader= ' + e.data.transformHeader);

       //Use the transform header.
        transformHeader(e.data.data);
    }

    //...
};

[pokoli]

@rbrundritt wow, this is really tricky but I do not think there is another workarround.

Do you think we can document it and thus close this issue?

[fabricedesre]

That won't work if you use a CSP that prevents eval() to protect against XSS though...

[AlayVora]

Going through same issue. Do we have any recommended solution or documentation related to this issue?

[ac-ih]

@mholt any eta or leads on how/when this issue will be resolved? Or perhaps an easier workaround (the one by imReker is risky)?
@pokoli

[mholt]

Sorry, I haven't actually been maintaining this library for several years now. @pokoli is your person! :)

[0biWanKenobi]

ping

[pokoli]

@0biWanKenobi do you have any proposal to solve the issue?

[Fanna1119]

any update on this?

[KoalaBear84]

I'm getting the same for beforeFirstChunk and chunk. No idea how to solve it.

Papa.parse(event.target.files[0], {
	worker: true,
	header: true,
	dynamicTyping: true,
	beforeFirstChunk: function (chunk) {
		debugger;
		return chunk;
	},
	chunk: function chunk(results, parser) {
		debugger;
	}
});

[KoalaBear84]

I now do somewhat understand the issue:
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm#things_that_dont_work_with_structured_clone

Function cannot be 'posted' / used in a Worker.

I see that PapaParse does something special for only functions like step, chunk, complete and error, and not for the other function parameters:

PapaParse/papaparse.js

Lines 213 to 216 in e2d570e

	_config.step = isFunction(_config.step);
	_config.chunk = isFunction(_config.chunk);
	_config.complete = isFunction(_config.complete);
	_config.error = isFunction(_config.error);

But I also see when I'm not using beforeFirstChunk, it 'works', but it does NOT chunk, it just get's back with 57K rows, even though I have limited the chunk on 10240 bytes:

Papa.RemoteChunkSize = Papa.LocalChunkSize = 10 * 1024;

Oh, nvm, when you use a worker I guess the above chunksizes are now available and you need to fill chunkSize in the config, then it works

Don't have enough knowledge to fix it. I see there some bad attempts made for the transform in a push request 😅