Error with materialize=true and threads in Redshift: psycopg2.errors.InFailedSqlTransaction

[amankushwaha25]

I am using Redshift. When the materialise =True and threads are set; I am getting an error:
psycopg2.errors.InFailedSqlTransaction

The exact command is below:
data-diff "database1" "table1" "database2" "table2" --no-tracking -k id -c columnName --stats --json -j 8 --materialize-to-table differenctOutputTable

[amankushwaha25]

@alex-mirkin @erezsh @vmatt
Let me know if you guys are facing the issue too. I would love to help in resolving it?

[erezsh]

Thanks for reporting the error. I don't know why it's happening, but if it happens only when both materialise and threads are set, it's worth checking if the error is caused by a write-lock. I'm guessing writing to the same table from multiple threads is causing write-locks to lock on each other, leading the queries to fail or timeout.