I have written an ingestion script in python (using neo4j driver) to wrangle some data and construct cypher CREATE queries.
My code runs fine as a single-threaded program but I would like to parallelize and use threading to speed it up. I have added some code to parallelize the operations (using the concurrent.futures.ProcessPoolExecutor module) but when I run my code again with multiple workers I get intermittent Neo.TransientError.Transaction.DeadlockDetected errors. Note for each "worker" I am using a separate session from a global driver object.
Is there a way to prevent these locks so that I can parallelize this ingestion script?
Thanks