Connection reset when importing data from large csv

Neo4j Graph Platform Import / Export
, ,
1

I'm using Neo4j Server Version: 5.14.0 (community)
Neo4j Browser Version: 5.12.0

I'm trying to import data from a large csv present in my Nexus, below is the cypher query. While executing this query it run's for an hour and it fails with Connection reset error after inserting half of the csv

CALL apoc.periodic.iterate("LOAD CSV WITH HEADERS FROM 'https://nexus-url/repository/xxxx/dbextract.csv' AS row RETURN row", "MERGE (d:Database {dbname: row.database}) MERGE (s:Schema {schemaname: row.schema}) MERGE (t:Table {tablename: row.table}) MERGE (d)-[:HAS_SCHEMA]->(s) MERGE (s)-[:HAS_TABLE]->(t)",{batchSize: 2000, parallel: false, retires:3})

ERROR: Failed to invoke procedure 'apoc.periodic.iterate': Caused by: java.net.SocketException: Connection reset}

2

It's probably a memory issue.

Did you create constraints for :Database(dbname) and :Schema(schemaname) and Table(tablename) ?

Can you see anything in your database logs why it failed?
Did you change the memory config to allow for a bit more memory? I think the default config is still too small (allow for 1-2GB heap memory)

In 5.x you can now also use the built-in functionality.

LOAD CSV WITH HEADERS FROM '[https://nexus-url/repository/xxxx/dbextract.csv ](https://nexus-url/repository/xxxx/dbextract.csv)' AS row
CALL { WITH row
MERGE (d:Database {dbname: row.database}) 
MERGE (s:Schema {schemaname: row.schema}) 
MERGE (t:Table {tablename: row.table}) 
MERGE (d)-[:HAS_SCHEMA]->(s) 
MERGE (s)-[:HAS_TABLE]->(t)
} IN TRANSACTIONS OF 2000 ROWS;