Load CSV memory pool error: "The memory pool limit was exceeded"

Neo4j Graph Platform
1

I try to load a large CSV to Neo4j 5.2 with cypher-shell and run into an error I have not seen before:

Unable to complete transaction.: The memory pool limit was exceeded. The corresponding setting can be found in the error message

The Neo4j logs are empty, the heap size is 5 times larger than the total file size. The transaction should even actually fit into memory but I use CALL ... IN TRANSACTIONS.

The query should not run into an Eager operator:

LOAD CSV WITH HEADERS FROM 'file:///omop/CONCEPT_RELATIONSHIP_clean.csv' AS line FIELDTERMINATOR ','

CALL {
  WITH line
  MATCH (source:Concept { concept_id: line.concept_id_1 })
  MATCH (target:Concept { concept_id: line.concept_id_2 })
  CREATE (source)-[r:VOCAB_REL]->(target)
  SET r.type = line.relationship_id, r.valid_start_date = line.valid_start_date, r.valid_end_date = line.valid_end_date, r.invalid_reason = line.invalid_reason

} IN TRANSACTIONS;

Any ideas what might cause the error?

2

@martin2

but yo have not specified a transaction batch size.

Change

} IN TRANSACTIONS;

to

} IN TRANSACTIONS OF 1000 ROWS

https://neo4j.com/docs/cypher-manual/current/clauses/call-subquery/#_batching

3

Thanks! There is a default for the transaction size: "If omitted, the default batch size is 1000 rows"