Hi everyone,
We’re developing a tool that uses Neo4j (Community Edition) to analyze workflows represented as graphs. To analyse the data, we first import the data into the Neo4j. The process consists of:
- Importing data from SQL databases into Neo4j (first ~10 queries)
- Preparation steps on the data (setting properties, aggregations, helper nodes/relationships)
Each import starts with an empty Neo4j instance. The full preparation phase includes ~400 Cypher queries executed in a fixed order. All queries:
- Mutate the graph (i.e., write/update)
- Use
CALL { ... } IN TRANSACTIONS OF 5000 ROWS
- Some use
IN CONCURRENT TRANSACTIONS to speed things up
This usually completes in ~4 minutes. However, we’re encountering an intermittent issue:
Sometimes, one query (usually after query 300+) hangs for 1–2 hours, then completes.
However, if we cancel and re-run it (with the same data and parameters), it finishes in seconds.
Some context:
- The problem is not tied to dataset size; even small datasets can trigger it.
- The same dataset may work fine one time, and hang the next.
- The problem only occurs after the SQL import phase, once all data is already in Neo4j. Thus, possible connection problems are not the cause.
- The query that hangs varies, and doesn’t necessarily involve complex logic.
- Not all queries that hang are using CONCURRENT TRANSACTIONS. Thus, I think it is not related to a deadlock.
We’ve noticed that adding a small delay between phases reduces the frequency of these issues. Even though we could do this, it would be more like a workaround then a solution.
Has anyone experienced similar symptoms?
- What could be the reason for the occasionally long running queries.
- What are the best practices for concurrency and batching in Community Edition?
- Are there tuning strategies (other than switching to Enterprise) that help mitigate this?
Thanks for any insights you can offer!
I would suggest you provide more info:
- versions of everything
- if you know which query, an execution plan (
EXPLAIN)
Yes, sorry forgot the versions:
- Currently we use Neo4j 5.24.2, but we saw the same problem also with older versions
- Unfortunately, it does not always hang on the same query. But as an example this is the query that hanged the last time:
MATCH (:`Dimension` {id: <id>})<-[:`PART_OF_DIMENSION`]-(caseId)
WHERE exists((caseId)-[:`HAS_VIOLATIONS`]->())
WITH caseId
CALL {
WITH caseId
OPTIONAL MATCH (caseId)<-[:`PART_OF_CASE`]-()-[r1:`BEFORE`|`BEFORE_START` {suspended: true}]->()-[r2:`BEFORE` {reverse: true}]->()
SET r1.disarray = not r1.simplification,
r2.disarray = not r2.simplification
SET r1.violation = r1.disarray, r2.violation = r2.disarray
WITH caseId
OPTIONAL MATCH (caseId)<-[:`PART_OF_CASE`]-()-[r1:`BEFORE` {reverse: true}]->()-[r2:`BEFORE`|`BEFORE_END` {suspended: true}]->()
SET r1.disarray = not r1.simplification,
r2.disarray = not r2.simplification
SET r1.violation = r1.disarray, r2.violation = r2.disarray
}
IN TRANSACTIONS OF 5000 ROWS
Cypher 5
Planner COST
Runtime SLOTTED
Runtime version 5.24
+---------------------+----+-------------------------------------------------------------------------------------------------+----------------+
| Operator | Id | Details | Estimated Rows |
+---------------------+----+-------------------------------------------------------------------------------------------------+----------------+
| +ProduceResults | 0 | | 700 |
| | +----+-------------------------------------------------------------------------------------------------+----------------+
| +EmptyResult | 1 | | 700 |
| | +----+-------------------------------------------------------------------------------------------------+----------------+
| +TransactionForeach | 2 | IN TRANSACTIONS OF $autoint_1 ROWS ON ERROR FAIL | 700 |
| |\ +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 3 | r2.violation = r2.disarray | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 4 | r1.violation = r1.disarray | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Eager | 5 | read/set conflict for property: disarray (Operator: 6 vs 3, and 3 more conflicting operators) | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 6 | r2.disarray = NOT r2.simplification | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 7 | r1.disarray = NOT r1.simplification | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Eager | 8 | read/set conflict for property: disarray (Operator: 6 vs 17, and 5 more conflicting operators) | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Apply | 9 | | 700 |
| | |\ +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Optional | 10 | caseId | 700 |
| | | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Filter | 11 | NOT r2 = r1 AND r2.suspended = true | 2 |
| | | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Expand(All) | 12 | (anon_10)-[r2:BEFORE|BEFORE_END]->(anon_11) | 31 |
| | | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Filter | 13 | r1.reverse = true | 43 |
| | | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Expand(All) | 14 | (anon_9)-[r1:BEFORE]->(anon_10) | 217 |
| | | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Expand(All) | 15 | (caseId)<-[anon_8:PART_OF_CASE]-(anon_9) | 322 |
| | | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | | +Argument | 16 | caseId | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 17 | r2.violation = r2.disarray | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 18 | r1.violation = r1.disarray | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Eager | 19 | read/set conflict for property: disarray (Operator: 21 vs 17, and 5 more conflicting operators) | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 20 | r2.disarray = NOT cache[r2.simplification] | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +SetProperty | 21 | r1.disarray = NOT cache[r1.simplification] | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Optional | 22 | caseId | 700 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +CacheProperties | 23 | cache[r1.simplification], cache[r2.simplification] | 2 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Filter | 24 | NOT r2 = r1 AND r2.reverse = true | 2 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Expand(All) | 25 | (anon_6)-[r2:BEFORE]->(anon_7) | 8 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Filter | 26 | r1.suspended = true | 12 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Expand(All) | 27 | (anon_5)-[r1:BEFORE|BEFORE_START]->(anon_6) | 234 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Expand(All) | 28 | (caseId)<-[anon_4:PART_OF_CASE]-(anon_5) | 322 |
| | | +----+-------------------------------------------------------------------------------------------------+----------------+
| | +Argument | 29 | caseId | 700 |
| | +----+-------------------------------------------------------------------------------------------------+----------------+
| +Filter | 30 | getDegree((caseId)-[:HAS_VIOLATIONS]->()) > 0 | 700 |
| | +----+-------------------------------------------------------------------------------------------------+----------------+
| +Expand(All) | 31 | (anon_0)<-[anon_1:PART_OF_DIMENSION]-(caseId) | 934 |
| | +----+-------------------------------------------------------------------------------------------------+----------------+
| +Filter | 32 | anon_0.id = $autostring_0 | 1 |
| | +----+-------------------------------------------------------------------------------------------------+----------------+
| +NodeByLabelScan | 33 | anon_0:Dimension | 10 |
+---------------------+----+-------------------------------------------------------------------------------------------------+----------------+
Total database accesses: ?
- The query execution is triggered via a Java program, with Spring Boot 3.3.4
- The queries are created via the Cypher-DSL 2023.9.7 (we are not using Spring Data Neo4j)
