# Failure when calling apoc.periodic.iterate multiple times

**URL:** <https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841>\
**Category:** Procedures & APOC\
**Created:** [June 4, 2019, 10:22am UTC](https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841 "2019-06-04T10:22:53Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mandyreal](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/mandyreal/32/5101_2.png) [@mandyreal](https://community.neo4j.com/u/mandyreal)\
**Post date:** [June 4, 2019, 10:22am UTC](https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841/1 "2019-06-04T10:22:54Z")

</div>

I’m running the following cypher query using neo4j-drivier in a nodejs app to several times (different combinations of each node label):

CALL apoc.periodic.iterate("  
CALL apoc.load.json('file://{fileName}') YIELD value AS line RETURN line ", " match (a:{item[0]} {ID:line.startid}), (b:{item[1]} {ID:line.endid}) merge (a)-[{label.toLowerCase()}:{label} {ID:line.id}]-\>(b) {updateStatements.join("")} return ${label.toLowerCase()}  
",  
{  
batchSize: 1000,  
iterateList: true,  
parallel: false,  
params: {},  
concurrency: 50,  
retries: 0  
}  
)

After some time, App terminates with the following error:

query failed Failed to invoke procedure `apoc.periodic.iterate`: Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@ad566f2 rejected from java.util.concurrent.ThreadPoolExecutor@6e8e6e59[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8436]  
(node:17822) UnhandledPromiseRejectionWarning: Neo4jError: Failed to invoke procedure `apoc.periodic.iterate`: Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@ad566f2 rejected from java.util.concurrent.ThreadPoolExecutor@6e8e6e59[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8436]

Any idea on which parm/config setting needs to be tweaked? Im running an app on a MB pro with 8GB ram. Json files vary from 300k to 1M++ records. Thanks.

---

<div class="post-metadata">

**Author:** ![andrew\_bowman](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/andrew_bowman/32/73_2.png) [@andrew\_bowman](https://community.neo4j.com/u/andrew_bowman)\
**Post date:** [June 4, 2019, 3:39pm UTC](https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841/2 "2019-06-04T15:39:40Z")

</div>

Do you have indexes on the relevant label + property combinations? If not that could be heap intensive. You may also consider lowering your batch size.

---

<div class="post-metadata">

**Author:** ![mandyreal](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/mandyreal/32/5101_2.png) [@mandyreal](https://community.neo4j.com/u/mandyreal)\
**Post date:** [June 4, 2019, 3:57pm UTC](https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841/3 "2019-06-04T15:57:39Z")

</div>

Thanks Andrew. That’s right, there are indexes on the label + property. I've also tried running with lower and higher batch size but still get the same error. Prior to these queries, the nodes are loaded first. So these queries are for loading the relationships.

A sample generated query looks something like this:

CALL apoc.periodic.iterate("  
CALL apoc.load.json('file:///Path/File.json')  
YIELD value AS line RETURN line  
", "  
match (a:Company), (b:Person) where a.ID = line.startid and b.ID = line.endid  
merge (a)-[relationship:RelType {ID: line.id}]-\>(b) FOREACH(\_ IN CASE WHEN trim(line.description) \<\> '' THEN [1] ELSE END | SET relationship.description = line.description ....  
return relationship

Indexes exist on :Person(ID) and :Company(ID), and there are about over 1million nodes for both Person and Company.

---

<div class="post-metadata">

**Author:** ![andrew\_bowman](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/andrew_bowman/32/73_2.png) [@andrew\_bowman](https://community.neo4j.com/u/andrew_bowman)\
**Post date:** [June 4, 2019, 4:01pm UTC](https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841/4 "2019-06-04T16:01:18Z")

</div>

You'll probably not want to return anything, as the batched query in APOC won't do anything with the returned values. And you can forgo the FOREACH and just use filtering:

```auto
match (a:Company), (b:Person) 
where a.ID = line.startid and b.ID = line.endid
merge (a)-[relationship:RelType {ID: line.id}]->(b)
WITH relationship, line
WHERE line.description <> ''
SET relationship.description = line.description

```

---

<div class="post-metadata">

**Author:** ![mandyreal](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/mandyreal/32/5101_2.png) [@mandyreal](https://community.neo4j.com/u/mandyreal)\
**Post date:** [June 4, 2019, 4:58pm UTC](https://community.neo4j.com/t/failure-when-calling-apoc-periodic-iterate-multiple-times/7841/5 "2019-06-04T16:58:10Z")

</div>

Thanks Andrew for the suggestion. The queries ran significantly faster after using WITH instead of FOREACH and removing the RETURN. I re-ran the queries using smaller batchsize of 500 but still failed with same error. The error seems to come up when processing/iterating through huge files (around 300k - 1mil records).