# How to periodic commit apoc.load.json + can apocs be nested?

**URL:** <https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786>\
**Category:** Procedures & APOC\
**Tags:** apoc\
**Created:** [August 20, 2020, 9:59am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786 "2020-08-20T09:59:16Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![daniel.muellner](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/daniel.muellner/32/4441_2.png) [@daniel.muellner](https://community.neo4j.com/u/daniel.muellner)\
**Post date:** [August 20, 2020, 9:59am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/1 "2020-08-20T09:59:16Z")

</div>

I am trying to import a 80mb json file with around 2mio lines.  
Therefore i am running into a out of heap space error.

To prevent the error i wanted to use periodic commit which i was not successfull with.

Thats the command i tried:

```auto
CALL apoc.periodic.commit(
"CALL apoc.load.json('file:/master.json')
YIELD value
MERGE (p:Product {name:value.name})
WITH p, value 
UNWIND value.categories as cat
MERGE (c:Categorie {name:cat})
MERGE (c)-[r:INHERITS]->(p)",
{batchSize:100, parallel:true})

```

Is it even possible to nest two apocs?  
And how can i achieve batch import with json files in neo4j?

---

<div class="post-metadata">

**Author:** ![Cobra](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/cobra/32/35243_2.png) [@Cobra](https://community.neo4j.com/u/Cobra)\
**Post date:** [August 20, 2020, 10:23am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/2 "2020-08-20T10:23:44Z")

</div>

Hello @daniel.muellner 🙂

> [@daniel.muellner](#):
>
> s it even possible to nest two apocs?

Yes it's possible 🙂

Did you have a look at [apoc.periodic.iterate()](https://neo4j.com/labs/apoc/4.1/graph-updates/periodic-execution/)?

Regards,  
Cobra

---

<div class="post-metadata">

**Author:** ![daniel.muellner](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/daniel.muellner/32/4441_2.png) [@daniel.muellner](https://community.neo4j.com/u/daniel.muellner)\
**Post date:** [August 20, 2020, 11:24am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/3 "2020-08-20T11:24:46Z")

</div>

Thank you for your confirmation.  
Although i read the docs I was confused because it does not show a nested apoc call and i ran into weird errors as i mixed the syntax of periodic.commit() and periodic.iterate().

The corrected script is:  
(i added the limit and removed the batchsize params)

```auto
CALL apoc.periodic.commit(
"CALL apoc.load.json('file:/master_small.json')
YIELD value
MERGE (p:Product {name:value.name})
WITH value, p limit $limit
UNWIND value.categories as cat
MERGE (c:Categorie {name:cat})
MERGE (c)-[r:INHERITS]->(p)",
{limit:100})

```

Running this code throws this error:

```auto
{
  "No element found in java.util.Arrays$ArrayItr@224f659a": 1
}

```

When i run it without the apoc.periodic.commit() it works with no errors though:

```auto
CALL apoc.load.json('file:/master_small.json')
YIELD value
MERGE (p:Product {name:value.name})
WITH value, p 
UNWIND value.categories as cat
MERGE (c:Categorie {name:cat})
MERGE (c)-[r:INHERITS]->(p)

```

---

<div class="post-metadata">

**Author:** ![Cobra](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/cobra/32/35243_2.png) [@Cobra](https://community.neo4j.com/u/Cobra)\
**Post date:** [August 20, 2020, 11:31am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/4 "2020-08-20T11:31:45Z")

</div>

I never see this error but it looks like something is empty 🤔

Can you try with [apoc.periodic.iterate()](https://neo4j.com/labs/apoc/4.1/graph-updates/periodic-execution/)?

I don't think commit is the right function for what you want to 🤔

---

<div class="post-metadata">

**Author:** ![daniel.muellner](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/daniel.muellner/32/4441_2.png) [@daniel.muellner](https://community.neo4j.com/u/daniel.muellner)\
**Post date:** [August 20, 2020, 11:48am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/6 "2020-08-20T11:48:06Z")

</div>

I will give you some more insights into my plans:

I want to import a json file with the following structure:  
 ![json](https://us1.discourse-cdn.com/flex021/uploads/neo4jcommunity/original/2X/f/f242e47a072128b7006ddc6e7a2fdd8b47c5a1dd.png)  
(just with a lot more entries and properties....you get the point)

My goal is to import all products as nodes and categories as nodes.  
As you can see the categories are repetetive and hierarchical. So i want no duplicates of the categories.  
Without the periodic import and a small json the result is this:

 ![graph](https://us1.discourse-cdn.com/flex021/uploads/neo4jcommunity/original/2X/c/ce1e075ef0e1443148dd15b1c47280725bf05b40.png)  
Top level category "Garten & Freizeit" is in the middle and the subcategories are next level nodes and at the end there should be the products ("name" inside the json).

* * *

Back to your idea: periodic.iterate()  
I tried the following code:

```auto
CALL apoc.periodic.iterate(
	"CALL apoc.load.json('file:/master_small.json')
	YIELD value
	MERGE (p:Product {name:value.name})",
	"WITH value, p 
	UNWIND value.categories as cat,
	MERGE (c:Categorie {name:cat})
	MERGE (c)-[r:INHERITS]->(p)",
	{batchSize:100, parallel:true})

```

I received the following error:  
(I am not entirely sure where to split the cypher queries (as iterate() needs two?!)

 ![error](https://us1.discourse-cdn.com/flex021/uploads/neo4jcommunity/original/2X/1/18078005222370feaf125c0534e3612f02fbab42.png)

I appreciate your help a lot 🙂

---

<div class="post-metadata">

**Author:** ![Cobra](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/cobra/32/35243_2.png) [@Cobra](https://community.neo4j.com/u/Cobra)\
**Post date:** [August 20, 2020, 11:53am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/7 "2020-08-20T11:53:24Z")

</div>

```auto
CALL apoc.periodic.iterate('
    CALL apoc.load.json("file:/master_small.json")
	YIELD value
	MERGE (p:Product {name:value.name})
    RETURN value, p
    ','
    WITH value, p 
    UNWIND value.categories AS cat,
    MERGE (c:Categorie {name:cat})
    MERGE (c)-[r:INHERITS]->(p)
    ', {batchSize:100, parallel:true})

```

---

<div class="post-metadata">

**Author:** ![daniel.muellner](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/daniel.muellner/32/4441_2.png) [@daniel.muellner](https://community.neo4j.com/u/daniel.muellner)\
**Post date:** [August 20, 2020, 11:58am UTC](https://community.neo4j.com/t/how-to-periodic-commit-apoc-load-json-can-apocs-be-nested/23786/8 "2020-08-20T11:58:24Z")

</div>

I think i found it: there was a sneaky little comma too much inside the UNWIND line.  
And i closed the first query after "YIELD value" already

The final working code is the following:

```auto
CALL apoc.periodic.iterate(
	"CALL apoc.load.json('file:/master_small.json')
	YIELD value",
	"MERGE (p:Product {name:value.name})
	WITH value, p 
	UNWIND value.categories as cat
	MERGE (c:Categorie {name:cat})
	MERGE (c)-[r:INHERITS]->(p)",
	{batchSize:100, parallel:true})

```

Thank you @Cobra for sticking with me!