# CSV loading, out of memory problem

**URL:** <https://community.neo4j.com/t/csv-loading-out-of-memory-problem/20336>\
**Category:** Import / Export\
**Created:** [June 20, 2020, 10:40am UTC](https://community.neo4j.com/t/csv-loading-out-of-memory-problem/20336 "2020-06-20T10:40:03Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jupiter](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/jupiter/32/12220_2.png) [@jupiter](https://community.neo4j.com/u/jupiter)\
**Post date:** [June 20, 2020, 10:40am UTC](https://community.neo4j.com/t/csv-loading-out-of-memory-problem/20336/1 "2020-06-20T10:40:03Z")

</div>

hello,  
I'm trying to import data using this code:

```auto
:auto USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
MERGE (followed:User {twitter_id: followers.followed})
MERGE (follower:User {twitter_id: followers.follower})
MERGE(follower)-[:FOLLOW]->(followed)

```

But seems that my RAM is not enough to execute the query, I tried to split it into two queries and I just tried to execute this one:

```auto
:auto USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
MERGE (:User {twitter_id: followers.followed})
MERGE (:User {twitter_id: followers.follower})

```

but I always get the same problem, how should I write the query to solve? increasing the commit frequency doesn't help.  
I have to say that the first two MERGE will fail a lot of times because there are a lot of duplicated entries in the file, I don't know if it is important.

thanks

---

<div class="post-metadata">

**Author:** ![sameerG](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/sameerg/32/3830_2.png) [@sameerG](https://community.neo4j.com/u/sameerG)\
**Post date:** [June 21, 2020, 10:34am UTC](https://community.neo4j.com/t/csv-loading-out-of-memory-problem/20336/2 "2020-06-21T10:34:35Z")

</div>

Follow the below mentioned sequence and see if memory error goes away

1. Issue command $neo4j-home \> bin/neo4j-admin memrec --memory=50%of RAM in g --database=Your\_db\_name
2. Look at the output parameters.
3. Stop Neo4j server
4. Copy parameters to neo4j.conf file and apply the changes.
5. Start the server
6. Modify Query (:auto USING PERIODIC COMMIT 200 or 300)
7. See if it still going out of memory
8. If yes then you can run `PROFILE` on your queries to see if they use EAGER loading  
and either modify queries or run multiple passes on the same file, so it does not do this.

---

<div class="post-metadata">

**Author:** ![intouch\_vivek](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/intouch_vivek/32/4097_2.png) [@intouch\_vivek](https://community.neo4j.com/u/intouch_vivek)\
**Post date:** [June 21, 2020, 12:51pm UTC](https://community.neo4j.com/t/csv-loading-out-of-memory-problem/20336/3 "2020-06-21T12:51:22Z")

</div>

Hi Jupiter,

Welcome to the community!!  
Kindly lets know the available RAM in your system and also the dataset size you are trying to ingest.  
As per the query you have mentioned it should not have any problem in ingest unless and until you are very less available memory. Also I hope you have not modified memory configuration setting in the config file.

Also for the time being try below

```auto

USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
MERGE (followed:User {twitter_id: followers.followed})

USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
MERGE (follower:User {twitter_id: followers.follower})

USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
Match (followed:User {twitter_id: followers.followed})
Match (follower:User {twitter_id: followers.follower})
MERGE(follower)-[:FOLLOW]->(followed)

```

---

<div class="post-metadata">

**Author:** ![jupiter](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/jupiter/32/12220_2.png) [@jupiter](https://community.neo4j.com/u/jupiter)\
**Post date:** [June 25, 2020, 5:58pm UTC](https://community.neo4j.com/t/csv-loading-out-of-memory-problem/20336/4 "2020-06-25T17:58:17Z")

</div>

> [@intouch\_vivek](#):
>
> ```auto
> USING PERIODIC COMMIT 500
> LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
> MERGE (followed:User {twitter_id: followers.followed})
> 
> USING PERIODIC COMMIT 500
> LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
> MERGE (follower:User {twitter_id: followers.follower})
> 
> USING PERIODIC COMMIT 500
> LOAD CSV WITH HEADERS FROM "file:///followers.csv" AS followers
> Match (followed:User {twitter_id: followers.followed})
> Match (follower:User {twitter_id: followers.follower})
> MERGE(follower)-[:FOLLOW]->(followed)
> 
> ```

This solved for me, thanks!