# Best method or recommandations to load big data from Oracle Database

**URL:** <https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525>\
**Category:** Procedures & APOC\
**Tags:** apoc, cypher, migrated\
**Created:** [February 4, 2025, 2:47pm UTC](https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525 "2025-02-04T14:47:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dak](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/dak/32/29892_2.png) [@Dak](https://community.neo4j.com/u/Dak)\
**Post date:** [February 4, 2025, 2:47pm UTC](https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525/1 "2025-02-04T14:47:29Z")

</div>

Hello,  
I have slowness ( **6 days loading** ) problems with loading databases from Oracle Database

We currently have a big problem for loading data from the oracle database to neo4j enterprise. We totalize 71 million of nodes and 360 million of relationships and this is set to grow. It is important to remember that we will have to do this full load on a regular basis because the data evolves very quickly.  
The loading is done with local cypher-shell.bat of the neo4j server and essentially via the apoc.periodic.iterate procedure with a sql connection oracle + sql query and the second part which executes the apoc.create.node and apoc.create.relationship insert requests.

The insertion of nodes and its properties is done in bacthSize :10000 and parallel :true, it takes an average of 10min.

The insertion of links and their properties is done in bacthSize:1000 and parallel: false, so far we have not been able to load everything yet because it takes a lot of time, sometimes more than 24 hours still running. It is important to note that the SQL part takes an average of 15min to 20min to execute.

**The set (data+schema+index) currently weighs 186 GB and took about 6 days to load**

Loading test is done locally on the neo4j enterprise server (trial version) with the following features:

**Machine:**  
Windows Server 2016 OS, 8 Intel Xeon Gold 6348 processors of 2.6Ghz with 16 UTA in total, 96GB RAM memory, HD HDD (2 partitions C: and D: each 260GB free), D: is mainly used for the storage of transaction logs.

**Some changes to neo4j.conf**  
server.directories.transaction.logs.root = D:/Neo4jEnterprise/data/transactions dbms.security.aut\_enabled=false  
server.memory.heap.initial\_size=28200m  
server.memory.heap.max\_size=28200m  
server.memory.pagecache.size=55800m  
db.logs.query.enabled=OFF  
db.logs.query.obfuscate\_literals=true  
db.logs.query.parameter\_logging\_enabled=false  
db.tx\_log.rotation.retention\_policy=670 files  
db.recorvery.fail\_on\_nissing\_files=false  
server.jvm.additional=-XX:+ExitOnOutOfMemoryError

Thanks in advance

---

<div class="post-metadata">

**Author:** ![john.stegeman](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/john.stegeman/32/26277_2.png) [@john.stegeman](https://community.neo4j.com/u/john.stegeman)\
**Post date:** [February 4, 2025, 8:47pm UTC](https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525/2 "2025-02-04T20:47:08Z")

</div>

Do you have indexes on the nodes for relationship creation? If you don't, then finding the nodes at the end of the relationship is going to be "slow"

---

<div class="post-metadata">

**Author:** ![Dak](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/dak/32/29892_2.png) [@Dak](https://community.neo4j.com/u/Dak)\
**Post date:** [February 5, 2025, 2:39pm UTC](https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525/3 "2025-02-05T14:39:51Z")

</div>

Hi,  
I created the node and relationship indexes after loading the nodes and before loading the relationships otherwise the relationship loading took more time than that.

**Index of node (property ID is a string)** : i have nine labels of nodes

```auto
CREATE INDEX index_node1 IF NOT EXISTS FOR (n:NODE1) ON (n.ID)

```

and

**relationships**

```auto
CREATE INDEX composite_range_rel_index_relation FOR ()-[r:RELATION]-() ON (r.DAT_DEB, r.DAT_FIN, r.ETIQ_PROVEN);

```

Thanks

---

<div class="post-metadata">

**Author:** ![david\_rosenblum](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/david_rosenblum/32/4883_2.png) [@david\_rosenblum](https://community.neo4j.com/u/david_rosenblum)\
**Post date:** [February 5, 2025, 3:56pm UTC](https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525/4 "2025-02-05T15:56:43Z")

</div>

Hi @Dak - I have reached out to you directly to support.  
Thank you!

---

<div class="post-metadata">

**Author:** ![Dak](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/dak/32/29892_2.png) [@Dak](https://community.neo4j.com/u/Dak)\
**Post date:** [February 5, 2025, 5:28pm UTC](https://community.neo4j.com/t/best-method-or-recommandations-to-load-big-data-from-oracle-database/72525/5 "2025-02-05T17:28:27Z")

</div>

Ok, thank, I just waiting the directives...