ETL Tool Importing Massive Size Data & Constraint Error

Neo4j Graph Platform ETL-Tool
1

Hey everyone,

I'm working on importing a database from Oracle to Neo4j Desktop with ETL Tool Graph App. I've got 2 issues with that.

First one is when I import the data, I get this error even though import is completed.

" Invalid constraint syntax, ON and ASSERT should not be used. Replace ON with FOR and ASSERT with REQUIRE. (line 1, column 1 (offset: 0)) "CREATE CONSTRAINT ON (n:Mrrel) ASSERT n.aui IS UNIQUE"

I'm aware that this command is changed now but I don't know where ETL Tool gets this command from and I can't fix it.

Second problem is my database is huge. There are approximately 250million rows of data in total. Even importing 300 thousand rows takes around 2-3 hours. So importing 250million rows seems like impossible.

etltool
etltool1677×880 45.4 KB

Here's my ETL Tool screen. Changing settings here doesn't help

Is there any method to speed up this process? If anyone has an idea I'd love to hear it.

Thanks.

2

@ozanbicer

  1. i presume this is an export from Oracle and into a Neo4j v5.8.0 database. Correct? or is it a different Neo4j version?

  2. is your data, and specifically a significant portion, from Oracle thus mapped to nodes with name :Mrrel. Since the constraint is not created and as a constraint thus automatically creates an index on said label/property perhaps the abscence of said constraint and accompanying index is leading to the performance degradation. Can you manually create the constraint in the Neo4j database itself prior to export?

3

  1. Yes it's an export from Oracle to Neo4j v5.8.0. Also I've tried with a Neo4j 5.3.0 database, but same problem remains.

  2. Actually, Mrrel is a pretty small portion of my database. But as far as I have observed, I am getting this error in every table/label.

Even though I define the constraints manually, it still tries to create the same constraints with the depreceated command and because of this I keep getting the same error.

I don't know how much this error affects the performance, but defining constraints is not a big problem for me. The main problem is data transfer speed. For a that large database, maybe I should try offline-bulk import mentioned here. Neo4j ETL - Neo4j ETL Docs But I guess ETL Tool Graph App doesn't support that? and I believe I will get the same constraint error there too.