cancel
Showing results for 
Search instead for 
Did you mean: 

Head's Up! Site migration is underway. Phase 2: migrate recent content

ETL Tool mapping and data transfer takes a lot of time

AlexisPa
Node

Hi everyone,

I'm testing the Neo 4J ETL Tool with several MySQL and MSSQL databases and so far mapping and data transfer takes a long time to the point that for some I can't get the mapping done and for none I have been able to finish the data transfer. The details below:

Target Database: Neo4j Graph database version 5.1.0

Processor: Intel(R) Xeon(R) Gold 5217 CPU @ 3.00GHz 2.99 GHz (2 processors)

Ram: 64GB

Windows 10

 Test 1&2: MySQL and MSSQL database stuck on data transfer

After mapping and discarding some tables, I tested both import online import modes with the default configuration (Unwind Row Size: 1000, Transaction Batch Size: 10000): The process is stuck at "- Creating nodes with label DspIexGmp" since more than 24h. The table itself has a size of 100MB, the database in total a size of 400 MB.

Test 3: MSSQL database stuck on mapping

For a different MSSQL database, the mapping is stuck since 10h after the following lines:

- Crawling routines
- Retrieved 0 routines
- Not retrieving synonyms, since this was not requested
- Not retrieving sequences, since this was not requested

The Neo4j log gives me always the same lines over and over again during either tests mapping and data transfer:

[2022-10-28 13:15:56.009] [info] Online check request: https://dist.neo4j.org/neo4j-desktop/win/latest.yml
[2022-10-28 13:15:56.073] [info] Online check response: 200 version: 1.5.2
files

Does anyone have any ideas what can help? I am open to alternative solutions on how to import a existing database to Neo4j but have very limited knowledge about how to do so.

Thanks in advance!

 

If more information is required feel free to ask, I can also send the full logs both from the Tool and the Desktop Application.

2 REPLIES 2

ThomasVarias
Node Clone

Have you considered import it through a CSV file? Maybe that wat you can make progress.

Thomas

Not yet, I thought the tool would be the easiest way to import my database. When importing the CSVs, will I need to map the relationships between the tables myself or can this be automatized as well?