My long importing query never ends

Neo4j Graph Platform Cypher
21

Sorry for my bad explanation.
1010 is the max number of workId, so creating uniqueness constraint on that property i would expect 1010 nodes in total.

this is the max number of rows:

last row
last row1920×1080 147 KB

should i consider something different on my query?
Is there a way to reduce the process?

22

Try just doing a MERGE on the :Work nodes by the workId as before, but for this run don't consider the properties that you were appending to lists previously, so don't do ON MATCH SET operations. Just get the nodes created.

The ids of the items that you were to have as lists, were you planning on adding specific nodes for them? If so, get those nodes created next (and make sure you have indexes and/or unique constraints to support MERGE operations on them). Most likely those correspond to elements you were going to save as lists.

Once all nodes are created, then go about subsequent runs to MATCH to existing nodes and create the relationships between them. That should be faster than trying to do list appending one at a time for 1 million nodes.

23

thanks Andrew, i think i figured out, i'll let you know soon, let's see if the query is going to stop first

24

Glad to hear it, let us know what you figured out when you get the chance.

25

Hi,

I just want to share my own experience with importing CSV data into Neo4j database.
I also tried doing that using my own queries that built the nodes, properties and relationship from CSV files.
For small sized data this is fine. But once you go larger (for example 10^5 nodes and higher) you can take a vacation after running the query and still wait for a while once you get back home :slight_smile:

A much better approach and it is written in the docs, is to use the import tool:

bin/neo4j-admin import

There are few cons for using this tool:

  1. You will need to convert your CSV file for a very specific format of the import tool.
  2. The import tool creates a new database, so you can't use previous data and then import.

Still, the import process itself is a blast and will finish very quickly so I think overall it is well worth the time I spent on converting to use the import tool.

Good luck!

1 Like
26

Hi guys,

after 4 days of continous running finally it ended!
Before to go on i want also thank ofer.bar for your suggestion, i definitely retry following what you told me(after a giant dump of my DB of course!) but i'm not sure i understood how to exactly do it but i'm going to learn and try to understand, i'll let you know.
May i ask something else about my project here or do i need to open new posts?

27

Hi,

Sure you can ask any question.

You can start with the docs, it is pretty detailed and I managed to figured it out from there:

Don't forget to pick your database version in the doc (3.5/4.0) although I'm not sure it was changed with 4.0...

I hope this helps