Batch Import data

cypher
batching

(Kunal Goyal) #1

Hello Everyone,

I am importing the data in neo4j using batch import. I am getting the data from impala using java client code , creating data batches of 1000 records and passing this data for importing using unwind.
i am using following query for importing relationship

UNWIND [mydata] as row MATCH (from:ENT{ID: toString(row.id_1)}),(to:ENT {ID:toString(row.id_2)})  MERGE (from)-[r:ASSO { ENT_AS_ID: toString(row.ent_as_id)}]->(to) ON CREATE SET r.TYPE_AS_ID = toString(row.type_as_id),r.QUA =toString(row.qua) 

i am trying to import 2 million records but what is happening whenever my code stuck or i need to restart my import job then it starts from initial . and again check whole data and it is getting slow as data inserted in neo4jdb

is my query correct for batch import and can we create indexing for relationship as well ?
am i using match and merge correctly for batch import.
my main objective is to speed up the import process.


(David Fauth) #2

Do you have an index on :ENT(ID)?