cancel
Showing results for 
Search instead for 
Did you mean: 

apoc.load.json from url got stuck

nengkhoiba
Node

Hi everyone,

I have the below query to load 4mb of json data and it got stuck without any warning or error only kept on spinning.

If anybody can help or suggest me a solution that will be helpful.

You can directly run the query if you want to check

CALL apoc.periodic.iterate(
	"CALL apoc.load.json('https://s3.eu-central-1.amazonaws.com/demo.hcpspace.app/bulk_connections.json')
	YIELD value as row",
	"MERGE(n:People{uuid: row.uuid})
	WITH n,row
		UNWIND row.uuids as uuid
		MERGE(p:People{uuid: uuid})
		MERGE(n)-[:FOLLOW]->(p)",
	{batchSize:100, parallel:true})
1 ACCEPTED SOLUTION

glilienfield
Ninja
Ninja

If you execute the apoc.load.json and look at the results, there are 70,251 records. I tried your query and it kept spinning. I add the following index and it completed in around 7 sec.

CREATE INDEX ON :People(uuid)

I would say the problem is the MERGE statements that have to look up the node by 'uuid'. Without the index this is a node scan, which takes longer and longer as the number of nodes increases. 

View solution in original post

2 REPLIES 2

glilienfield
Ninja
Ninja

If you execute the apoc.load.json and look at the results, there are 70,251 records. I tried your query and it kept spinning. I add the following index and it completed in around 7 sec.

CREATE INDEX ON :People(uuid)

I would say the problem is the MERGE statements that have to look up the node by 'uuid'. Without the index this is a node scan, which takes longer and longer as the number of nodes increases. 

nengkhoiba
Node

I totally forgot to create index. I was recreating the db and forgot to add index. Thank you for pointing out

Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

On November 16 and 17 for 24 hours across all timezones, you’ll learn about best practices for beginners and experts alike.