Adding data to a graph is very slow

Neo4j Graph Platform
1

I am using the Neo4j Desktop app with a local database. I've loaded approximately 7,000 nodes and am now adding 4,400 edges between two node types. However, the process has been running for over an hour and has only added 550 edges so far.

I copy and paste the code into the desktop browser query window using the following syntax:

MATCH (a:NotaryAgent { Id: "1706" }), (d:Document { Doc_Id: "45FD4FB0-37F7-49BF-8FB5-5CEA689ECF6A" }) CREATE (a)-[:NOTARIZED { NotarizedDate: "2024-03-13" }]->(d);
MATCH (a:NotaryAgent { Id: "1706" }), (d:Document { Doc_Id: "79D3A532-974C-4D41-A221-21201F3651F7" }) CREATE (a)-[:NOTARIZED { NotarizedDate: "2024-03-13" }]->(d);

This approach is not scalable, as I need to add significantly more nodes and edges. My system is a high-performance PC with 16 cores and 128 GB of RAM.

Does anyone know what might be causing this issue and how I can add large volumes of data more efficiently?

Thank you!

2

@gpadvorac

What version of Neo4j?

Do you have indexes on :NotaryAgent(Id) and :Document(Doc_id) ?

Do you have a PROFILE or EXPLAIN for said query

Are you using parameterized queries ? Parameters - Cypher Manual

3

Neo4j Desktop Version 1.6.1
DBMS Version 5.26.0

No indexes yet. I'm just getting started on Neo4j and have not looked at indexes. I was trying to create a proof of concept to get a project off the ground.

Attached is a screenshot of the profile and I'm not using parameterized queries. I showed code I'm executing in the post above.

"I copy and paste the code into the desktop browser query window using the following syntax:

MATCH (a:NotaryAgent { Id: "1706" }), (d:Document { Doc_Id: "45FD4FB0-37F7-49BF-8FB5-5CEA689ECF6A" }) CREATE (a)-[:NOTARIZED { NotarizedDate: "2024-03-13" }]->(d);
MATCH (a:NotaryAgent { Id: "1706" }), (d:Document { Doc_Id: "79D3A532-974C-4D41-A221-21201F3651F7" }) CREATE (a)-[:NOTARIZED { NotarizedDate: "2024-03-13" }]->(d);"

image
image1021×504 23.7 KB

4

also it was running so slow, i let it run overnight and it added only 2k of 4k edges. This morning I saw this error so I assume it stopped running when this popped up.

image
image529×583 29.7 KB

This is a snippet from the logs:

[2025-01-06 17:51:17.384] [info]  Graph App[neo4j-browser-id]: Checking 5.26.1 for suitable desktop API version
[2025-01-06 17:51:17.387] [info]  Graph App[neo4j-browser-id]: Version 5.26.1 satisfied. { desktopApiVersion: '1.4.0', packageDesktopApiVersion: '^1.4.0' }
[2025-01-06 18:11:16.849] [error] Graph App[38ccea17-2844-444f-bc96-a137b6a23a7d]: Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)
    at SimpleURLLoaderWrapper.emit (node:events:526:28)
    at SimpleURLLoaderWrapper.emit (node:domain:475:12)
[2025-01-06 18:11:16.854] [error] Graph App[b7986f4f-3467-48b2-8868-dd2cd18ed64a]: Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)
    at SimpleURLLoaderWrapper.emit (node:events:526:28)
    at SimpleURLLoaderWrapper.emit (node:domain:475:12)
[2025-01-06 18:11:16.858] [error] Graph App[neo4j-browser-id]: Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)
    at SimpleURLLoaderWrapper.emit (node:events:526:28)
    at SimpleURLLoaderWrapper.emit (node:domain:475:12)
[2025-01-06 18:11:16.862] [error] Graph App[neo4j-bloom-id]: Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)
    at SimpleURLLoaderWrapper.emit (node:events:526:28)
    at SimpleURLLoaderWrapper.emit (node:domain:475:12)
[2025-01-06 18:11:16.866] [error] Graph App[fa072a73-a483-42c8-b19b-6af2b1039713]: Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)

...


[2025-01-06 18:31:32.981] [error] Failed to send event to Desktop Project mixpanel {
  event: 'neo4j-browser:command-cypher',
  properties: {
    graphPlatformId: 'neo4j-desktop',
    graphPlatformVersion: '1.6.1',
    platform: 'Windows',
    platformVersion: '10.0.22631',
    distinct_id: 'client-ea6a3f80-a88b-4b48-b154-df3f3689d957',
    appId: 'neo4j-browser',
    appVersion: '5.26.1',
    browserVersion: '5.26.1',
    neo4jEdition: 'enterprise',
    connectedTo: 'NON-AURA HOST',
    type: 'cypher',
    source: 'PLAY-BUTTON',
    averageWordCount: 126217,
    averageLineCount: 3541,
    estimatedNumberOfStatements: 1,
    token: '2556a08ecb18f6fe0a0d5488e704927e',
    mp_lib: 'node'
  }
}
[2025-01-06 18:51:16.847] [error] Graph App[38ccea17-2844-444f-bc96-a137b6a23a7d]: Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)
    at SimpleURLLoaderWrapper.emit (node:events:526:28)
    at SimpleURLLoaderWrapper.emit (node:domain:475:12)

...


[2025-01-06 19:33:08.850] [error] Online check error: {
  name: 'TimeoutError',
  message: 'Promise timed out after 5000 milliseconds',
  stack: 'TimeoutError: Promise timed out after 5000 milliseconds\n' +
    '    at Timeout._onTimeout (C:\\Program Files\\Neo4j Desktop\\resources\\app.asar\\node_modules\\p-timeout\\index.js:27:54)\n' +
    '    at listOnTimeout (node:internal/timers:559:17)\n' +
    '    at process.processTimers (node:internal/timers:502:7)'

Any help is greatly apriciated.
Thank you.

5

@gpadvorac

I would start of with creating indexes. The current provided plan

  gathers all nodes with label :NotaryAgent.. to which there are some 2900
  gathers all nodes with label :Document ... to which there are 3542
  filter all :NotaryAgent where Id=???  to which there is but 1
  filter all :Document where Doc_id=???? to which there is 1
  ....
  .......

with indexes this could be

use index to find all :NotaryAgent nodes which Id=??? to which there are 1  
use index to find all :Document nodes which Doc_Id=??? to which there are 1
.....
......

is there a reason not to use indexes ?

6

The only reason not to use indexes was I did not know about them till now and I will try your recommendation. Thanks.

Regarding the errors and the logs, it looks like it was trying to check for updates and had a problem resolving a name. This is strange because I just re-installed Neo4J a few days ago so everything should be very current:

Could not check for updates Error: net::ERR_NAME_NOT_RESOLVED
    at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:7169)

Any idea how to get past this?

1 Like