Neo4j performance differences

General Newbie Questions
1

Hello. I just configured Neo4j cluster with 5 nodes. I tried some queries, and I spotted that whilst some of them are pretty quick (about 4-8ms), some are stuck for quite a long time, ie. 900ms. What may be the cause of it?

Here is a sample log:

2023-08-03 18:02:49.039+0000 INFO  id:105936 - transaction id:105935 - 995 ms: (planning: 4, waiting: 0) - 312 B - 32468 page hits, 1 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40148    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource) WHERE s.resourceId = '7316062' RETURN s - {} - runtime=pipelined - {}
2023-08-03 18:02:49.092+0000 INFO  Query started: id:105982 - transaction id:105981 - 4 ms: (planning: 4, waiting: 0) - 0 B - 0 page hits, 0 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40148    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource)-[r:CAN_BE_USED_WITH]-() WHERE s.resourceId = '7468007' RETURN r - {} - runtime=null - {}
2023-08-03 18:02:49.363+0000 INFO  id:105949 - transaction id:105948 - 936 ms: (planning: 4, waiting: 0) - 312 B - 32469 page hits, 0 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40098    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource)-[r:CAN_USE]-() WHERE s.resourceId = '7725881' RETURN r - {} - runtime=pipelined - {}
2023-08-03 18:02:49.369+0000 INFO  id:105940 - transaction id:105939 - 996 ms: (planning: 4, waiting: 0) - 312 B - 32479 page hits, 4 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40304    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource)-[r:CAN_BE_USED_WITH]-() WHERE s.resourceId = '6444194' RETURN r - {} - runtime=pipelined - {}
2023-08-03 18:02:49.397+0000 INFO  id:105951 - transaction id:105950 - 941 ms: (planning: 5, waiting: 0) - 312 B - 32468 page hits, 0 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40344    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource)-[r:CAN_BE_USED_WITH]-() WHERE s.resourceId = '7934168' RETURN r - {} - runtime=pipelined - {}
2023-08-03 18:02:49.411+0000 INFO  id:105957 - transaction id:105956 - 919 ms: (planning: 4, waiting: 0) - 312 B - 32474 page hits, 1 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40168    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource)-[r:FOUND_AT]-() WHERE s.resourceId = '7060516' RETURN r - {} - runtime=pipelined - {}
2023-08-03 18:02:49.416+0000 INFO  Query started: id:105985 - transaction id:105982 - 4 ms: (planning: 4, waiting: 0) - 0 B - 0 page hits, 0 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40344    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Person)-[r:WAS_IN]-() WHERE s.personId = '727960' RETURN r - {} - runtime=null - {}
2023-08-03 18:02:49.417+0000 INFO  id:105985 - transaction id:105982 - 5 ms: (planning: 4, waiting: 0) - 312 B - 4 page hits, 0 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40344    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Person)-[r:WAS_IN]-() WHERE s.personId = '727960' RETURN r - {} - runtime=pipelined - {}
2023-08-03 18:02:49.480+0000 INFO  id:105964 - transaction id:105963 - 914 ms: (planning: 3, waiting: 0) - 312 B - 32468 page hits, 1 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40072    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Resource)-[r:CAN_BE_USED_WITH]-() WHERE s.resourceId = '7871379' RETURN r - {} - runtime=pipelined - {}
2023-08-03 18:02:49.491+0000 INFO  Query started: id:105987 - transaction id:105983 - 3 ms: (planning: 3, waiting: 0) - 0 B - 0 page hits, 0 page faults - bolt-session    bolt    neo4j-java/4.4.5-27f38b532a360cf35485df12ced05e31409afcd3        client/148.187.37.179:40072    server/148.187.37.171:7687>    test_db - neo4j - MATCH (s:Company) WHERE s.companyId = '180578' RETURN s - {} - runtime=null - {}
2

I imagine it is because the queries taking in excess of 900ms result in much more data. Compare the page hits between those and the two completing in around 5ms.

3

Hello, thank you for answering. Yes, indeed results happen to be quite big. Is there any good option to improve the latency? The DB itself is not that big (it's about 7GB) and if possible I could process it even in memory. Now I'm trying with these parameters:

db.memory.pagecache.warmup.preload=true
server.memory.query_cache.per_db_cache_num_entries=10000

server.memory.pagecache.size=10GiB
server.memory.pagecache.scan.prefetchers=128

What else could I try?

4

@ex3d

a. For the queries that are 900ms + do they use any indexes for the given labels and properties included in the where clause?

b. any opportunity to submit the query as a parameterized query?

c. if you run the query multiple times does the 2nd and subsequent return faster?

d. Though it probably shouldnt matter, probably, but what version of Neo4j ?