Neo4j server slow startup time

Neo4j Graph Platform Server
21

@dana_canzano I have Neo4j 4.0, but my database uses 3.5.14. Sorry my data base of 900GB includes neostore.transaction.db. Without that it is about 300 GB. I got this number using

:sysinfo

My server has 256GB RAM. And regarding the heap size and page cache I set them to 90GB and 70 GB respectfully in the Neo4j config file. I have other services that run on this server. So I had to allot some RAM to those.

If I dedicate this server to neo4j, what should be my Pagecache and heap size allocation if I have a 256 GB RAM

22

understood. How did you determine to set them to said values? if the size of the database is 300GB (i.e. not including neostore.transaction.db* files) then i would suggest setting setting

dbms.memory.pagecache.size=300GB
dbms.memory.heap.init_size=32G
dbms.memory.heap.max_size=32G

unless there was a specific reason for setting pagecache to 70G and heap to 90G? Was there a reason these values were choosen?

Also, I'm not sure I understand

@dana.canzano I have Neo4j 4.0, but my database uses 3.5.14

23

As I said earlier,I have other services running on the server, and they need about 50 GB of RAM. So, I set the remaining RAM to pagecache and heap size. But the main reason to allot more RAM to heap is because I was getting heap errors, while doing certain queries, so I allotted more RAM to it. Other than that, I didn't follow any specific guidelines on how much to allot to heapsize and pagecache.

But 300GB for pagecache would be higher than the available RAM (256GB).

And regarding the Neo4j version, what I meant is I have the neo4j 4.0 installed. But since I'm having issues with apoc procedures on Neo4j 4.0, i used version 3.5.14 while creating the database. In Neo4j desktop you can have multiple versions for different databases.

24

my apologies on suggesting dbms.memory.pagecache.size=300G when the server itself only has 256G.
So if those other services require 50G, thus leaving a max of 176G of RAM left over for Neo4j and general OS operations, I would reset to

dbms.memory.pagecache.size=125GB
dbms.memory.heap.init_size=32G
dbms.memory.heap.max_size=32G

and regarding setting heap to 90G because you had encountered 'heap errors' then ideally we would want to investigate what queries were responsible and see if they were properly written.

25

I face an issue with neo4j while deploying my project in the Ubuntu (16.04) server. For some reason, whenever I start using neo4j on the server the swap memory slowly creeps up to 100% (of just ~4GB of swap space) while regular memory remains largely underutilized (around 9% usage of ~128GB of RAM). In my local machine, I am not facing this issue and the python script to create neo4j database from csv files takes only 15 minutes to execute. But in the server it takes days and finally it consume full swap memory. I tried deploying in aws server and I am facing the same issue there also. In both servers, the I have upgraded the java version to 11.0.5 to make it compatible with my local.