Hello everyone,
I have a question about graph db memory usage.
I create a in memory graph using gds.graph.project() and I work on it. When I drop this in memory graph, memory usage does not change. I have to stop-start the neo4j connection to refresh memory usage.
Is there any way to fix this and when I drop the memory graph reset the memory usage?
Hello,
Neo4j (and GDS) runs on a JVM, which has a garbage collector to clean up unused memory.
It can take a bit until the memory is actually deallocated.
How do you monitor the memory usage? / How long are you waiting for the memory usage to change?
We are checking our memory usage with htop and it hasn't changed for 24 hours.
We do not have an active graph right now, we dropped them all. And we think that we don't use anything right now but how can we check this for sure?
Our workflow is like:
creating an in memory graph -> executing queries (especially fastRP algorithm) -> drop the in memory graph.
Could this be related to the fastRP algorithm? I hope I have conveyed the problem correctly. Thanks for your support already.
as @florentin_dorre mentioned, GDS and Neo4j run within the JVM. The JVM allocates memory from the OS and then manages its own heap on top of that, using a Garbage Collector (GC).
htop doesn't know about the JVM heap or its GC, it only sees the memory that the JVM allocated and it would only go down if and once the JVM decides to release memory back to the OS (also called "uncommit" memory).
When you drop a graph from the catalog, memory-wise, nothing happens at first. All the memory that the graph was using is now elligable for garbage collection. Later, at some point, the Garbage Collector will run and decide to collect the graph. At that point, its memory is reclaimed by the JVM heap; the OS (and htop) still sees it as used because the JVM manages that memory. Most of the time the JVM decides to re-use that now-available memory rather than giving it back to the OS.
Only under certain conditions and with certain JVM flags set would the JVM decide to return memory to the OS, which wou would then see in htop. We, from within GDS, cannot control when this happens, neither can you as the user, it is up to the Garbage Collector to decide if and when to uncommit. It may also decide to never do that.
There are a few things you can check on your end to see if it would be possible at all; You need to:
-Xms value be smaller that your -Xmx value somewhere in the neo4j.conf.For more technical details, you can search for "jvm uncommit memory" using your favorite search engine.
At any rate, this is not an issue with any algortihm, or GDS, or Neo4j, it's how any programm than runs on the JVM behaves. Typically, the JVM is best left to its own devices when memory management is concerned, enforcing a certain behavior from the outside can have unexpected consequences, mostly performance wise. The JVM also comes with its own set of profiling tools you'd use instead of htop, e.g. jvisualvm.
Hi @paul.horn ,
In the Neo4j conf file setting the max memory limit for example 250G.
Initial memory usage of the neo4j does not starts directly on 250G. It starts around 15-20G and then if we are going to create an in-memory graph+fastrp+drop the graph it goes up to 250G and never decreases.
I am talking in terms of htop view by the way.
In the other hand you are saying that (if I understand correctly) even if we drop the graph or trying to use some methods for freeing memory we are not able to see them from htop. I am totally okay with that. My observation was as follows:
Doing the same sequence (create+fastrp+drop) twice, neo4j goes down because of the insufficent memory.
Our goal was returning back to initial memory usage after first sequence.
yeah, htop will not be able to see inside the JVM and if it reports that the java process is using 250G of memory, it very well might be that most of it is free/available from inside the JVM.
You can try run this command from where you run htop otherwise:
jcmd $(jps | grep -E 'EntryPoint$' | cut -d' ' -f1) GC.heap_info
This should print something like
garbage-first heap total 360448K, used 220058K [0x0000000400000000, 0x0000000800000000)
region size 8192K, 21 young (172032K), 4 survivors (32768K)
Metaspace used 108491K, committed 109696K, reserved 1179648K
class space used 13505K, committed 14016K, reserved 1048576K
In this case, the total heap is 352M, of which 215M is being used.
Compared with the htop output, which shows 812M, which is the JVM heap and all the memory the JVM and GC need for themselves.
Doing the same sequence (create+fastrp+drop) twice, neo4j goes down because of the insufficent memory.
Well, that should work, ideally. Did you configure Neo4j in a certain way, especially around heap sizes, page cache sizes?
Our goal was returning back to initial memory usage after first sequence.
It won't be returning to the initial state, there's some caches of various sorts involved, that will retain some memory after the operation. And since, on the JVM, memory is reclaimed on demand and not when something is no longer needed, it might not go down at all immediately after dropping the graph. It should, however, eventually clean up the old graph when you do a second project+algo run.
Hi @paul.horn ,
I understood what you said. We will do our experiments with using your advices about memory checking and give feedback to here.
Actually the conf file does not changed so much. Only the parameter for the max usage was set to some point, rest is default. If we won't be able to observe what we want, we can return back to here and maybe take your advices about conf file memory configs.
Could you please leave the topic open for a while ?
Thank you so much.