cancel
Showing results for 
Search instead for 
Did you mean: 

Memory allocation for multiple indexes

jabb
Node

Hi, I'm using neo4j for quite a while now, so am familiar with the basics.

So my dataset contains multiple CSVs - nodes & edges files, which I import using bulk loader.

It contains various columns with integers and string values. Now I need to index them - multiple of them.

Before that here are some stats:

Total server RAM: 128 GB
Total CSV size: 34 GB

### neo4.conf -> configured as per results from neo4j-admin memrec

Initial & max heap size: 31 GB
Page Cache size: 78 GB

 Now when I load the data and start the neo4j server, the initial RAM increases upto 32GB, which is reasonable because of the heap size. But when I index a column from node CSV (it contains integers ranging from 1-4):

CREATE INDEX word FOR (t:TOK) ON (t.word);

The RAM boosts upto ~ 65 GB. Now when I index other columns with the same datatype, the RAM increases only by 3 gigs or less, for each and every new column.

I tried changing the order in which I was indexing the columns, but the general pattern is that the 1st index occupies maximum RAM, while the next ones take significantly less.

Now here are my questions:

  • What performance implications does the amount of memory allocated has on my indexed data?
  • How does neo4j decide the amount of memory it allocates for the indexes?
  • What's the best approach to optimally distribute the memory among various indexes?
0 REPLIES 0
Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

On November 16 and 17 for 24 hours across all timezones, you’ll learn about best practices for beginners and experts alike.