Neo4j Transaction Logs eating up entire hard disk space

Neo4j Graph Platform
, ,
1

I am trying to write to my Neo4j 4.4 Community Edition database in Google Cloud with 100 GB disk space, 16 GB RAM and 4 CPU Cores

Till it wrote 75000 nodes, there was no issue.

Then when I started querying, it became slow, so I created indexes.

Now, when I try to write only 4000 nodes, the transaction logs started taking up 91 GB out of 100 GB

Making it impossible to continue my program.

I tried following the instructions from How do I recover from No space left on device - Knowledge Base and checked my config.

[dbms.tx_log.rotation.retention_policy] is set to 1 day ..

but the logs in /var/lib/neo4j/data/transactions/neo4j are still present even after 7 days.

What am I missing? is it the neo4j version problem or is it with the indexes I created or the config?

Please help me out here as I am beyond my schedule for this project.

Thank you.

2

@santoshgsk

Transaction logs are really only necessary so as to support incremental backup, of which community edition does not support. So you might be better off with

dbms.tx_log.rotation.retention_policy=2 files
dbms.tx_log.rotation.rotation_szie=100mb

note log rotation occurs at time of checkpoint which by default is every 15 mins or every 100k txns.

3

Thank you @dana_canzano for your reply.

What should I do about the existing transaction files? If I delete them, will my database be corrupted?

4

@santoshgsk

You can set the txn retention/rotation dynamically via a call dbms.setConfigValue() Update dynamic settings - Operations Manual

and so set the retention/rotations as described above and then issue a checkpoint via call db.checkpoint();