Now, even if it happens (1 in 10 times) that the process goes to its completion, at the stage where I am at
2020-05-12 15:25:44.046+0000 INFO [o.n.c.s.StoreCopyClient] Copying neostore
2020-05-12 15:25:44.048+0000 INFO [o.n.c.s.StoreCopyClient] Copied neostore 16.00 kB
2020-05-12 15:25:44.049+0000 INFO [o.n.c.s.StoreCopyClient] Done, copied 642 files
2020-05-12 15:25:44.202+0000 INFO [o.n.b.BackupService] Start receiving transactions from 3796108
2020-05-12 15:25:46.884+0000 INFO [o.n.b.BackupService] Finish receiving transactions at 3796108
2020-05-12 15:25:46.920+0000 INFO [o.n.b.BackupService] Start recovering store
I get this error after:
command failed: Backup failed: Error starting org.neo4j.com.storecopy.ExternallyManagedPageCache$GraphDatabaseFactoryWithPageCacheFactory$1
I saw a post about it on Backup Error · Issue #11992 · neo4j/neo4j · GitHub and changed the max open files on my system, but that didn't help either.
The last records in the log on the server I am backing up are:
2020-05-12 15:25:42.811+0000 INFO [o.n.k.i.s.c.CountsTracker] About to rotate counts store at transaction 3796320 to [/home/neo4j-enterprise-3.3.3/data/databases/graph.db/neostore.counts.db.b], from [/home/neo4j-enterprise-3.3.3/data/databases/graph.db/neostore.counts.db.a].
2020-05-12 15:25:42.819+0000 INFO [o.n.k.i.s.c.CountsTracker] Successfully rotated counts store at transaction 3796320 to [/home/neo4j-enterprise-3.3.3/data/databases/graph.db/neostore.counts.db.b], from [/home/neo4j-enterprise-3.3.3/data/databases/graph.db/neostore.counts.db.a].
2020-05-12 15:25:47.723+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [3796320]: Store flush completed
2020-05-12 15:25:47.723+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [3796320]: Starting appending check point entry into the tx log...
2020-05-12 15:25:47.724+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [3796320]: Appending check point entry into the tx log completed
2020-05-12 15:25:47.725+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [3796320]: Check pointing completed
2020-05-12 15:25:47.725+0000 INFO [o.n.k.i.t.l.p.LogPruningImpl] Log Rotation [898]: Starting log pruning.
2020-05-12 15:25:47.728+0000 INFO [o.n.k.i.t.l.p.LogPruningImpl] Log Rotation [898]: Log pruning complete.
2020-05-12 15:30:25.788+0000 WARN [o.n.k.i.c.MonitorGc] GC Monitor: Application threads blocked for 204ms.
The local backup log (where I'm backing up to) doesn't have any errors.