Hi everyone
We've been using neo4j (enterprise) for quite some time now, starting with version 4.1 and slowly, but regularily upgrading to 4.3.8 (which we're running in production for a small SaaS product on an openshift cluster).
Lately, we've decided that it's time for the next version bump to 4.4.12, expecting that this would work flawless as all the other version upgrades in the past. Runtime-wise everything is working fine, no problems here, but we're running into some serious problems with our backup procedure.
We're using the neo4j-admin backup command for doing (remote) backups, specifically by configuring a cronjob on openshift, which runs a specific pod (also with neo4j 4.4.12 installed) and executes the following command:
neo4j-admin backup --from=$NEO4J_HOST --database=* --backup-dir=$LOCAL_BACKUP_DIRECTORY --verbose
This went fine for all versions < 4.4.12, but now the backup fails like 80% of the time, without the following error message:
Execution of backup failed. Log file contains entries with prefix -90, and the lowest supported prefix is KernelVersion{V2_3,version=-10}. This indicates that the log files originates from an older version of neo4j, which we don't support migrations from.
We already made sure that we do not have any resource problems on the backup container and also use a fresh backup directory for each backup, so that we don't run into some "there's already an older backup, maybe that's the problem" kind of situations. Also, the error message itself doesn't make a lot of sense from the outside, so we're currently entirely blocked from proceeding with the upgrade due to our backup procedure failing most of the time.
Does anyone of you have an idea what the reason for the odd error message could be? Or what we could do to remedy the problem?
Thanks in advance!