Migration paths 3.5 to 4.4 community only

Neo4j Graph Platform
1

Hi, I am currently deploying neo4j 3.5 community edition for several docker instances. We are now trying to upgrade to 4.4 to maintain LTS, and I need clarification on a couple of topics.

The migration guide highlights the path of using neo4j-admin copy. Sometimes the guides denote that community edition can only be upgraded to enterprise edition this way; in other instances, it makes no references to enterprise edition being needed at all. This topic specifically does not mention enterprise in it: Migrate a single instance - Upgrade and Migration Guide

Is there any way to use neo4j-admin copy as directed, to convert 3.5-community to 4.x-community? Alternatively if not, is there any way to convert 4.x-enterprise to 4.x-community instead, if we chose to use neo4j desktop (local enterprise) to facilitate the migration to leverage the tool, and then switch back to community after the copy?

We cannot convert to enterprise edition, and the sequential migration of 3.5->4.0->4.1->4.2->4.3->4.4 is something we would like to do anything we can to avoid, given the quantity of instances we need to upgrade.

Any help or advice is appreciated, thank you for your time

2

Wow the documentation is really confusing about the 3.5 to 4.4 upgrade path in community edition. I'll see what I can do to get it fixed.

According to our documentation here:

The 3.5 community to 4.4 community upgrade procedure does unfortunately force you through the 3.5.latest > 4.0 > 4.1 > 4.2 > 4.3 > 4.4 path.

create a dump of your data

Use neo4j-admin dump to back up your data. You'd have to use a 3.5 container to do that. It will be something like:

docker run -it --rm \
-v /your/data/dir:/data \
-v /some/backup/dir:/backups \
neo4j:3.5 \
    neo4j-admin dump --to=/backups

You might need to chmod a+w the backup destination folder first.

see if restoring the 3.5 dump works in 4.4

I would say it's definitely worth testing whether the 3.5 dump can be restored directly in a clean 4.4 neo4j database before going through the whole upgrade path.
See Docker maintenance operations - Operations Manual

Use an empty folder to mount to /data. It's important the destination database is fresh. You'll still have to manually update the neo4j.conf.

Follow sequential upgrade path

First do the checklist:

upgrade to 4.0

I think because you're using docker this won't be as difficult as the documentation makes it sound.
To upgrade 3.5 to 4.0, is documented here:

For docker, other than manually updating the neo4j.conf you have to do:

docker run -it --rm \
-v /your/data/dir/4.0:/data \
-v /your/conf/dir/4.0:/conf \
-v /some/backup/dir/3.5:/backups \
neo4j:4.0 \
    neo4j-admin load --from=/backups/<db_name>.dump –database=<db_name> --force

Then to complete migration start neo4j:

docker run -it --rm \
-v /your/data/dir/4.0:/data \
-e NEO4J_dbms_allow__upgrade=true \
neo4j:4.0

upgrade to 4.4

I think this is automatable, but the process is similar.
It's a loop of backup in old version, restore in new version, start new neo4j to complete migration.

I've written a sample script to do that, but it does require monitoring to verify the data after each upgrade and to close each neo4j container. It is attached. It would need a lot of modifying for your use case. It did work for my extremely simple data.

docker-upgrade-35-44.txt (2.4 KB)

upgrade 4.0 to 4.4 shortcut?

I don't recommend this for production environments but it is a lot quicker.

If you're not too worried about backing up every minor version along the way, and you have a way to verify your data integrity after upgrading you can do:

versions_next=("4.1" "4.2" "4.3" "4.4")
for ver in ${versions_next[@]}; do
    echo "Completing ${ver} migration"
    docker run -it --rm \
    --publish=7474:7474 --publish=7687:7687 \
    -v $PWD/mounts/data/4.0:/data \
    -e NEO4J_dbms_allow__upgrade=true \
    -e NEO4J_AUTH=none \
    neo4j:${ver}
done

Obviously NEO4J_AUTH should be your actual password or authentication. $PWD/mounts/data/4.0 is the migrated 4.0 database data folder.

1 Like
3

Hi Jenny, this is incredibly helpful - thank you for the fast and very thorough reply to this. I will continue testing using this as a better guiding method for sure, and report back if anything else comes up - this comes off much more straightforward though, so hopefully not.

The main thing I think we're starting to see now with testing is there are a lot of small nuances to changes in syntax, some quirky documented changes to apoc uuid, and the python driver 1.7 -> 4.4 has some undocumented breaking changes in how it returns values from certain functions ( .data() doesn't give a Node object anymore, just a pure dictionary ), but understand that some of that is inevitable with taking this leap.

Would be curious if you have any suggested resources on the changes in cypher syntax / python front. Have pieced together things from the different docs, git repos, etc so far, but haven't found a good one stop shop for every footgun yet.

Thank you again

1 Like