Migration issue from 3.5 to 4.x /Store format upgrade from v0.A.8

Neo4j Graph Platform
1

Hello All,

I am not able to upgrade my 3.5 db to 4.x I have tried several things I think the issue is my store version.

Here are the steps I followed.

  1. create a database dump file from 3.5
  2. bin/neo4j-admin load the dump into 4.4.4
  3. set dbms.allow_upgrade=true

I have tried with different version 4.0, 4.1 but no success. I am running out of options need some help.

Thank you

2023-03-31 13:20:51.098+0000 INFO [system/00000000] Starting upgrade of database
2023-03-31 13:20:51.457+0000 INFO [system/00000000] Migrating Store files (1/11):
2023-03-31 13:20:51.611+0000 INFO [system/00000000] 10% completed

..
...
2023-03-31 13:20:51.823+0000 INFO [system/00000000] 90% completed
2023-03-31 13:20:51.824+0000 INFO [system/00000000] 100% completed
2023-03-31 13:20:52.063+0000 INFO [system/00000000] Starting transaction logs migration.
2023-03-31 13:20:52.336+0000 INFO [system/00000000] Transaction logs migration completed.
2023-03-31 13:20:52.359+0000 INFO [system/00000000] Successfully finished upgrade of database, took 1s 251ms
2023-03-31 13:20:54.890+0000 ERROR [neo4j/17f81d18] Error upgrading database. Database left intact and will likely not be able to start: java.lang.IllegalArgumentException: Unknown store version 'v0.A.8'
2023-03-31 13:20:54.891+0000 ERROR [tsm/c8eb976c] Error upgrading database. Database left intact and will likely not be able to start: java.lang.IllegalArgumentException: Unknown store version 'v0.A.8'
2023-03-31 13:20:56.184+0000 INFO Sending metrics to CSV file at /var/lib/neo4j/metrics
2023-03-31 13:20:56.308+0000 INFO Bolt enabled on 0.0.0.0:7687.
2023-03-31 13:20:59.750+0000 INFO Remote interface available at http://localhost:7474/
2023-03-31 13:20:59.759+0000 INFO id: ADF4A0F776B26E12B0CC78B9A9F5A3E7C1435931796F1DE2C8630D9A9399F26A
2023-03-31 13:20:59.759+0000 INFO name: system
2023-03-31 13:20:59.762+0000 INFO creationDate: 2023-03-31T12:10:02.969Z
2023-03-31 13:20:59.762+0000 INFO Started.

some more logs :

2023-03-31 18:00:04.406+0000 INFO Starting...

2023-03-31 18:00:05.309+0000 INFO This instance is ServerId{04e69f90} (04e69f90-7b9a-404d-9bba-35e51ea08d42)

2023-03-31 18:00:08.246+0000 INFO ======== Neo4j 4.4.4 ========

2023-03-31 18:00:12.793+0000 INFO [neo4j/17f81d18] Starting upgrade of database

2023-03-31 18:00:12.838+0000 INFO [tsm/c8eb976c] Starting upgrade of database

2023-03-31 18:00:12.861+0000 ERROR [neo4j/17f81d18] Error upgrading database. Database left intact and will likely not be able to start: org.neo4j.kernel.impl.storemigration.StoreUpgrader$UnexpectedUpgradingStoreVersionException: Not possible to upgrade a store with version 'v0.A.8' to current store version AF4.3.0 (Neo4j 4.4.4).

2023-03-31 18:00:12.862+0000 ERROR [tsm/c8eb976c] Error upgrading database. Database left intact and will likely not be able to start: org.neo4j.kernel.impl.storemigration.StoreUpgrader$UnexpectedUpgradingStoreVersionException: Not possible to upgrade a store with version 'v0.A.8' to current store version AF4.3.0 (Neo4j 4.4.4).

2023-03-31 18:00:13.148+0000 DEBUG Did not find any file named auth.ini in /var/lib/neo4j/data/dbms

2023-03-31 18:00:13.154+0000 INFO Sending metrics to CSV file at /var/lib/neo4j/metrics

2023-03-31 18:00:13.225+0000 INFO Bolt enabled on 0.0.0.0:7687.

2

@romeshn
your easiest approach is with 4.4.x run bin/neo4j-admin copy and point it to the 3.5.x db. This will read the 3.5.x db and create a copy, with new name but in 4.4.x format.

note neo4j-admin copy will produce output describing the cypher statements for indexes and constraints that need to be played back into the new 4.4.x copy. it also writes these statements into a file which you can use so as to extract said CREATE statments

3

Thanks for replying to me.

I see the same error, Caused by: java.lang.IllegalArgumentException: Unknown store version 'v0.A.8'

root@00f4817523b0:/var/lib/neo4j# rm -rf /data/databases/tsmdb

root@00f4817523b0:/var/lib/neo4j# bin/neo4j-admin copy --from-path=/restore/graph.db --to-database=tsmdb --verbose --force

Selecting JVM - Version:11.0.16+8, Name:OpenJDK 64-Bit Server VM, Vendor:Oracle Corporation

Executing command line: /usr/local/openjdk-11/bin/java -cp /var/lib/neo4j/plugins/:/var/lib/neo4j/conf/:/var/lib/neo4j/lib/* -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFinalNonStaticFields -XX:+DisableExplicitGC -XX:MaxInlineLevel=15 -XX:-UseBiasedLocking -Djdk.nio.maxCachedBufferSize=262144 -Dio.netty.tryReflectionSetAccessible=true -Djdk.tls.ephemeralDHKeySize=2048 -Djdk.tls.rejectClientInitiatedRenegotiation=true -XX:FlightRecorderOptions=stackdepth=256 -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -Dlog4j2.disable.jmx=true -Dfile.encoding=UTF-8 org.neo4j.cli.AdminTool copy --from-path=/restore/graph.db --to-database=tsmdb --force --verbose

neo4j 4.4.4

VM Name: OpenJDK 64-Bit Server VM

VM Vendor: Oracle Corporation

VM Version: 11.0.16+8

JIT compiler: HotSpot 64-Bit Tiered Compilers

VM Arguments: [-XX:+UseG1GC, -XX:-OmitStackTraceInFastThrow, -XX:+AlwaysPreTouch, -XX:+UnlockExperimentalVMOptions, -XX:+TrustFinalNonStaticFields, -XX:+DisableExplicitGC, -XX:MaxInlineLevel=15, -XX:-UseBiasedLocking, -Djdk.nio.maxCachedBufferSize=262144, -Dio.netty.tryReflectionSetAccessible=true, -Djdk.tls.ephemeralDHKeySize=2048, -Djdk.tls.rejectClientInitiatedRenegotiation=true, -XX:FlightRecorderOptions=stackdepth=256, -XX:+UnlockDiagnosticVMOptions, -XX:+DebugNonSafepoints, -Dlog4j2.disable.jmx=true, -Dfile.encoding=UTF-8]

org.neo4j.cli.CommandFailedException: There was a problem during copy.

at com.neo4j.dbms.commandline.StoreCopyCommand.execute(StoreCopyCommand.java:250)

at org.neo4j.cli.AbstractCommand.call(AbstractCommand.java:71)

at org.neo4j.cli.AbstractCommand.call(AbstractCommand.java:34)

at picocli.CommandLine.executeUserObject(CommandLine.java:1953)

at picocli.CommandLine.access$1300(CommandLine.java:145)

at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)

at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)

at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)

at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)

at picocli.CommandLine.execute(CommandLine.java:2078)

at org.neo4j.cli.AdminTool.execute(AdminTool.java:93)

at org.neo4j.cli.AdminTool.main(AdminTool.java:79)

Caused by: java.lang.IllegalArgumentException: Unknown store version 'v0.A.8'

at org.neo4j.kernel.impl.store.format.RecordFormatSelector.selectForVersion(RecordFormatSelector.java:109)

at org.neo4j.kernel.impl.store.NeoStores.verifyRecordFormat(NeoStores.java:211)

at org.neo4j.kernel.impl.store.NeoStores.(NeoStores.java:122)

at org.neo4j.kernel.impl.store.StoreFactory.openNeoStores(StoreFactory.java:139)

at org.neo4j.kernel.impl.store.StoreFactory.openAllNeoStores(StoreFactory.java:93)

at com.neo4j.dbms.commandline.storeutil.StoreCopy.copyTo(StoreCopy.java:209)

at com.neo4j.dbms.commandline.StoreCopyCommand.execute(StoreCopyCommand.java:246)

... 11 more

4

After steps 1-3, I see this: "2023-03-31 13:20:52.359+0000 INFO [system/00000000] Successfully finished upgrade of database, took 1s 251ms". Did you check the data/databases folder in V4.4.4 to see if it created a 'graph.db'database with all store files.

In your copy statement, you used a file from /restore/ folder and not from /databases/ folder. You should use the database that was created in V4.4.4.

From V4.4.4 Terminal run this command and check the result:
bin/neo4j-admin store-info data/databases/graph.db
you should see: Store format version: SF4.3.0
Store format introduced in: 4.3.0

For smooth upgrade to version 5.x, the minimum store format version should be 4.3.0.

5

@romeshn

per Store Format Versions Reference Guide - Knowledge Base, v0.A.8 corresponds with Neo4j v3.2. However your subject line states migration issue from 3.5 ?

6

If that's the case then romeshn should first upgrade his db to 3.5 from 3.2 and take a dump from 3.5 version for further upgrade to v5.x

7

Here are my suggestions:

Step 1: Upgrade your db to 3.5.
Step 2: If upgrade is successful, simply copy the graph.db folder.
Step 3: Go to 4.3 or any higher version
Step 4: Paste the folder from Step 2 in data/databases folder of Step 3.
Step 5: Open the terminal (from Step 3) and run this:
bin/neo4j-admin copy --from-database=graph.db --to-database=graphdb --force
(select any name for to-database)
Step 6: Once step 5 is successful, then run in Terminal:
bin/neo4j-admin store-info data/databases/graphdb (replace graphdb with the correct name)
Result should show:Store format version: SF4.3.0
Store format introduced in: 4.3.0

Once all the above steps are completed, copy the graphdb folder from data/databases (as in Step 6).

Step 7: Paste the copied db folder in V5.5 or above data/databases folder.

Final Step: Open Terminal and run this: bin/neo4j-admin database copy graphdb graphdb5 --force
Final Check: In Terminal run this:bin/neo4j-admin database info graphdb5 (use the correct name)
Result should show you format version 5 info.

That's all!

8

All,
finally, I was able to upgrade to 4.4.4. @dana_canzano thanks for pointing out the version v3.2. I took this database from 3.5 server that means db never got updated even tho it was running on 3.5 server. @ameyasoft thanks for laying out all the steps it took some time to complete since I was running on docker but no complaints:slight_smile:
the steps I followed,

  1. upgrade to 3.5
  2. copy the database to 4.4.4 server and use neo4j-admin copy ...
  3. CREATE DATABASE
  4. Recreated all the indexes

Thank you

1 Like