Unable to upload backup file with neo4j-admin database upload command

Neo4j Graph Platform Import / Export
1

I am able to upload .dump-files using this command:

neo4j-admin database upload neo4j --from-path=./data --to-uri=neo4j+s://... --overwrite-destination=true

But when I try to do the same with .backup files I only get this error message:

org.neo4j.cli.CommandFailedException: Could not find any archive files

First I export the .backup file from an Aura professional instance, then I try to upload the same .backup file that I recently exported to the same aura professional database.

I am using this docker image neo4j:5.23.0-community and according to this site: Upload to Neo4j Aura - Operations Manual the command should be able to handle .backup files on versions from 5.22 and forward. To me the the error message seems to indicate that the backup file can't be found but since the .dump file can be found in the very same directory, I am suspecting that the .backup file for some reason is not accepted.

As a new user I am not allowed to upload files to this topic and if I paste the entire content of the logfiles, the character limit of a topic is exceeded but here is part of the content of the logfiles and regarding the backup-file I am not attaching it, partly because of the restrictions and partly because the problem does not seem to be the .backup file itself. If needed I will find a way to share the files.

debug.log

2024-10-21 19:28:33.450+0000 INFO  [o.n.g.f.m.GlobalModule] Logging config in use: File '/var/lib/neo4j/conf/server-logs.xml'
2024-10-21 19:28:33.550+0000 WARN  [o.n.k.i.JvmChecker] The max heap memory has not been configured. It is recommended that it is always explicitly configured, to ensure the system has a balanced configuration. Until then, a JVM computed heuristic of 16793993216 bytes is used instead. If you are running neo4j server, you need to configure server.memory.heap.max_size in neo4j.conf. If you are running neo4j embedded, you have to launch the JVM with -Xmx set to a value. You can run neo4j-admin server memory-recommendation for memory configuration suggestions.
2024-10-21 19:28:33.550+0000 WARN  [o.n.k.i.JvmChecker] The initial heap memory has not been configured. It is recommended that it is always explicitly configured, to ensure the system has a balanced configuration. Until then, a JVM computed heuristic of 1056964608 bytes is used instead. If you are running neo4j server, you need to configure server.memory.heap.initial_size in neo4j.conf. If you are running neo4j embedded, you have to launch the JVM with -Xms set to a value. You can run neo4j-admin server memory-recommendation for memory configuration suggestions.
2024-10-21 19:28:33.678+0000 INFO  [o.n.i.d.DiagnosticsManager] 
                                                                ********************************************************************************
                                                                                             [ System diagnostics ]                             
                                                                ********************************************************************************
                                                                --------------------------------------------------------------------------------
                                                                                         [ System memory information ]                          
                                                                --------------------------------------------------------------------------------
                                                                Total Physical memory: 62.55GiB
                                                                Free Physical memory: 47.03GiB
                                                                Committed virtual memory: 20.67GiB
                                                                Total swap space: 2.000GiB
                                                                Free swap space: 2.000GiB
                                                                
                                                                --------------------------------------------------------------------------------
                                                                                           [ JVM memory information ]                           
                                                                --------------------------------------------------------------------------------
                                                                Free  memory: 159.4MiB
                                                                Total memory: 192.0MiB
                                                                Max   memory: 15.64GiB
                                                                Garbage Collector: G1 Young Generation: [G1 Eden Space, G1 Survivor Space, G1 Old Gen]
                                                                Garbage Collector: G1 Old Generation: [G1 Eden Space, G1 Survivor Space, G1 Old Gen]
                                                                Memory Pool: CodeHeap 'non-nmethods' (Non-heap memory): committed=2.938MiB, used=2.880MiB, max=7.254MiB, threshold=0B
                                                                Memory Pool: Metaspace (Non-heap memory): committed=21.81MiB, used=21.60MiB, max=-1B, threshold=0B
                                                                Memory Pool: CodeHeap 'profiled nmethods' (Non-heap memory): committed=6.438MiB, used=6.433MiB, max=116.4MiB, threshold=0B
                                                                Memory Pool: Compressed Class Space (Non-heap memory): committed=2.438MiB, used=2.314MiB, max=1.000GiB, threshold=0B
                                                                Memory Pool: G1 Eden Space (Heap memory): committed=152.0MiB, used=0B, max=-1B, threshold=?
                                                                Memory Pool: G1 Old Gen (Heap memory): committed=32.00MiB, used=24.92MiB, max=15.64GiB, threshold=0B
                                                                Memory Pool: G1 Survivor Space (Heap memory): committed=8.000MiB, used=2.038MiB, max=-1B, threshold=?
                                                                Memory Pool: CodeHeap 'non-profiled nmethods' (Non-heap memory): committed=2.438MiB, used=1.728MiB, max=116.4MiB, threshold=0B
                                                                
                                                                --------------------------------------------------------------------------------
                                                                                        [ Operating system information ]                        
                                                                --------------------------------------------------------------------------------
                                                                Operating System: Linux; version: 5.10.0-1044-oem; arch: amd64; cpus: 16
                                                                Max number of file descriptors: 1048576
                                                                Number of open file descriptors: 243
                                                                Process id: 7
                                                                Byte order: LITTLE_ENDIAN
                                                                Local timezone: Etc/UTC
                                                                Memory page size: 4096
                                                                Unaligned memory access allowed: true
                                                                
                                                                --------------------------------------------------------------------------------
                                                                                              [ JVM information ]                               
                                                                --------------------------------------------------------------------------------
                                                                VM Name: OpenJDK 64-Bit Server VM
                                                                VM Vendor: Eclipse Adoptium
                                                                VM Version: 17.0.12+7
                                                                JIT compiler: HotSpot 64-Bit Tiered Compilers
                                                                VM Arguments: [-XX:+UseG1GC, -XX:-OmitStackTraceInFastThrow, -XX:+AlwaysPreTouch, -XX:+UnlockExperimentalVMOptions, -XX:+TrustFinalNonStaticFields, -XX:+DisableExplicitGC, -Djdk.nio.maxCachedBufferSize=1024, -Dio.netty.tryReflectionSetAccessible=true, -Djdk.tls.ephemeralDHKeySize=2048, -Djdk.tls.rejectClientInitiatedRenegotiation=true, -XX:FlightRecorderOptions=stackdepth=256, -XX:+UnlockDiagnosticVMOptions, -XX:+DebugNonSafepoints, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --enable-native-access=ALL-UNNAMED, -Dlog4j2.disable.jmx=true, -Dfile.encoding=UTF-8]
                                                                
                                                                --------------------------------------------------------------------------------
                                                                                               [ Java classpath ]                               
                                                                --------------------------------------------------------------------------------
                                                                 [classpath] /var/lib/neo4j/lib/reactor-core-3.6.8.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-query-router-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/netty-codec-4.1.112.Final.jar
                                                                 [classpath] /var/lib/neo4j/lib/commons-logging-1.3.3.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-token-api-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-cypher-rendering-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/arrow-format-16.1.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-concurrent-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-server-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/jetty-alpn-server-10.0.20.jar
                                                                 [classpath] /var/lib/neo4j/lib/jaxb-runtime-2.3.2.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-cloud-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/caffeine-3.1.8.jar
                                                                 [classpath] /var/lib/neo4j/lib/magnolia_2.13-1.1.8.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-native-5.23.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/scala-library-2.13.11.jar
                                                                 [classpath] /var/lib/neo4j/lib/neo4j-browser-5.21.0.jar
                                                                 [classpath] /var/lib/neo4j/lib/jackson-datatype-jsr310-2.17.0.jar
...

neo4j.log

2024-10-21 19:28:33.104+0000 INFO  Logging config in use: File '/var/lib/neo4j/conf/user-logs.xml'
2024-10-21 19:28:33.116+0000 INFO  Starting...
2024-10-21 19:28:33.700+0000 INFO  This instance is ServerId{f15f3b4f} (f15f3b4f-2266-48d2-b935-eddf6e2e4ff8)
2024-10-21 19:28:34.338+0000 INFO  ======== Neo4j 5.23.0 ========
2024-10-21 19:28:35.506+0000 INFO  Anonymous Usage Data is being sent to Neo4j, see https://neo4j.com/docs/usage-data/
2024-10-21 19:28:35.530+0000 INFO  Bolt enabled on 0.0.0.0:7687.
2024-10-21 19:28:35.882+0000 INFO  HTTP enabled on 0.0.0.0:7474.
2024-10-21 19:28:35.883+0000 INFO  Remote interface available at http://localhost:7474/
2024-10-21 19:28:35.885+0000 INFO  id: A6EABCFD7DBE15A56963170C7D061BC73104EF29F7ED304A139589B0ED949AFF
2024-10-21 19:28:35.885+0000 INFO  name: system
2024-10-21 19:28:35.885+0000 INFO  creationDate: 2024-10-21T19:17:59.692Z
2024-10-21 19:28:35.886+0000 INFO  Started.
2

Hello @admin9 ,

Did you check whether the backup file is named after the database, neo4j?
Please try to run again your command:

  • If the platform is Windows, try using the backslash \ rather than the forwardslash /
  • Enable verbosity through the --verbose flag
  • Try to use an absolute path for the --from-path parameter

Give it a try and let me know :slight_smile:

3

Hello @cdavide8 ,

Yes, the file is named neo4j and I am on Ubuntu. Here is the CLI output:

root@7a1784d5b5d5:/var/lib/neo4j# ls data/
databases  instances.backup  neo4j.backup  server_id  transactions

As you can see, there are two files in the directory (neo4j.backup and instances.backup) both have been exported from an Aura professional database but now when I tested with the neo4j-file everything worked fine but with the backup-file it fails with the following error message:

root@7a1784d5b5d5:/var/lib/neo4j# neo4j-admin database upload instances --from-path=/var/lib/neo4j/data --to-uri=neo4j+s://... --overwrite-destination=true --verbose
Executing command line: /opt/java/openjdk/bin/java -cp /var/lib/neo4j/plugins/*:/var/lib/neo4j/conf/*:/var/lib/neo4j/lib/* -XX:+UseParallelGC -XX:-OmitStackTraceInFastThrow -XX:+UnlockExperimentalVMOptions -XX:+TrustFinalNonStaticFields -XX:+DisableExplicitGC -Djdk.nio.maxCachedBufferSize=1024 -Dio.netty.tryReflectionSetAccessible=true -XX:+ExitOnOutOfMemoryError -Djdk.tls.ephemeralDHKeySize=2048 -XX:FlightRecorderOptions=stackdepth=256 -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --enable-native-access=ALL-UNNAMED -Dlog4j2.disable.jmx=true -Dfile.encoding=UTF-8 org.neo4j.cli.AdminTool database upload instances --from-path=/var/lib/neo4j/data --to-uri=neo4j+s://... --overwrite-destination=true --verbose
neo4j 5.23.0
VM Name: OpenJDK 64-Bit Server VM
VM Vendor: Eclipse Adoptium
VM Version: 17.0.12+7
JIT compiler: HotSpot 64-Bit Tiered Compilers
VM Arguments: [-XX:+UseParallelGC, -XX:-OmitStackTraceInFastThrow, -XX:+UnlockExperimentalVMOptions, -XX:+TrustFinalNonStaticFields, -XX:+DisableExplicitGC, -Djdk.nio.maxCachedBufferSize=1024, -Dio.netty.tryReflectionSetAccessible=true, -XX:+ExitOnOutOfMemoryError, -Djdk.tls.ephemeralDHKeySize=2048, -XX:FlightRecorderOptions=stackdepth=256, -XX:+UnlockDiagnosticVMOptions, -XX:+DebugNonSafepoints, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --enable-native-access=ALL-UNNAMED, -Dlog4j2.disable.jmx=true, -Dfile.encoding=UTF-8]
Configuration files used (ordered by priority):
/var/lib/neo4j/conf/neo4j-admin.conf
/var/lib/neo4j/conf/neo4j.conf
--------------------
Neo4j aura username (default: neo4j):neo4j
Neo4j aura password for neo4j:
org.neo4j.cli.CommandFailedException: Could not find any archive files
        at org.neo4j.export.UploadCommand.execute(UploadCommand.java:247)
        at org.neo4j.cli.AbstractCommand.call(AbstractCommand.java:92)
        at org.neo4j.cli.AbstractCommand.call(AbstractCommand.java:37)
        at picocli.CommandLine.executeUserObject(CommandLine.java:2045)
        at picocli.CommandLine.access$1500(CommandLine.java:148)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2465)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2457)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2419)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2421)
        at picocli.CommandLine.execute(CommandLine.java:2174)
        at org.neo4j.cli.AdminTool.execute(AdminTool.java:94)
        at org.neo4j.cli.AdminTool.main(AdminTool.java:82)
Caused by: org.neo4j.cli.CommandFailedException: Could not find any archive files
        at org.neo4j.export.UploadCommand.makeDumpUploader(UploadCommand.java:306)
        at org.neo4j.export.UploadCommand.execute(UploadCommand.java:243)
        ... 12 more

So it actually seems like there is something in the instances.backup file that the cli-tool does not like. What could that be? As said, both files are exported from an Aura professional database.

4

Hello,

This is not uploading because the command expects you to specify the database name not the filename. Therefore it should be 'neo4j' not 'instances' (the filename). The command inspects the backup file for the correct database name and errors if it fails to find it.

Therefore if you do:
neo4j-admin database upload neo4j --from-path=/var/lib/neo4j/data --to-uri=neo4j+s://... --overwrite-destination=true --verbose

it should work. However there is another problem in that you cannot have multiple backups with the same database name within the same path specified by --from-path= as the tool won't know which one you want to upload so your second attempt will also fail for a different reason, that is until you remove one of instances.backup or neo4j.backup.

5

I would suggest also to check the permissions of the instances.backup file:

ls -l /var/lib/neo4j/data

1 Like
6

Hi,

This, indeed seems to work. Thank you! I do not, however, think that this advice conforms to the documentation. In the documentation there is an example command:

neo4j-admin database upload <database> --from-path=<path-to-directory-with-database-dump> --to-uri=<neo4j+s://your-aura-instance-id.databases.neo4j.io> --overwrite-destination=true

And the <database> parameter is explained as such:
"Name of the database that should be uploaded. The name is used to select a file which is expected to be named <database>.dump or <database>.backup."

So in case you @james.edwards know who is in charge of the documentation it may be a good idea to ask them to update it.

What I still did not quite understand was why <database> should be neo4j, you say that is the database name. How do I know that that is the database name?