cancel
Showing results for 
Search instead for 
Did you mean: 

Merge functionality using admin import tool

I want to merge two Neo4j Databases. For that, I am trying to use admin import tool.
As the performance is of highest concern for me, spinning up of a new db is perfect.
Hence, I am trying to get the database details as apoc.export.csv from both the databases and then use admin import function to import the csv files.

bin/neo4j-admin import --database=neo4j \
    --nodes=/data/$commonfileNamePrefix"_src_nodeheader.csv",/data/$commonfileNamePrefix"_src_nodes.csv" \
    --nodes=/data/$commonfileNamePrefix"_dest_nodeheader.csv",/data/$commonfileNamePrefix"_dest_nodes.csv" \
    --relationships=/data/$commonfileNamePrefix"_src_relationheader.csv",/data/$commonfileNamePrefix"_src_relationships.csv" \
    --relationships=/data/$commonfileNamePrefix"_dest_relationheader.csv",/data/$commonfileNamePrefix"_dest_relationships.csv" \
    --ignore-empty-strings=true --skip-duplicate-nodes=true

However, this solution works fine, when either of the db is empty or both are empty. Its failing when both have data.
Please provide any suggestions or inputs. Any flag which can be used with import command also would help.
Error:

aused by:ERROR in input
  data source: BufferedCharSeeker[source:/data/2022.05.02-12.16.37_dest_relationships.csv, position:328792, line:8761]
  in field: _end:END_ID:10
  for header: [:IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, _start:START_ID, _end:END_ID, _type:TYPE, bit_length:string, start_bit_offset:string]
  raw field value: 8799
  original error: Requested index -1, but length is 1000000

There were bad entries which were skipped and logged into /var/lib/neo4j/import.report
WARNING Import failed. The store files in /data/databases/neo4j are left as they are, although they are likely in an unusable state. Starting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
org.neo4j.internal.batchimport.input.InputException: ERROR in input
  data source: BufferedCharSeeker[source:/data/2022.05.02-12.16.37_dest_relationships.csv, position:328792, line:8761]
  in field: _end:END_ID:10
  for header: [:IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, _start:START_ID, _end:END_ID, _type:TYPE, bit_length:string, start_bit_offset:string]
  raw field value: 8799
  original error: Requested index -1, but length is 1000000
        at org.neo4j.internal.batchimport.input.csv.CsvInputParser.next(CsvInputParser.java:234)
        at org.neo4j.internal.batchimport.input.csv.LazyCsvInputChunk.next(LazyCsvInputChunk.java:98)
        at org.neo4j.internal.batchimport.input.csv.CsvInputChunkProxy.next(CsvInputChunkProxy.java:75)
        at org.neo4j.internal.batchimport.ExhaustingEntityImporterRunnable.run(ExhaustingEntityImporterRunnable.java:57)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.neo4j.internal.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:110)
        Suppressed: org.neo4j.internal.batchimport.input.InputException: ERROR in input
  data source: BufferedCharSeeker[source:/data/2022.05.02-12.16.37_src_relationships.csv, position:337554, line:8761]
1 REPLY 1

Cobra
Ninja
Ninja

Hello @sumagowrishan

Did you read the documentation here?

The neo4j-admin import command does not create a database, the command only imports data and make it available for the database. The database must not exist before the neo4j-admin import command has been executed, and the database should be created afterwards. The command will exit with an error message if the database already exists.

Regards,
Cobra

Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

On November 16 and 17 for 24 hours across all timezones, you’ll learn about best practices for beginners and experts alike.