Neo4j-admin copy will not copy over parameters or indexes

Neo4j Graph Platform
1

This was a painful lesson. I recently started using neo4j-admin copy ... which is great. But after hours of trying to figure out why the copied version as dramatically slower than the original, I discovered that I have to create all the indexes again. Not only that, but the parameters are also not copied over.

Wondering if there is a setting in the copy command to overcome (IMHO - copying an entire database should be an exact copy, including variables and indexes).

2

Copy a database store - Operations Manual states

The command copies only the data store but not the schema store (index and constraints), and therefore, it allows for a large range of migrations. However, if there is a schema defined, you have to recreate it by running the Cypher commands that the neo4j-admin copy operation outputs.

so 2 points.

  1. it states indexes are not copied forward
  2. the output of neo4j-admin copy will produce the index statements you need to run for example
neo4j@system> create database rmasrani;
0 rows available after 123 ms, consumed after another 0 ms
neo4j@system> :use rmasrani
neo4j@rmasrani> create  index for (n:Person) on (n.id);
0 rows available after 134 ms, consumed after another 0 ms
Added 1 indexes
neo4j@rmasrani> unwind range (1,1000) as x create (n:Person {id:x});
0 rows available after 744 ms, consumed after another 0 ms
Added 1000 nodes, Set 1000 properties, Added 1000 labels
neo4j@rmasrani> :use system
neo4j@system> stop database rmasrani
              ;
0 rows available after 246 ms, consumed after another 0 ms
neo4j@system> :exit

and upon running neo4j-admin copy

./neo4j-admin copy --from-database=rmasrani --to-database=rmasraniNew
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
Starting to copy store, output will be saved to: /home/neo4j/single/instance1/neo4j-enterprise-4.2.3/logs/neo4j-admin-copy-2021-04-07.20.23.46.log
2021-04-07 20:23:47.436+0000 INFO  [StoreCopy] ### Copy Data ###
2021-04-07 20:23:47.438+0000 INFO  [StoreCopy] Source: /home/neo4j/single/instance1/neo4j-enterprise-4.2.3/data/databases/rmasrani (page cache 8m)
2021-04-07 20:23:47.443+0000 INFO  [StoreCopy] Target: /home/neo4j/single/instance1/neo4j-enterprise-4.2.3/data/databases/rmasraninew (page cache 8m)
2021-04-07 20:23:47.444+0000 INFO  [StoreCopy] Empty database created, will start importing readable data from the source.
2021-04-07 20:23:48.220+0000 INFO  [o.n.i.b.ImportLogic] Import starting

Import starting 2021-04-07 20:23:48.307+0000
  Estimated number of nodes: 1.00 k
  Estimated number of node properties: 1.01 k
....
.......
..........
2021-04-07 20:23:53.316+0000 INFO  [StoreCopy] ... found 1 schema definitions. The following can be used to recreate the schema:
2021-04-07 20:23:53.329+0000 INFO  [StoreCopy]

CALL db.createIndex('index_9e262879', ['Person'], ['id'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]})
2021-04-07 20:23:53.331+0000 INFO  [StoreCopy] You have to manually apply the above commands to the database when it is started to recreate the indexes and constraints. The commands are saved to /home/neo4j/single/instance1/neo4j-enterprise-4.2.3/logs/neo4j-admin-copy-2021-04-07.20.23.46.log as well for reference.

to which the last line, and 2nd to last report the indexes in the database I copied from

Regarding parameters? which parameters specifically? parameters of neo4j.conf? query parameters or something else

3

Thanks for this. I was basically doing just that – copying the database, and re-creating the indexes.

As for parameters, I am talking about the following:

:param funcList => **["TOP"**, "UXX"****, "FXX"****, "IXX"****, "OXX"****];

:param externalCalls => **["AFRM"**, "FILI"****, "FILO"****, "FTP"****, "IDCI"****, "IDCM"****, "IDCO"****, "SFRM"****, "WEBI"****];

:param objectFilter => **['AUTH'**, 'CDAT'****, 'CLAS'****, 'DTEL'****, 'FUGR'****, 'FUNC'****, 'INTF'****, 'METH'****, 'MSAG'****, 'PROG'****, 'SQLT'****, 'STOB'****, 'STRU'****, 'SUBM'****, 'SXSD'****, 'TABL'****, 'TABU'****, 'TDAT'****, 'TRAN'****, 'TTYP'****, 'TYPE'****, 'VDAT'****, 'VIEW'****, 'WDYN'****, 'WEBI'****];

:param progFilter => **["ALE"**, "SAPLNLSK_G"****, "SAPLERFC"****, "SAPLCOKO"****, "SAPMHTTP"****, "AutoABAP"****, "saplogon"****, "RFCPING"****, "SAPLRSAOS"****, "(BATCH)"****, "java"****, "RE"****, "RFC"****, "RW"****];

:param progFilterBeginsWith => **["D"**, "C:"****, "GP"****, "/"****, "_"****, "<"****];

:param progFilterContainsString => **["=E"**, "TABLEFRAME"****, "TABLEPROC"****, "BAPI"****, "FIELD_EXIT"****, "Generated"****];

4

@rmasrani

but :param are session centric. When the session exits, the parameter are gone. And since you need to stop the database to then run neo4j-admin copy and a stopped database would end all sessions I would not expect :param to be carried forward.

5

I actually did not realize that :param is session specific. Now that I know it makes sense.