How to pass a variable into apoc.periodic.iterate

Neo4j Graph Platform Procedures & APOC
, , ,
1

Hi,
I'm trying to read a list of json files with apoc.load.directory and then import each file using apoc.load.json and apoc.periodic.iterate.

My code looks like this:

call apoc.load.directory('*.json', 'stack') yield value as filepaths
unwind filepaths as filepath
call apoc.periodic.iterate("call apoc.load.json(filepath) yield value return split(filepath, $splitter)[1] as l, value",
"merge (v:l{Id: value.Id}) on create set v = value, v.Id = toInteger(value.Id)", {parallel: true}) yield total return total

I get this error:

Failed to invoke procedure `apoc.periodic.iterate`: Caused by: org.neo4j.exceptions.SyntaxException: Variable `filepath` not defined (line 1, column 29 (offset: 28))
"EXPLAIN call apoc.load.json(filepath) yield value return split(filepath, $splitter)[1] as l, value"

In accordance with the documentation, this code works (basically doesn't use apoc.periodic.iterate so all variables are in the same scope):

call apoc.load.directory('*.json', 'stack') yield value as filepaths
unwind filepaths as filepath
call apoc.load.json(filepath) yield value
with split(filepath, $splitter)[1] as l, value
merge (v:l{Id: value.Id}) on create set v = value, v.Id = toInteger(value.Id)
return value, l

However the dataset is large and I really have to make use of the iterate procedure.
How can I pass the variable filepath into the procedure call?
Thanks!

2

Try this. Where is $splitter coming from.

call apoc.load.directory('*.json', 'stack') yield value as filepaths
unwind filepaths as filepath
call apoc.periodic.iterate(
  "call apoc.load.json($filepath) yield value return split($filepath, $splitter)[1] as l, value",
  "merge (v:l{Id: value.Id}) on create set v = value, v.Id = toInteger(value.Id)", 
  {params: {filepath: filepath}, parallel: true}
) yield total 
return total
3

Hi Gary,
thanks for help. I've been trying something similar, but I haven't seen anywhere that I should use the params keyword.
$splitter was a parameter that I set earlier and it's value stayed the same throughout the whole query.
To not mix global parameters and variables, I've modified my query and it finally worked!
(Also fixed some other coding error)

call apoc.load.directory('*.json', 'stack') yield value as filepaths
unwind filepaths as filepath
with filepath, '\\' as splitter 
call apoc.periodic.iterate("call apoc.load.json($filepath) yield value return value, split($filepath, $splitter)[1] as l",
"merge (v:l{Id: toInteger(value.Id)}) on create set v = value, v.Id = toInteger(value.Id)",
{params:{filepath: filepath, splitter: splitter}, batchSize: 1000, parallel: true})
yield total, committedOperations, failedOperations, errorMessages, operations, updateStatistics
return *
1 Like