# Query running endlessly for large input file **URL:** \ **Category:** Cypher\ **Created:** [April 21, 2020, 8:17pm UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434 "2020-04-21T20:17:57Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![kevinvivk](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/kevinvivk/32/4300_2.png) [@kevinvivk](https://community.neo4j.com/u/kevinvivk)\ **Post date:** [April 21, 2020, 8:17pm UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/1 "2020-04-21T20:17:57Z")
Hello, I am using Neo4J Community 3.5.17. I wanted to find the closest :Fraud node to a :Person node for a list of fids(Person unique identifier) and am using the following query. Please note fids are currently of string type whereas the input and output files contain fid in int type. `profile cypher runtime=interpreted load csv with headers from 'file:///shortest_path_data/test.csv' as line with line.fid as fid match (n:Person) where n.fid=toString(fid) with n call apoc.path.expandConfig(n,{labelFilter:'/Fraud', maxLevel:10, optional:true, limit:1}) yield path return toInteger(n.fid) as fid,length(path)/2 as distance;` This query worked for an input file of 30K fids, however runs endlessly and triggers GC for 300K fids. Here is the debug log- ```auto 2020-04-21 19:55:57.554+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=417944, gcTime=313357, gcCount=11} 2020-04-21 19:59:39.207+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=322338, gcTime=322433, gcCount=10} 2020-04-21 20:05:03.770+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=327987, gcTime=328082, gcCount=9} ``` Current Heap Size is 31g and pagecache size is 100g Please guide me with how to go forward with this. Thanks and Regards, Kevin ---
**Author:** ![dana\_canzano](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/dana_canzano/32/27392_2.png) [@dana\_canzano](https://community.neo4j.com/u/dana_canzano)\ **Post date:** [April 21, 2020, 11:47pm UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/2 "2020-04-21T23:47:27Z")
if you rerun and include a PERIODIC COMMIT ( [LOAD CSV - Cypher Manual](https://neo4j.com/docs/cypher-manual/4.0/clauses/load-csv/#load-csv-setting-the-rate-of-periodic-commits) ) such that you commit every 5k records, for example does this provide any improvement? ---
**Author:** ![andrew\_bowman](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/andrew_bowman/32/73_2.png) [@andrew\_bowman](https://community.neo4j.com/u/andrew_bowman)\ **Post date:** [April 22, 2020, 12:35am UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/3 "2020-04-22T00:35:50Z")
In addition to Dana's suggestion, make sure you have an index on :Person(fid) ---
**Author:** ![kevinvivk](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/kevinvivk/32/4300_2.png) [@kevinvivk](https://community.neo4j.com/u/kevinvivk)\ **Post date:** [April 22, 2020, 5:29am UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/4 "2020-04-22T05:29:38Z")
Hi Team, I do have an index on :Person(fid). On using periodic commit, I get the following error- ```auto Cannot use periodic commit in a non-updating query (line 1, column 36 (offset: 35)) "using periodic commit 5000 load csv with headers from 'file:///shortest_path_data/test.csv' as line with line.fid as fid match (n:Person) where n.fid=toString(fid) with n call apoc.path.expandConfig(n,{labelFilter:'/Fraud', maxLevel:10, optional:true, limit:1}) yield path return toInteger(n.fid) as fid,length(path)/2 as distance;" ``` Thanks and Regards, Kevin ---
**Author:** ![andrew\_bowman](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/andrew_bowman/32/73_2.png) [@andrew\_bowman](https://community.neo4j.com/u/andrew_bowman)\ **Post date:** [April 22, 2020, 7:14am UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/5 "2020-04-22T07:14:57Z")
Ah, you need to prefix the query with `:auto` for this to work in the browser or cypher-shell. For the explanation why, see here: > ---
**Author:** ![kevinvivk](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/kevinvivk/32/4300_2.png) [@kevinvivk](https://community.neo4j.com/u/kevinvivk)\ **Post date:** [April 22, 2020, 10:05am UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/6 "2020-04-22T10:05:50Z")
Hi Andrew, Getting the following error- ```auto Invalid input ':': expected (line 1, column 36 (offset: 35)) ":auto using periodic commit 5000 load csv with headers from 'file:///shortest_path_data/test.csv' as line with line.fid as fid match (n:Person) where n.fid=toString(fid) with n call apoc.path.expandConfig(n,{labelFilter:'/Fraud', maxLevel:10, optional:true, limit:1}) yield path return toInteger(n.fid) as fid,length(path)/2 as distance;" ``` Regards, Kevin ---
**Author:** ![andrew\_bowman](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/andrew_bowman/32/73_2.png) [@andrew\_bowman](https://community.neo4j.com/u/andrew_bowman)\ **Post date:** [April 22, 2020, 11:42pm UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/7 "2020-04-22T23:42:44Z")
How are you running this? Via the Neo4j Browser (if so which version)? Cypher-shell? Client code via a driver? ---
**Author:** ![kevinvivk](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/kevinvivk/32/4300_2.png) [@kevinvivk](https://community.neo4j.com/u/kevinvivk)\ **Post date:** [April 23, 2020, 4:54am UTC](https://community.neo4j.com/t/query-running-endlessly-for-large-input-file/17434/8 "2020-04-23T04:54:21Z")
I am using cypher-shell.