cancel
Showing results for 
Search instead for 
Did you mean: 

Cannot load data from cypher script

deven_pandey
Node Clone

Below is my content of cyper file.

WITH "file:///cite1.json" AS url
CALL apoc.load.json(url) YIELD value
UNWIND value.citation as q with q where q.venue.id is not null
MERGE (citation:citations {id:q.id}) ON CREATE
SET citation.id = q.id, citation.title = q.title, citation.year = q.year, citation.n_citation = q.n_citation, citation.page_start = q.page_start, citation.page_end = q.page_end, citation.doc_type = q.doc_type, citation.publisher = q.publisher, citation.volume = q.volume, citation.issue = q.issue, citation.doi = q.doi

MERGE (venue:venue {id:q.venue.id}) ON CREATE SET venue.id = q.venue.id, venue.raw = q.venue.raw, venue.type = q.venue.type
MERGE (venue)-[:FOR]->(citation)

FOREACH (refName IN q.references | MERGE (ref:References {name:refName}) MERGE (citation)-[:REFERRED_BY]->(ref))

FOREACH (a IN q.authors |
MERGE (author:authors {id:a.id}) ON CREATE
SET author.name = a.name, author.id = a.id
MERGE (citation)<-[:AUTHORED_BY]-(author)
MERGE (citation)-[:WROTE_BY]->(author))

FOREACH (f IN q.fos |
MERGE (fos:fos {name:f.name}) ON CREATE
SET fos.name = f.name, fos.w = f.w
MERGE (citation)<-[:FOS_DETAILS]-(fos)
)

When I run

cat /tmp/load_data.cypher | /usr/bin/cypher-shell -u usename -p password

nothing happens. no error but data is not loaded

13 REPLIES 13

dana_canzano
Neo4j
Neo4j

is your cypher statement terminated by a ; ?

also you should be aware that if environment variables NZ_USER and NZ_PASSWORD are defined, representing the username and password to connect with, then you no longer need to user -u username -p password

When I use semicolon at the end I get below error. I used semicolon after the final right parentheses. Same query works when logged into cypher-shell

invalid input 'p': expected 'n/n' or 's/s'

What version of Neo4j? though I suspect this shouldnt be of issue but I am unable to replicate using Neo4j 4.1.0 and on a ubuntu os.

I added a ; as the final character to the cypher statement and thus the last 3 line looks like

 tail -3 test
SET fos.name = f.name, fos.w = f.w
MERGE (citation)<-[:FOS_DETAILS]-(fos)
);

and my file has 22 lines, as evidence

wc -l test
22 test

and yet

 cat test | ./cypher-shell
neo4j@neo4j-lg:~/single/instance1/neo4j-enterprise-4.1.0/bin$

i.e. no error is logged?

Yes no error logged but it doesn't even inserts data

ok.. now im confused.
but previously you indicated you recieved an error

When I use semicolon at the end I get below error. I used semicolon after the final right parentheses. Same query works when logged into cypher-shell

invalid input 'p': expected 'n/n' or 's/s'

and your last update now indicates

Yes no error logged but it doesn't even inserts data

so the error is no more? ???

Ok let me elborate.

this is the script.

#!/bin/bash

cd /tmp
ls -1 cite* > list_file

for i in cat list_file
do
cat $i > tmp_file
sed -e "s/\r//g" tmp_file > $i
#sed -e "s/^M//" tmp_file > $i
echo "$i"

echo "WITH "file:///$i" AS url CALL apoc.load.json(url) YIELD value UNWIND value.citation as q with q where q.venue.id is not null MERGE (citation:citations {id:q.id}) ON CREATE SET citation.id = q.id, citation.title = q.title, citation.year = q.year, citation.n_citation = q.n_citation, citation.page_start = q.page_start, citation.page_end = q.page_end, citation.doc_type = q.doc_type, citation.publisher = q.publisher, citation.volume = q.volume, citation.issue = q.issue, citation.doi = q.doi MERGE (venue:venue {id:q.venue.id}) ON CREATE SET venue.id = q.venue.id, venue.raw = q.venue.raw, venue.type = q.venue.type MERGE (venue)-[:FOR]->(citation) FOREACH (refName IN q.references | MERGE (ref:References {name:refName}) MERGE (citation)-[:REFERRED_BY]->(ref)) FOREACH (a IN q.authors | MERGE (author:authors {id:a.id}) ON CREATE SET author.name = a.name, author.id = a.id, author.org = a.org MERGE (citation)<-[:AUTHORED_BY]-(author) MERGE (citation)-[:WROTE_BY]->(author)) FOREACH (f IN q.fos | MERGE (fos:fos {name:f.name}) ON CREATE SET fos.name = f.name, fos.w = f.w MERGE (citation)<-[:FOS_DETAILS]-(fos));" > /tmp/load_data.cypher

#cat /tmp/load_data.cypher
cat /tmp/load_data.cypher | /usr/bin/cypher-shell

Now I don't get an error but it doesn't load data as well

devendra@cs-web-drupal:/var/lib/neo4j/data/tmp$ ./load_data.sh
cite2.json
cite2.json processed

Shows processed instantly. However on cypher shell running this query takes some time

What version of Neo4j ? Again maybe it doesnt matter ? maybe it does? ?? ?

also, your cypher is performing a MERGE which is effectively a create or replace. if its replacing then you would see no new nodes/relationships created

While running manually it shows some relationship. How do I ensure that is is actually inserting processing or replacing data. Is there a way? sorry I am new to Neo4j

deven_pandey
Node Clone

Here's the result running manually.

devendra@neo4j> WITH "file:///cite4.json" AS url
CALL apoc.load.json(url) YIELD value
UNWIND value.citation as q with q where q.venue.id is not null
MERGE (citation:citations {id:q.id}) ON CREATE
SET citation.id = q.id, citation.title = q.title, citation.year = q.year, citation.n_citation = q.n_citation, citation.page_start = q.page_start, citation.page_end = q.page_end, citation.doc_type = q.doc_type, citation.publisher = q.publisher, citation.volume = q.volume, citation.issue = q.issue, citation.doi = q.doi

            MERGE (venue:venue {id:q.venue.id}) ON CREATE SET venue.id = q.venue.id, venue.raw = q.venue.raw, venue.type = q.venue.type
            MERGE (venue)-[:FOR]->(citation)

            FOREACH (refName IN q.references | MERGE (ref:References {name:refName}) MERGE (citation)-[:REFERRED_BY]->(ref))

            FOREACH (a IN q.authors |
            MERGE (author:authors {id:a.id}) ON CREATE
            SET author.name = a.name, author.id = a.id
            MERGE (citation)<-[:AUTHORED_BY]-(author)
            MERGE (citation)-[:WROTE_BY]->(author))


            FOREACH (f IN q.fos |
            MERGE (fos:fos {name:f.name}) ON CREATE
            SET fos.name = f.name, fos.w = f.w
            MERGE (citation)<-[:FOS_DETAILS]-(fos)
            );

0 rows available after 124104 ms, consumed after another 0 ms
Created 1902 relationships

deven_pandey
Node Clone

Version info

Connected to Neo4j 4.2.0

clem
Graph Steward

What I do when I have trouble with an import (or anything at all complex) with Cypher, is make a version that is simple as possible and gradually build up from there. (You can also use PROFILE to see where the data "disappears".)

e.g. start with

WITH "file:///cite1.json" AS url
CALL apoc.load.json(url) YIELD value
RETURN value.citation, value.venue.id

and see if you are getting what you expect. (If you had a typo in a field name, Cypher doesn't complain...)

and slowly add statements. Also, do this on a clone of your DB, which you can delete if it doesn't work, so you don't pollute your existing data.

It's a bit hard to eyeball this much Cypher code and see what's wrong.

You might want to follow this thread on Debugging Cypher:

My same query works from within cypher-shell only problem is it doesn't work when I put same query in a file and try to run that using cypher script.

What if you have a very simple query in your file, like RETURN 'Hello world'. Does it work?

We need to separate the script vs. executing from a file.