NiFi goes Neo

[martin_ebert (M. Ebert)]

Hi community,
I've already tried Kettle and Neo. That went without a hitch.
Now I want to use NiFi and Neo together and encounter problems:

Neo processors are not included in the standard installation of NiFi, but there is a "nifi-graph-bundle" in the official NiFi Github Repo (nifi/nifi-nar-bundles at master · apache/nifi · GitHub).:

Following the instructions I come across some errors.

Since I haven't worked with Maven and Java yet, I don't know if it's the configuration.

Error message after executing "mvn -T 2.0C clean install" in the nifi root directory:

Can someone help me and even provide me with the required nar file? Initially I wanted to reach the graph database via JDBC, but of course the corresponding Execute SQL processor doesn't understand Cypher.

[martin_ebert (M. Ebert)]

First of all, the graph bundle is coming but unfortunately not with nifi 1.9.2 but with 1.10.0 RC.

I also found out what it takes to successfully create the package: Execute

mvn clean install -DskipTests

to compile tests, but skip running them.

So is there currently no way to work with Neo4j on nifi 1.9.2?

[martin_ebert (M. Ebert)]

NiFi 1.10.0 is out now.

#Bad News
The graph bundle is not included in the original nifi 1.10.0 but you can create the bundle by yourself.

#Good News
The processor works perfectly and only with URL, username and password.

You can choose:

The Controller Service is set up in seconds:

Processor is really simple:

Most importantly, it works:

[tom1 (Tom)]

Thanks for the guide.

I'm trying the same setup and can successfully query Neo4j using Cypher in the ExecuteGraphQuery when returning one node. However, when I query for multiple nodes I get an empty array back. For example, following this tutorial: First Steps with Cypher - graphgists

Running this query:

CREATE (you:Person {name:"You"})
RETURN you

Result:

[{"name":"You"}]

But a Cypher query with multiple nodes, gives an empty JSON array:

Query:

MATCH  (you:Person {name:"You"})
CREATE (you)-[like:LIKE]->(neo:Database {name:"Neo4j" })
RETURN you,like,neo

Result:
[{"you":{},"like":{},"neo":{}}]

Does anyone know how to solve this? we're using the graph bundle from 1.10

[yyyguy]

Hey @tom1,

What database version of Neo4j are you using? I have found an issue with particular versions of the Neo4j database not working properly with my NiFi implementation.

-yyyguy

[tom1 (Tom)]

Hi @yyyguy
We're using a Neo4j Docker image with version 3.5.15 and can connect fine:

[yyyguy]

That's good. Have you confirmed that the Neo4JCypherClientService controller service is Enabled?

I apologize for asking such basic questions. It is just that I have had this controller service sitting in an "Enabling" state without me realizing it.

-yyyguy

[tom1 (Tom)]

Yes, definitely enabled and we can run the Cypher queries fine:

[martin_ebert (M. Ebert)]

Did anyone manage to ingest the FlowFile content? No matter if Parquet or json format.

I could well imagine that this could work with Load JSON. I can save the FlowFile content as an attribute if needed and reuse it directly in the Cypher statement.

I just want to save the intermediate step (save data locally and write it to Neo4j)

[yyyguy]

Great. If you do a simple query like:

MATCH (n:Person)
RETURN n.name

it should return multiple results, however you are seeing no results returned? Is that correct?

-yyyguy

[tom1 (Tom)]

@yyyguy

MATCH (n:Person) RETURN n.name

Yes, that query runs fine giving this:

However, a query that returns multiple nodes in the Neo4j browser:

Gives an empty JSON array in the Flowfile content:

[yyyguy]

@tom1 can you run the query that I provided. I would like to see what the results are without the "you" filter. Also, can you provide a screenshot of the results in the LogAttribute processor on the success branch?

-yyyguy

[tom1 (Tom)]

Sure:

Gives this in the 'Success' branch:

Flowfiles all contain this:

[yyyguy]

Ok, so in the next query can you please try this? Note: This assumes that you already have a Database node with name = "Neo4j"

MATCH (you:Person {name:"You"})
MERGE (you)-[:LIKE]->(neo:Database {name:"Neo4j" })
RETURN you.name, neo.name

-yyyguy

[tom1 (Tom)]

Now this query:

@yyyguy

MATCH (you:Person {name:"You"})
MERGE (you)-[:LIKE]->(neo:Database {name:"Neo4j" })
RETURN you.name, neo.name

OK this works:

[yyyguy]

@tom1, it looks like if you do not have any properties associated with a relationship [like:LIKE], the query will fail.

Do you have any relationship properties that you plan to include in your database design? If not, do you think that is workable?

-yyyguy

[yyyguy]

BTW, it looks like you have created 3 relationships between the You node and the Neo4J node. MERGE is a better option perhaps.

-yyyguy

[tom1 (Tom)]

OK great - thank you for your help. We'll see if we can rework our schema and then try our query again

[kevinneo4j (Kevin)]

hi

Thanks @yyyguy and @martin_ebert for information above. I have Apache Nifi 1.12 working against Neo4j 3.5.15.

However, when using Apache Nifi 1.12.0 I have problems connecting to the Neo4J version 4.0.4. The Neo4JCypherClientService throws a "org.neo4j.driver.v1.exceptions.ServiceUnavailableException. Connection to the database terminated"

You mentioned that you did have problems connecting with different versions of Neo4j. I was wondering if you had a chance to look at this and if there was a fix on the way.

Regards,
Kevin

[martin_ebert (M. Ebert)]

Hi Kevin,
you talk about NiFi 1.11.x. 1.12 is not released yet. Anyways. I noticed the exact same error message. The neo4j nar files are not 4.0 compatible.