cancel
Showing results for 
Search instead for 
Did you mean: 

Join the community at Nodes 2022, our free virtual event on November 16 - 17.

Integrating neo4j aura to AWS Elasticsearch

Keydex
Node

Hello everyone! I'm new to neo4j aura and elasticsearch and understand that aura only supports APOCs. So I learned there's an apoc component that can connect to elasticsearch.

Goal
My goal is to use elasticsearch to allow quick searching for data within the neo4j server. However as I cannot find a lot of resources online I might be going about this in an incorrect method as I was told we needed this to optimize searches as searching gets expensive directly on neo4j.

What I've Done
So I spun up two elasticsearch clusters, one using version 6 and 7.4, and tried doing a call apoc.es.stats("awsendpoint") however I get Failed to invoke procedure apoc.es.stats: Caused by: java.lang.RuntimeException: Can't read url or key. HTTP response code: 403 for URL.

This is strange as my amazon rules for IP address is wildcarded for all IPs. So perhaps I'm misunderstanding something on amazon's side.

So I have a few questions:

  1. Is this the correct method of integration using apocs to connect an aws elasticsearch to neo4j. The elasticsearch currently has an all public access rule for testing and whitelist all IP addresses. I perhaps may have the wrong url? I'm using the url on the aws dashboard that says "endpoint" but I'm getting an error to connect. What's weird is there's no method of authentication when calling this apoc.

  2. Even if it does get connected, I assume I would have to post data to the elasticsearch server whenever data gets imported into the neo4j server. Therefore is a correct method to do this to use call apoc.trigger.add( to add a trigger whenever data is changed to store it on elastic search.

Alternatives
An alternative method I'm looking at is when a user makes a post request to put data into neo4j, we also push that data into elastic search and then put it on neo4j. So we have a backend in between the user and neo4j.

Let me know if you have a direction or documents I can read and thanks for letting me be a part of the neo4j ocmmunity!

1 REPLY 1

Keydex
Node

If anyone in the future is looking at this post. I have discovered that neo4j has an indexing/text search available as an apoc in aura.

What's interesting is that it's built on Apache Lucene which Elasticsearch is also built on. And so far querying has been working quick and fast with one label, 4 keys/properties, 130,000 records/nodes. I only have one issue where the call only streams data, and I want it to return a limit like 50 during its search but uncertain how to do that.

Documentation here.

https://neo4j.com/docs/cypher-manual/current/administration/indexes-for-full-text-search/

Essentially you can build an index using db.index.fulltext.createNodeIndex and then query using CALL db.index.fulltext.queryNodes("titlesAndDescriptions", "your_query_here*") YIELD node, score