Hello everyone! I'm new to neo4j aura and elasticsearch and understand that aura only supports APOCs. So I learned there's an apoc component that can connect to elasticsearch.
My goal is to use elasticsearch to allow quick searching for data within the neo4j server. However as I cannot find a lot of resources online I might be going about this in an incorrect method as I was told we needed this to optimize searches as searching gets expensive directly on neo4j.
What I've Done
So I spun up two elasticsearch clusters, one using version 6 and 7.4, and tried doing a
call apoc.es.stats("awsendpoint") however I get Failed to invoke procedure
apoc.es.stats: Caused by: java.lang.RuntimeException: Can't read url or key. HTTP response code: 403 for URL.
This is strange as my amazon rules for IP address is wildcarded for all IPs. So perhaps I'm misunderstanding something on amazon's side.
So I have a few questions:
Is this the correct method of integration using apocs to connect an aws elasticsearch to neo4j. The elasticsearch currently has an all public access rule for testing and whitelist all IP addresses. I perhaps may have the wrong url? I'm using the url on the aws dashboard that says "endpoint" but I'm getting an error to connect. What's weird is there's no method of authentication when calling this apoc.
Even if it does get connected, I assume I would have to post data to the elasticsearch server whenever data gets imported into the neo4j server. Therefore is a correct method to do this to use
call apoc.trigger.add( to add a trigger whenever data is changed to store it on elastic search.
An alternative method I'm looking at is when a user makes a post request to put data into neo4j, we also push that data into elastic search and then put it on neo4j. So we have a backend in between the user and neo4j.
Let me know if you have a direction or documents I can read and thanks for letting me be a part of the neo4j ocmmunity!
If anyone in the future is looking at this post. I have discovered that neo4j has an indexing/text search available as an apoc in aura.
What's interesting is that it's built on Apache Lucene which Elasticsearch is also built on. And so far querying has been working quick and fast with one label, 4 keys/properties, 130,000 records/nodes. I only have one issue where the call only streams data, and I want it to return a limit like 50 during its search but uncertain how to do that.
Essentially you can build an index using
db.index.fulltext.createNodeIndex and then query using
CALL db.index.fulltext.queryNodes("titlesAndDescriptions", "your_query_here*") YIELD node, score