Cannot Load data into Neo4j using AWS S3 pre signed URL

Neo4j Graph Platform Import / Export
1

Hi , I am trying to load data from AWS s3 bucket using a Pre signed URL and getting the below error

Command -
LOAD CSV WITH HEADERS FROM '<presigned_url>' AS ROW RETURN ROW LIMIT 10;

Error -
Neo.ClientError.Statement.ExternalResourceFailed: Couldn't load the external resource at: https://.s3.amazonaws.com/temp.csv?AWSAccessKeyId=***********&Signature=****security-token=

My Neo4j is running on Kubernetes. Should my file have to be publicly accessible ?? What other options I can check ? I have also set dbms.security.allow_csv_import_from_file_urls=true

2

a KB was created for this sometime ago and as described at

can you review and see if it fills in the missing details

3

I saw two probable solutions there -

  1. set dbms.security.allow_csv_import_from_file_urls=true
  2. neo4j.conf, comment out or delete dbms.directories.import=import

how to comment out or delete dbms.directories.import=import during Neo4j installation through Kubernetes.

4

In Kubernetes, neo4j runs within docker containers. Use these docs and use the "environment variable syntax" for passing the config you need to your container, and the docker/kubernetes install works just like the regular Neo4j install method.

5

David,
We are using helm command to install neo4j. I am not sure how to set properties neo4j.conf while installing through heml. I am new to Kubernetes. Could you please help? Also do we need check our bucket policies/ IAM roles ? Can this be a reason for this error ?

6

David
Below are the kubernetes commands used in our .gitlab-ci.yaml file

  • export KOPS_STATE_STORE=s3://<bucket_name>
    • export NAME=<cluster_name>
    • export KUBECONFIG=$HOME/.kube/config
    • kops export kubecfg --name=$NAME
    • kubectl apply -f ./k8s/ingress-bolt.yaml
    • kubectl apply -f ./k8s/ingress.yaml
    • helm upgrade --install neo4j stable/neo4j -f k8s/values.yaml --namespace dev-tem
    • kubectl get services
    • kubectl get pods --all-namespaces

Let me know if you need any other information.
Thanks

7

What you've got here is not the right approach. What you rather need to do is get a copy of the helm chart templates and adjust the environment variables passed to the pod to add your own custom configuration. Any variables you do with export here will have no effect.

When you type helm upgrade --install stable/neo4j this repo is where that code resides. You should grab the code this repo, and adjust the environment variables right here: