cancel
Showing results for 
Search instead for 
Did you mean: 

First time Cluster in AWS : Connection Refused

mike_r_black
Ninja
Ninja

I'm trying to set up my first cluster in AWS. I have 3 EC2 instances. Following the manual and it appears as if the cluster is established, but when I try to cypher-shell from one of the instances I get a Connection Refused. Any tips where to go about looking to try and figure out how to get this cluster going? I'll paste some what I've found that I think could be relevant. Some AWS setup notes:

  • Each EC2 instance is in the same VPC and Security group
  • All traffic is allowed within the Security Group so all ports between EC2s for the cluster to communicate should be allowed
  • Cluster definitions are referencing the private IPv4 addresses

Service Status of Node 1

ubuntu@ip-172-31-24-72:~$ sudo service neo4j status
● neo4j.service - Neo4j Graph Database
   Loaded: loaded (/lib/systemd/system/neo4j.service; disabled; vendor preset: enabled)
   Active: active (running) since Sun 2019-08-25 04:03:07 UTC; 4min 29s ago
 Main PID: 6189 (java)
    Tasks: 53 (limit: 1152)
   CGroup: /system.slice/neo4j.service
           └─6189 /usr/bin/java -cp /var/lib/neo4j/plugins:/etc/neo4j:/usr/share/neo4j/lib/*:/var/lib/neo4j/plugins/* -server -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFina

Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]: 2019-08-25 04:03:12.972+0000 INFO  My connection info: [
Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]:         Discovery:   listen=0.0.0.0:5000, advertised=localhost:5000,
Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]:         Transaction: listen=0.0.0.0:6000, advertised=localhost:6000,
Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]:         Raft:        listen=0.0.0.0:7000, advertised=localhost:7000,
Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]:         Client Connector Addresses: bolt://localhost:7687,http://localhost:7474,https://localhost:7473
Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]: ]
Aug 25 04:03:12 ip-172-31-24-72 neo4j[6189]: 2019-08-25 04:03:12.976+0000 INFO  Discovering other core members in initial members set: [172.31.23.197:5000, 172.31.29.208:5000]
Aug 25 04:03:44 ip-172-31-24-72 neo4j[6189]: 2019-08-25 04:03:44.297+0000 INFO  Bound to cluster with id 13a49e26-3218-4a54-9974-0662e7e530e1
Aug 25 04:03:44 ip-172-31-24-72 neo4j[6189]: 2019-08-25 04:03:44.303+0000 INFO  Discovered core member at 172.31.29.208:5000
Aug 25 04:03:49 ip-172-31-24-72 neo4j[6189]: 2019-08-25 04:03:49.212+0000 INFO  Discovered core member at 172.31.23.197:5000

Service Status Node 2

ubuntu@ip-172-31-29-208:~$ sudo service neo4j status
● neo4j.service - Neo4j Graph Database
   Loaded: loaded (/lib/systemd/system/neo4j.service; disabled; vendor preset: enabled)
   Active: active (running) since Sun 2019-08-25 04:03:33 UTC; 34min ago
 Main PID: 2667 (java)
    Tasks: 53 (limit: 1152)
   CGroup: /system.slice/neo4j.service
           └─2667 /usr/bin/java -cp /var/lib/neo4j/plugins:/etc/neo4j:/usr/share/neo4j/lib/*:/var/lib/neo4j/plugins/* -server -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFina

Aug 25 04:03:38 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:03:38.658+0000 INFO  Discovering other core members in initial members set: [172.31.23.197:5000, 172.31.24.72:5000]
Aug 25 04:03:48 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:03:48.572+0000 INFO  Bound to cluster with id 13a49e26-3218-4a54-9974-0662e7e530e1
Aug 25 04:03:48 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:03:48.635+0000 INFO  Discovered core member at 172.31.24.72:5000
Aug 25 04:03:49 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:03:49.211+0000 INFO  Discovered core member at 172.31.23.197:5000
Aug 25 04:14:04 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:14:04.353+0000 WARN  Lost core member at 172.31.23.197:5000
Aug 25 04:14:14 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:14:14.647+0000 INFO  Discovered core member at 172.31.23.197:5000
Aug 25 04:24:29 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:24:29.167+0000 WARN  Lost core member at 172.31.23.197:5000
Aug 25 04:24:39 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:24:39.741+0000 INFO  Discovered core member at 172.31.23.197:5000
Aug 25 04:34:53 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:34:53.722+0000 WARN  Lost core member at 172.31.23.197:5000
Aug 25 04:35:04 ip-172-31-29-208 neo4j[2667]: 2019-08-25 04:35:04.235+0000 INFO  Discovered core member at 172.31.23.197:5000

Service Status Node 3

ubuntu@ip-172-31-23-197:~$ sudo service neo4j status
● neo4j.service - Neo4j Graph Database
   Loaded: loaded (/lib/systemd/system/neo4j.service; disabled; vendor preset: enabled)
   Active: active (running) since Sun 2019-08-25 04:34:53 UTC; 3min 45s ago
 Main PID: 4062 (java)
    Tasks: 68 (limit: 1152)
   CGroup: /system.slice/neo4j.service
           └─4062 /usr/bin/java -cp /var/lib/neo4j/plugins:/etc/neo4j:/usr/share/neo4j/lib/*:/var/lib/neo4j/plugins/* -server -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFina

Aug 25 04:35:09 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:35:09.419+0000 INFO  Bound to cluster with id 13a49e26-3218-4a54-9974-0662e7e530e1
Aug 25 04:35:09 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:35:09.479+0000 INFO  Discovered core member at 172.31.24.72:5000
Aug 25 04:35:09 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:35:09.496+0000 INFO  Discovered core member at 172.31.29.208:5000
Aug 25 04:35:38 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:35:38.954+0000 INFO  Waiting to hear from leader...
Aug 25 04:36:06 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:36:06.955+0000 INFO  Waiting to hear from leader...
Aug 25 04:36:34 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:36:34.956+0000 INFO  Waiting to hear from leader...
Aug 25 04:37:02 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:37:02.957+0000 INFO  Waiting to hear from leader...
Aug 25 04:37:30 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:37:30.959+0000 INFO  Waiting to hear from leader...
Aug 25 04:37:58 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:37:58.960+0000 INFO  Waiting to hear from leader...
Aug 25 04:38:26 ip-172-31-23-197 neo4j[4062]: 2019-08-25 04:38:26.961+0000 INFO  Waiting to hear from leader...

I don't know why the 3rd node is waiting to hear from the leader?

1 ACCEPTED SOLUTION

I guess the advertised address settings are missing. Amend to your neo4j.conf file:

dbms.connectors.default_advertised_address=<internal_ip_address_of_this_host>

Note: if your clients (e.g. neo4j browser) are sitting outside on the public interface you might add as well:

dbms.connector.bolt.advertised_address=<external_ip_of_this_host>
dbms.connector.http.advertised_address=<external_ip_of_this_host>
dbms.connector.https.advertised_address=<external_ip_of_this_host>

View solution in original post

2 REPLIES 2

I guess the advertised address settings are missing. Amend to your neo4j.conf file:

dbms.connectors.default_advertised_address=<internal_ip_address_of_this_host>

Note: if your clients (e.g. neo4j browser) are sitting outside on the public interface you might add as well:

dbms.connector.bolt.advertised_address=<external_ip_of_this_host>
dbms.connector.http.advertised_address=<external_ip_of_this_host>
dbms.connector.https.advertised_address=<external_ip_of_this_host>

Thanks! That solved it!

Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

On November 16 and 17 for 24 hours across all timezones, you’ll learn about best practices for beginners and experts alike.