I'm running Neo4j 5.26.1 Community Edition in AWS ECS (Fargate) behind a Network Load Balancer. While the container starts successfully and health checks pass, I'm unable to connect from EC2 instances within the same VPC.
Setup Summary
- Neo4j 5.26.1 Community Edition running in ECS Fargate
- Network Load Balancer (internal) for stable endpoint
- EFS mount for persistence
- All resources in same VPC as EC2 machines
- Security groups configured to allow traffic between EC2 machines and Neo4j
Issue description
The ECS and NLB health checks are both passing. ECS container logs show that the DB has started and is running. However, every time I try to connect from my EC2 machine via a python client, I am faced with the same error.
I'm quite confident that this is an issue with the neo4j container's network connectors. I have reviewed the configuration guide in detail and read every relevant thread I could find, and still am stuck.
Client code
from neo4j import GraphDatabase
driver = GraphDatabase.driver(
"bolt://neo4j-lb-name.elb.us-east-1.amazonaws.com:7687",
auth=("neo4j", "password-placeholder")
)
with driver.session() as session:
result = session.run("RETURN 1 as num")
print(result.single()["num"])
driver.close()
error message
ServiceUnavailable: Couldn't connect to neo4j-lb-name.elb.us-east-1.amazonaws.com:7687 (resolved to ()):
Failed to establish connection to ResolvedIPv4Address(('10.XX.XX.241', 7687)) (reason [Errno 111] Connection refused)
Failed to establish connection to ResolvedIPv4Address(('10.XX.XX.206', 7687)) (reason [Errno 111] Connection refused)
Failed to establish connection to ResolvedIPv4Address(('10.XX.XXX.81', 7687)) (reason [Errno 111] Connection refused)
Failed to establish connection to ResolvedIPv4Address(('10.XX.XXX.223', 7687)) (reason [Errno 111] Connection refused)
Failed to establish connection to ResolvedIPv4Address(('10.XX.XXX.27', 7687)) (reason [Errno 111] Connection refused)
Failed to establish connection to ResolvedIPv4Address(('10.XX.XXX.143', 7687)) (reason [Errno 111] Connection refused)
Container logs
2025-01-31 10:17:36.166+0000 INFO Logging config in use: File '/var/lib/neo4j/conf/user-logs.xml'
2025-01-31 10:17:36.267+0000 INFO Starting...
2025-01-31 10:17:40.256+0000 INFO This instance is ServerId{d9d40645} (d9d40645-0ba0-4ce0-832b-18e16582700c)
2025-01-31 10:17:44.653+0000 INFO ======== Neo4j 5.26.1 ========
2025-01-31 10:17:52.100+0000 INFO Anonymous Usage Data is being sent to Neo4j, see https://neo4j.com/docs/usage-data/
2025-01-31 10:17:52.177+0000 INFO Bolt enabled on 0.0.0.0:7687.
2025-01-31 10:17:54.867+0000 INFO HTTP enabled on 0.0.0.0:7474.
2025-01-31 10:17:54.872+0000 INFO Remote interface available at http://neo4j-lb-name.elb.us-east-1.amazonaws.com:7474/
2025-01-31 10:17:54.878+0000 INFO id: F753CB1F0807B80B9EDD3E4D665E509C696B688DB3E31DC5F22702D61AFE1D57
2025-01-31 10:17:54.879+0000 INFO name: system
2025-01-31 10:17:54.879+0000 INFO creationDate: 2025-01-30T12:32:17.233Z
2025-01-31 10:17:54.882+0000 INFO Started.
Terraform Configuration
All resources are managed via terraform. Here is the configuration:
NLB
resource "aws_lb" "neo4j" {
name = "neo4j"
internal = true
load_balancer_type = "network"
subnets = ["subnet-xxxxx1", "subnet-xxxxx2"] # Multiple AZ subnets
enable_cross_zone_load_balancing = true
enable_deletion_protection = false
tags = {
Name = "neo4j"
}
}
resource "aws_lb_listener" "bolt" {
load_balancer_arn = aws_lb.neo4j.arn
port = "7687"
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.bolt.arn
}
}
resource "aws_lb_target_group" "bolt" {
name = "neo4j-bolt"
port = 7687
protocol = "TCP"
vpc_id = "vpc-xxxxx"
target_type = "ip"
health_check {
enabled = true
healthy_threshold = 2
unhealthy_threshold = 2
interval = 30
protocol = "HTTP"
port = 7474
path = "/"
}
deregistration_delay = 60
}
resource "aws_lb_listener" "http" {
load_balancer_arn = aws_lb.neo4j.arn
port = "7474"
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.http.arn
}
}
resource "aws_lb_target_group" "http" {
name = "neo4j-http"
port = 7474
protocol = "TCP"
vpc_id = "vpc-xxxxx"
target_type = "ip"
health_check {
enabled = true
healthy_threshold = 2
unhealthy_threshold = 2
interval = 30
protocol = "HTTP"
port = 7474
path = "/"
}
deregistration_delay = 60
}
ECS
resource "aws_ecs_task_definition" "neo4j" {
family = "neo4j"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
cpu = "1024"
memory = "4096"
execution_role_arn = "arn:aws:iam::xxxxx:role/ecsTaskExecutionRole"
task_role_arn = aws_iam_role.ecs_task_role.arn
container_definitions = jsonencode([
{
name = "neo4j"
image = "neo4j:5.26.1"
portMappings = [
{
containerPort = 7474
hostPort = 7474
protocol = "tcp"
},
{
containerPort = 7687
hostPort = 7687
protocol = "tcp"
}
]
healthCheck = {
command = ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:7474 || exit 1"]
interval = 30
timeout = 5
retries = 3
startPeriod = 60
}
environment = [
{
name = "NEO4J_AUTH"
value = "neo4j/password-placeholder"
},
{
name = "NEO4J_server_default__listen__address"
value = "0.0.0.0"
},
{
name = "NEO4J_server_default__advertised__address"
value = "nlb-dns-name-placeholder"
},
{
name = "NEO4J_dbms_logs_http_enabled"
value = "true"
}
]
mountPoints = [
{
sourceVolume = "neo4j-data"
containerPath = "/data"
readOnly = false
}
]
logConfiguration = {
logDriver = "awslogs"
options = {
"awslogs-group" = "/ecs/neo4j"
"awslogs-region" = "us-east-1"
"awslogs-stream-prefix" = "ecs"
"awslogs-create-group" = "true"
}
}
stopTimeout = 120
}
])
volume {
name = "neo4j-data"
efs_volume_configuration {
file_system_id = "fs-xxxxx"
transit_encryption = "ENABLED"
authorization_config {
access_point_id = "fsap-xxxxx"
iam = "ENABLED"
}
}
}
}
Security Groups
resource "aws_security_group" "ecs" {
name = "neo4j-ecs"
description = "Security group for Neo4j ECS service"
vpc_id = "vpc-xxxxx"
ingress {
description = "Allow HTTP traffic from EC2 instances clusters"
from_port = 7474
to_port = 7474
protocol = "tcp"
security_groups = ["sg-xxxxx1", "sg-xxxxx2"] # EC2 instances security groups
}
ingress {
description = "Allow Bolt traffic from EC2 instances clusters"
from_port = 7687
to_port = 7687
protocol = "tcp"
security_groups = ["sg-xxxxx1", "sg-xxxxx2"] # EC2 instances security groups
}
ingress {
description = "Allow traffic from NLB"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["10.0.0.0/16"] # VPC CIDR
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}