Enterprise Cluster Cannot Run SHOW commands

Neo4j Graph Platform Installation
1

Our company is new to Neo4j and obtained an enterprise license a couple weeks ago.
I have setup a cluster across three AWS EC2 servers using the docker instance of neo4j.
The cluster members connect and there are no errors in the log. We can add data into the neo4j database and do queries. HOWEVER, when we try to run commands such as "SHOW SERVERS" it errors with the following:

Remote execution failed with code N/A and message 'Failed to obtain connection towards WRITE server. Known routing table is: Ttl 1737737482515, currentTime 1737737182517, routers , writers , readers , database 'system''

Here is the neo4j.conf file from one of the servers (only difference is IPs with what is on other servers):

dbms.cluster.discovery.v2.endpoints=10.40.46.68:6000,10.40.32.119:6000,10.40.40.38:6000
dbms.cluster.discovery.version=V2_ONLY
server.default_advertised_address=10.40.46.68
server.default_listen_address=0.0.0.0

There are no errors in the logs of the cluster members. Here is one of the logs:

Changed password for user 'neo4j'. IMPORTANT: this change will only take effect if performed before the database is started for the first time.
2025-01-24 17:33:11.984+0000 INFO Logging config in use: Embedded default config 'default-user-logs.xml'
2025-01-24 17:33:11.991+0000 INFO The license agreement was accepted with environment variable NEO4J_ACCEPT_LICENSE_AGREEMENT=yes when the Software was started.
2025-01-24 17:33:12.023+0000 WARN Settings server.discovery.listen_address and server.discovery.advertised_address will be ignored if V2_ONLY is set
2025-01-24 17:33:12.025+0000 INFO Starting...
2025-01-24 17:33:15.003+0000 INFO ======== Neo4j 5.26.0 ========
2025-01-24 17:33:15.027+0000 INFO This instance is ServerId{72260d34} (72260d34-e174-44ef-b366-ceeb2a7519b6)
2025-01-24 17:33:15.290+0000 INFO Resolved endpoints with LIST{endpoints:'[10.40.46.68:6000, 10.40.32.119:6000, 10.40.40.38:6000]'} to '[10.40.32.119:6000, 10.40.40.38:6000, 10.40.46.68:6000]'
2025-01-24 17:33:15.380+0000 INFO Resolved endpoints with LIST{endpoints:'[10.40.46.68:6000, 10.40.32.119:6000, 10.40.40.38:6000]'} to '[10.40.32.119:6000, 10.40.40.38:6000, 10.40.46.68:6000]'
2025-01-24 17:33:27.410+0000 INFO Connected to 10.40.32.119/10.40.32.119:7000 [Category[name=RAFT] version:3.0]
2025-01-24 17:33:28.668+0000 INFO Sending metrics to CSV file at /metrics
2025-01-24 17:33:28.705+0000 INFO Anonymous Usage Data is being sent to Neo4j, see Usage data report - Neo4j Documentation
2025-01-24 17:33:28.709+0000 INFO Bolt enabled on 0.0.0.0:7687.
2025-01-24 17:33:28.711+0000 INFO Bolt (Routing) enabled on 0.0.0.0:7688.
2025-01-24 17:33:29.478+0000 INFO HTTP enabled on 0.0.0.0:7474.
2025-01-24 17:33:29.480+0000 INFO Remote interface available at http://10.40.46.68:7474/
2025-01-24 17:33:29.483+0000 INFO id: E2135B7E72B504250DC8DD06DF246E0A189259C5406E18969C15E6264EE06AF5
2025-01-24 17:33:29.483+0000 INFO name: system
2025-01-24 17:33:29.484+0000 INFO creationDate: 2025-01-24T17:33:24.14Z
2025-01-24 17:33:29.484+0000 INFO Started.

Finally in case it helps identify the issue. Here is the result of running :sysinfo

Databases
Name Address Role Status Default Error
neo4j 10.40.40.38:7687 primary online true -
system 10.40.46.68:7687 primary online - -
system 10.40.32.119:7687 primary online - -
system 10.40.40.38:7687 primary online - -

Appreciate any help we can get. Thanks!

  • Sean Smith
2

The SHOW commands are system db commands. You may want to switch over to the system db prior to running them.

3

Thank you!! Now I really feel like a NOOB, but it wasn't clear that you have to connect to system first

4

It is a bit confusing, especially given the sequence of things that I think is contributing to that error.

  • If you're using the Neo4j Browser, connected with the neo4j:// protocol, it will try to execute your queries within write transactions, which will route to the leader.
  • If you're connected to the neo4j db (or a different user db) it will subsequently route to the leader of that db (which is probably not the system leader in addition)
  • The user db you are connected to cannot run system db commands, so it will chuck that over to the local system db on that server.
  • But that local system db is not the system leader, and so it can't handle the write transaction and fails.

You won't encounter this with a single instance, since the single instance handles all write transactions for all dbs. But with a cluster, db leadership is distributed, so this kind of thing can happen.

Best approach is to switch to the system db, that way when the transaction is executed, it routes directly to the system leader and runs it there without trouble.