# Neo4j Causal Cluster fails to form despite service showing ok **URL:** \ **Category:** Cluster\ **Tags:** cluster\ **Created:** [May 20, 2024, 8:27pm UTC](https://community.neo4j.com/t/neo4j-causal-cluster-fails-to-form-despite-service-showing-ok/68003 "2024-05-20T20:27:09Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![codrin.stefan](https://avatars.discourse-cdn.com/v4/letter/c/db5fbb/32.png) [@codrin.stefan](https://community.neo4j.com/u/codrin.stefan)\ **Post date:** [May 20, 2024, 8:27pm UTC](https://community.neo4j.com/t/neo4j-causal-cluster-fails-to-form-despite-service-showing-ok/68003/1 "2024-05-20T20:27:09Z")
Hello, I am trying to form a cluster with 2 primaries (I dont need fault tolerance) and later add some secondary servers as well, but to begin only with 2 primaries. I have the following config in neo4j.conf: server.directories.import=import server.bolt.enabled=true server.http.enabled=true server.https.enabled=false server.metrics.csv.rotation.compression=zip server.jvm.additional=-XX:+UseG1GC server.jvm.additional=-XX:-OmitStackTraceInFastThrow server.jvm.additional=-XX:+AlwaysPreTouch server.jvm.additional=-XX:+UnlockExperimentalVMOptions server.jvm.additional=-XX:+TrustFinalNonStaticFields server.jvm.additional=-XX:+DisableExplicitGC server.jvm.additional=-XX:-RestrictContended server.jvm.additional=-Djdk.nio.maxCachedBufferSize=1024 server.jvm.additional=-Dio.netty.tryReflectionSetAccessible=true server.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048 server.jvm.additional=-Djdk.tls.rejectClientInitiatedRenegotiation=true server.jvm.additional=-XX:FlightRecorderOptions=stackdepth=256 server.jvm.additional=-XX:+UnlockDiagnosticVMOptions server.jvm.additional=-XX:+DebugNonSafepoints server.jvm.additional=--add-opens=java.base/java.nio=ALL-UNNAMED server.jvm.additional=--add-opens=java.base/java.io=ALL-UNNAMED server.jvm.additional=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED server.jvm.additional=-Dlog4j2.disable.jmx=true server.windows\_service\_name=neo4j server.default\_listen\_address=172.16.31.120 server.default\_advertised\_address=172.16.31.120 dbms.cluster.discovery.endpoints=172.16.30.66:5000,172.16.31.120:5000 initial.dbms.default\_primaries\_count=2 server.discovery.advertised\_address=172.16.31.120:5000 server.cluster.advertised\_address=172.16.31.120:6000 server.bolt.listen\_address=0.0.0.0:7687 server.bolt.advertised\_address=172.16.31.120:7687 The service starts fine on both nodes, however, debug.log shows the following error message on loop. The VM running neo4j are ubuntu 20.04 LTS with no sudo apt upgrade performed. While setting up the same config on an ubuntu 18 LTS the cluster can form fine and I can connect. Firewall is not in discussion as it's not present (nor VM level (firewalld) nor network restrictions) 2024-05-20 20:21:52.243+0000 INFO [c.n.c.d.a.AkkaTopologyService] Restarting discovery system after probable network partition 2024-05-20 20:21:53.044+0000 INFO [c.n.c.d.a.AkkaTopologyService] System bootstrapper selection is removed; was: '6d1484eb-ad8a-4435-a1e1-e46d28a0d15a' 2024-05-20 20:21:53.045+0000 INFO [c.n.c.d.d.TopologyState] Server {6d1484eb} is removed with version 5.19.0 in state AVAILABLE with connectors [BOLT=172.16.30.66:7687,CLUSTER=172.16.30.66:6000,RAFT=172.16.30.66:7000], initial settings ['initial.server.allowed\_databases'='','initial.server.denied\_databases'='','initial.server.mode\_constraint'='PRIMARY','initial.server.tags'='','server.cluster.initial\_mode\_constraint'='PRIMARY'], component versions [cluster-discovery=4,community-topology-graph=1,dbms-runtime=17,fabric-database=1,security-privileges=18,security-users=4,topology-graph=3] 2024-05-20 20:21:53.045+0000 INFO [c.n.c.d.d.TopologyState] Database {00000000}/system on {6d1484eb} is leaving and removing raft mapping (RaftMemberId{6d1484eb}) 2024-05-20 20:21:53.046+0000 INFO [c.n.c.d.d.TopologyState] Database {00000000}/system on {6d1484eb} is removed 2024-05-20 20:21:53.046+0000 INFO [c.n.c.c.c.m.RaftTargetMembershipAggregator] [system/00000000] Discovery source update {RaftMemberId{63e7037b}=VOTER} 2024-05-20 20:21:53.046+0000 INFO [Raft] [system/00000000] [Event] Target membership changed [targetMembers={RaftMemberId{63e7037b}=VOTER}] 2024-05-20 20:21:53.104+0000 WARN [a.e.DummyClassForStringSources] Using serializer [com.neo4j.causalclustering.discovery.akka.marshal.UniqueAddressSerializer] for message [akka.cluster.UniqueAddress]. Note that this serializer is not implemented by Akka. It's not recommended to replace serializers for messages provided by Akka. 2024-05-20 20:21:53.107+0000 INFO [c.n.c.d.a.AkkaTopologyService] Publishing initial data into actor system PublishInitialData[registeredConnectors={RAFT=172.16.31.120:7000, CLUSTER=172.16.31.120:6000, BOLT=172.16.31.120:7687}, localDatabaseInfos={DatabaseId{00000000}=DatabaseInfo[databaseServer=DatabaseServer[databaseId=DatabaseId{00000000}, serverId=ServerId{63e7037b}], operatorState=STARTING, statusMessage=null, mode=RAFT, raftMemberId=RaftMemberId{63e7037b}, databaseAccess=READ\_WRITE]}, localLeaderInfos={DatabaseId{00000000}=LeaderInfo{term -1, not yet elected}}, localVersions=ComponentVersionVector[cluster-discovery=4,community-topology-graph=1,dbms-runtime=17,fabric-database=1,security-privileges=18,security-users=4,topology-graph=3], systemBootstrapper=Optional.empty, serializerVersion=Optional.empty] 2024-05-20 20:21:53.108+0000 INFO [c.n.c.d.a.AkkaTopologyService] Successfully restarted discovery system 2024-05-20 20:21:55.186+0000 INFO [c.n.c.d.d.TopologyState] Server {6d1484eb} is added with version 5.19.0 in state AVAILABLE with connectors [BOLT=172.16.30.66:7687,CLUSTER=172.16.30.66:6000,RAFT=172.16.30.66:7000], initial settings ['initial.server.allowed\_databases'='','initial.server.denied\_databases'='','initial.server.mode\_constraint'='PRIMARY','initial.server.tags'='','server.cluster.initial\_mode\_constraint'='PRIMARY'], component versions 2024-05-20 20:21:55.188+0000 INFO [c.n.c.d.d.TopologyState] Component versions for server {6d1484eb} changed as [added: fabric-database=1, added: cluster-discovery=4, added: topology-graph=3, added: security-privileges=18, added: dbms-runtime=17, added: community-topology-graph=1, added: security-users=4] 2024-05-20 20:21:55.189+0000 INFO [c.n.c.d.a.AkkaTopologyService] System bootstrapper selected; is: '6d1484eb-ad8a-4435-a1e1-e46d28a0d15a' 2024-05-20 20:21:55.190+0000 INFO [c.n.c.d.d.TopologyState] Database {00000000}/system on {6d1484eb} is discoverable in mode RAFT and publishing RaftMemberId{6d1484eb} 2024-05-20 20:21:55.191+0000 INFO [c.n.c.c.c.m.RaftTargetMembershipAggregator] [system/00000000] Discovery source update {RaftMemberId{63e7037b}=VOTER, RaftMemberId{6d1484eb}=VOTER} 2024-05-20 20:21:55.191+0000 INFO [Raft] [system/00000000] [Event] Target membership changed [targetMembers={RaftMemberId{63e7037b}=VOTER, RaftMemberId{6d1484eb}=VOTER}] 2024-05-20 20:21:55.192+0000 INFO [c.n.c.p.CheckConnectivityTopologyListener] Checking connectivity to port RAFT for server 6d1484eb-ad8a-4435-a1e1-e46d28a0d15a, mode PRIMARY. Address 172.16.30.66:7000, result Success 2024-05-20 20:21:55.196+0000 INFO [c.n.c.p.CheckConnectivityTopologyListener] Checking connectivity to port CLUSTER for server 6d1484eb-ad8a-4435-a1e1-e46d28a0d15a, mode PRIMARY. Address 172.16.30.66:6000, result Success 2024-05-20 20:22:23.786+0000 INFO [c.n.c.d.d.TopologyState] Database {00000000}/system on {6d1484eb} is leaving and removing raft mapping (RaftMemberId{6d1484eb}) 2024-05-20 20:22:23.787+0000 INFO [c.n.c.d.d.TopologyState] Database {00000000}/system on {6d1484eb} is removed 2024-05-20 20:22:23.787+0000 INFO [c.n.c.c.c.m.RaftTargetMembershipAggregator] [system/00000000] Discovery source update {RaftMemberId{63e7037b}=VOTER} 2024-05-20 20:22:23.788+0000 INFO [Raft] [system/00000000] [Event] Target membership changed [targetMembers={RaftMemberId{63e7037b}=VOTER}] 2024-05-20 20:22:23.789+0000 INFO [c.n.c.d.d.TopologyState] Server {6d1484eb} is removed with version 5.19.0 in state AVAILABLE with connectors [BOLT=172.16.30.66:7687,CLUSTER=172.16.30.66:6000,RAFT=172.16.30.66:7000], initial settings ['initial.server.allowed\_databases'='','initial.server.denied\_databases'='','initial.server.mode\_constraint'='PRIMARY','initial.server.tags'='','server.cluster.initial\_mode\_constraint'='PRIMARY'], component versions [cluster-discovery=4,community-topology-graph=1,dbms-runtime=17,fabric-database=1,security-privileges=18,security-users=4,topology-graph=3] 2024-05-20 20:22:25.346+0000 INFO [c.n.c.d.d.TopologyState] Database {00000000}/system on {6d1484eb} is discoverable in mode RAFT and publishing RaftMemberId{6d1484eb} 2024-05-20 20:22:25.347+0000 INFO [c.n.c.c.c.m.RaftTargetMembershipAggregator] [system/00000000] Discovery source update {RaftMemberId{63e7037b}=VOTER, RaftMemberId{6d1484eb}=VOTER} 2024-05-20 20:22:25.347+0000 INFO [Raft] [system/00000000] [Event] Target membership changed [targetMembers={RaftMemberId{63e7037b}=VOTER, RaftMemberId{6d1484eb}=VOTER}] 2024-05-20 20:22:25.349+0000 INFO [c.n.c.d.d.TopologyState] Server {6d1484eb} is added with version 5.19.0 in state AVAILABLE with connectors [BOLT=172.16.30.66:7687,CLUSTER=172.16.30.66:6000,RAFT=172.16.30.66:7000], initial settings ['initial.server.allowed\_databases'='','initial.server.denied\_databases'='','initial.server.mode\_constraint'='PRIMARY','initial.server.tags'='','server.cluster.initial\_mode\_constraint'='PRIMARY'], component versions [cluster-discovery=4,community-topology-graph=1,dbms-runtime=17,fabric-database=1,security-privileges=18,security-users=4,topology-graph=3] 2024-05-20 20:22:25.354+0000 INFO [c.n.c.p.CheckConnectivityTopologyListener] Checking connectivity to port RAFT for server 6d1484eb-ad8a-4435-a1e1-e46d28a0d15a, mode PRIMARY. Address 172.16.30.66:7000, result Success 2024-05-20 20:22:25.358+0000 INFO [c.n.c.p.CheckConnectivityTopologyListener] Checking connectivity to port CLUSTER for server 6d1484eb-ad8a-4435-a1e1-e46d28a0d15a, mode PRIMARY. Address 172.16.30.66:6000, result Success 2024-05-20 20:22:25.765+0000 INFO [c.n.c.p.h.InboundChannelLogger] [raft-server] Initializing server channel [id: 0x34e86c70, L:/172.16.31.120:7000 - R:/172.16.30.66:39690] 2024-05-20 20:22:25.767+0000 INFO [c.n.c.p.h.InboundChannelLogger] [catchup-server] Initializing server channel [id: 0x38e8c2b4, L:/172.16.31.120:6000 - R:/172.16.30.66:39798] 2024-05-20 20:22:25.767+0000 INFO [c.n.c.p.h.InboundChannelLogger] [catchup-server] Initializing server channel [id: 0x88467ede, L:/172.16.31.120:6000 - R:/172.16.30.66:39800] 2024-05-20 20:22:25.768+0000 INFO [c.n.c.p.h.InboundChannelLogger] [catchup-server] Initializing server channel [id: 0xf9587b53, L:/172.16.31.120:6000 - R:/172.16.30.66:39802] 2024-05-20 20:22:45.126+0000 INFO [c.n.c.d.a.AkkaTopologyService] Restarting discovery system after probable network partition