Upgrading from 3.3.3 to 3.3.9

upgrade

(Senica) #1

Hi! I'm trying to upgrade from 3.3.3 to 3.3.9 as part of my upgrade path to 3.4.10. I went through an offline upgrade, everything comes up fine, but the cluster never forms. It's using docker if that matters

The debug log looks like this:

bash-4.4# tail -f debug.log 
2018-12-01 02:56:04.114+0000 INFO [c.n.c.d.SslHazelcastCoreTopologyService] Core topology changed {added=[{memberId=MemberId{60130e83}, info=CoreServerInfo{raftServer=hosts-neo4j-3:7000, catchupServer=hosts-neo4j-3:6000, clientConnectorAddresses=bolt://hosts-neo4j-3:7687,http://localhost:7474,https://localhost:7473, groups=[]}}, {memberId=MemberId{8d0efc05}, info=CoreServerInfo{raftServer=hosts-neo4j-1:7000, catchupServer=hosts-neo4j-1:6000, clientConnectorAddresses=bolt://hosts-neo4j-1:7687,http://localhost:7474,https://localhost:7473, groups=[]}}], removed=[]}
2018-12-01 02:56:04.115+0000 INFO [o.n.c.c.c.m.RaftMembershipManager] Target membership: [MemberId{8d0efc05}, MemberId{60130e83}]
2018-12-01 02:56:04.145+0000 INFO [o.n.c.i.ClusterBinder] Published: ClusterId{uuid=11e0d321-90f3-43de-98f8-a3c93840a082}
2018-12-01 02:56:04.145+0000 INFO [o.n.c.i.ClusterBinder] Already bound to cluster: ClusterId{uuid=11e0d321-90f3-43de-98f8-a3c93840a082}
2018-12-01 03:00:59.461+0000 INFO [c.n.c.d.SslHazelcastCoreTopologyService] Core member removed MembershipEvent {member=Member [hosts-neo4j-1]:5000 - fdf18fe3-9d82-4dd8-b32f-65bc39f2cd30,type=removed}
2018-12-01 03:00:59.462+0000 INFO [c.n.c.d.SslHazelcastCoreTopologyService] Core topology changed {added=[], removed=[{memberId=MemberId{8d0efc05}, info=CoreServerInfo{raftServer=hosts-neo4j-1:7000, catchupServer=hosts-neo4j-1:6000, clientConnectorAddresses=bolt://hosts-neo4j-1:7687,http://localhost:7474,https://localhost:7473, groups=[]}}]}
2018-12-01 03:00:59.462+0000 INFO [o.n.c.c.c.m.RaftMembershipManager] Target membership: [MemberId{60130e83}]
2018-12-01 03:01:26.527+0000 INFO [c.n.c.d.SslHazelcastCoreTopologyService] Core member added MembershipEvent {member=Member [hosts-neo4j-1]:5000 - afa27e23-f347-4a42-9781-11354a5c5c70,type=added}
2018-12-01 03:01:26.533+0000 INFO [c.n.c.d.SslHazelcastCoreTopologyService] Core topology changed {added=[{memberId=MemberId{8d0efc05}, info=CoreServerInfo{raftServer=hosts-neo4j-1:7000, catchupServer=hosts-neo4j-1:6000, clientConnectorAddresses=bolt://hosts-neo4j-1:7687,http://localhost:7474,https://localhost:7473, groups=[]}}], removed=[]}
2018-12-01 03:01:26.538+0000 INFO [o.n.c.c.c.m.RaftMembershipManager] Target membership: [MemberId{8d0efc05}, MemberId{60130e83}]

and evenutally says something of disconnecting.

Any ideas why or how to debug what is going on?


(Senica) #2

I figured out the issue.

In the docs here: https://neo4j.com/docs/operations-manual/current/upgrade/causal-cluster/#cc-upgrade-offline

It says
On each of the other Core Servers: ... Perform neo4j-admin unbind on the instance.

I needed to run this on ALL the server