cancel
Showing results for 
Search instead for 
Did you mean: 

Error Failed to start Neo4j after power outage

atesterguy
Node Clone

After our ESXI server shutdown due to a power outage (a UPS will be added VERY soon after this incident) my neo4j server wont start again.

Here is the error message I receive when I try to start the server.
Any help would be greatly appreciated.

2020-05-07 16:13:37.625+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@1f67761b' was successfully initialized, but failed to start. Please see the attached cause exception "Mismatching store id. Store StoreId: StoreId{creationTime=1587666667320, randomId=-1079962435470005166, storeVersion=3471765337752883975, upgradeTime=1587666667320, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1586700350222, randomId=-2629912999185239023, storeVersion=3471765337752883975, upgradeTime=1586700350222, upgradeTxId=1}". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@1f67761b' was successfully initialized, but failed to start. Please see the attached cause exception "Mismatching store id. Store StoreId: StoreId{creationTime=1587666667320, randomId=-1079962435470005166, storeVersion=3471765337752883975, upgradeTime=1587666667320, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1586700350222, randomId=-2629912999185239023, storeVersion=3471765337752883975, upgradeTime=1586700350222, upgradeTxId=1}".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@1f67761b' was successfully initialized, but failed to start. Please see the attached cause exception "Mismatching store id. Store StoreId: StoreId{creationTime=1587666667320, randomId=-1079962435470005166, storeVersion=3471765337752883975, upgradeTime=1587666667320, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1586700350222, randomId=-2629912999185239023, storeVersion=3471765337752883975, upgradeTime=1586700350222, upgradeTxId=1}".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:164)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:114)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:89)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:35)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@1f67761b' was successfully initialized, but failed to start. Please see the attached cause exception "Mismatching store id. Store StoreId: StoreId{creationTime=1587666667320, randomId=-1079962435470005166, storeVersion=3471765337752883975, upgradeTime=1587666667320, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1586700350222, randomId=-2629912999185239023, storeVersion=3471765337752883975, upgradeTime=1586700350222, upgradeTxId=1}".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:465)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:157)
... 3 more
Caused by: java.lang.RuntimeException: Error starting database server at /var/lib/neo4j/data/databases
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:167)
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:145)
at org.neo4j.server.database.CommunityGraphFactory.newDatabaseManagementService(CommunityGraphFactory.java:36)
at org.neo4j.server.database.LifecycleManagingDatabaseService.start(LifecycleManagingDatabaseService.java:88)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.dbms.database.DefaultSystemGraphInitializer@545e57d7' was successfully initialized, but failed to start. Please see the attached cause exception "Mismatching store id. Store StoreId: StoreId{creationTime=1587666667320, randomId=-1079962435470005166, storeVersion=3471765337752883975, upgradeTime=1587666667320, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1586700350222, randomId=-2629912999185239023, storeVersion=3471765337752883975, upgradeTime=1586700350222, upgradeTxId=1}".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:465)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:158)
... 9 more
Caused by: org.neo4j.graphdb.DatabaseShutdownException: This database is shutdown.
at org.neo4j.kernel.availability.DatabaseAvailabilityGuard.assertDatabaseAvailable(DatabaseAvailabilityGuard.java:172)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginKernelTransaction(GraphDatabaseFacade.java:185)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransactionInternal(GraphDatabaseFacade.java:165)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:119)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:113)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:101)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTx(GraphDatabaseFacade.java:96)
at org.neo4j.dbms.database.DefaultSystemGraphInitializer.isSystemGraphEmpty(DefaultSystemGraphInitializer.java:100)
at org.neo4j.dbms.database.DefaultSystemGraphInitializer.initializeSystemGraph(DefaultSystemGraphInitializer.java:76)
at org.neo4j.dbms.database.DefaultSystemGraphInitializer.initializeSystemGraph(DefaultSystemGraphInitializer.java:69)
at org.neo4j.dbms.database.SystemGraphInitializer.start(SystemGraphInitializer.java:30)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
... 11 more
Caused by: java.lang.RuntimeException: Mismatching store id. Store StoreId: StoreId{creationTime=1587666667320, randomId=-1079962435470005166, storeVersion=3471765337752883975, upgradeTime=1587666667320, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1586700350222, randomId=-2629912999185239023, storeVersion=3471765337752883975, upgradeTime=1586700350222, upgradeTxId=1}
at org.neo4j.kernel.recovery.Recovery.validateStoreId(Recovery.java:386)
at org.neo4j.kernel.database.Database.start(Database.java:377)
at org.neo4j.dbms.database.AbstractDatabaseManager.startDatabase(AbstractDatabaseManager.java:187)
at org.neo4j.dbms.database.DefaultDatabaseManager.startDatabase(DefaultDatabaseManager.java:132)
at org.neo4j.dbms.database.DefaultDatabaseManager.startDatabase(DefaultDatabaseManager.java:36)
at org.neo4j.dbms.database.AbstractDatabaseManager.forEachDatabase(AbstractDatabaseManager.java:165)
at org.neo4j.dbms.database.AbstractDatabaseManager.startAllDatabases(AbstractDatabaseManager.java:112)
at org.neo4j.dbms.database.AbstractDatabaseManager.start(AbstractDatabaseManager.java:106)

1 ACCEPTED SOLUTION

Given you have a backup copy, try to recursively remove the schema subfolder. Don't know for sure but this might force a full index rebuild.

View solution in original post

4 REPLIES 4

First take a backup copy of your graph.db folder into a safe location.

Remove the transaction log files (in 3.x you'll find them in the graph.db folder) and try to start the db up.

Which neo4j version are you using?

atesterguy
Node Clone

Hi im using version 4.0.4
And it seems like this idea did not work.
I added the dbms.recovery.fail_on_missing_files=false so it will create a new file but now the error says:

ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@343fddd9' was successfully initialized, but failed to start. Please see the attached cause exception "Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@343fddd9' was successfully initialized, but failed to start. Please see the attached cause exception "Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@343fddd9' was successfully initialized, but failed to start. Please see the attached cause exception "Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]".
        at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:164)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:114)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:89)
        at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:35)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService@343fddd9' was successfully initialized, but failed to start. Please see the attached cause exception "Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]".
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:465)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:157)
        ... 3 more
Caused by: java.lang.RuntimeException: Error starting database server at /var/lib/neo4j/data/databases
        at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:167)
        at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:145)
        at org.neo4j.server.database.CommunityGraphFactory.newDatabaseManagementService(CommunityGraphFactory.java:36)
        at org.neo4j.server.database.LifecycleManagingDatabaseService.start(LifecycleManagingDatabaseService.java:88)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        ... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.dbms.database.DefaultSystemGraphInitializer@545e57d7' was successfully initialized, but failed to start. Please see the attached cause exception "Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]".
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:465)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
        at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:158)
        ... 9 more
Caused by: org.neo4j.graphdb.DatabaseShutdownException: This database is shutdown.
        at org.neo4j.kernel.availability.DatabaseAvailabilityGuard.assertDatabaseAvailable(DatabaseAvailabilityGuard.java:172)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginKernelTransaction(GraphDatabaseFacade.java:185)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransactionInternal(GraphDatabaseFacade.java:165)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:119)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:113)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:101)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTx(GraphDatabaseFacade.java:96)
        at org.neo4j.dbms.database.DefaultSystemGraphInitializer.isSystemGraphEmpty(DefaultSystemGraphInitializer.java:100)
        at org.neo4j.dbms.database.DefaultSystemGraphInitializer.initializeSystemGraph(DefaultSystemGraphInitializer.java:76)
        at org.neo4j.dbms.database.DefaultSystemGraphInitializer.initializeSystemGraph(DefaultSystemGraphInitializer.java:69)
        at org.neo4j.dbms.database.SystemGraphInitializer.start(SystemGraphInitializer.java:30)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        ... 11 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.api.index.IndexingService@732f6050' failed to initialize. Please see the attached cause exception "Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]".
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:426)
        at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:66)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:102)
        at org.neo4j.kernel.recovery.TransactionLogsRecovery.start(TransactionLogsRecovery.java:192)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
        at org.neo4j.kernel.recovery.Recovery.performRecovery(Recovery.java:364)
        at org.neo4j.kernel.database.Database.start(Database.java:380)
        at org.neo4j.dbms.database.AbstractDatabaseManager.startDatabase(AbstractDatabaseManager.java:187)
        at org.neo4j.dbms.database.DefaultDatabaseManager.startDatabase(DefaultDatabaseManager.java:132)
        at org.neo4j.dbms.database.DefaultDatabaseManager.startDatabase(DefaultDatabaseManager.java:36)
        at org.neo4j.dbms.database.AbstractDatabaseManager.forEachDatabase(AbstractDatabaseManager.java:165)
        at org.neo4j.dbms.database.AbstractDatabaseManager.startAllDatabases(AbstractDatabaseManager.java:112)
        at org.neo4j.dbms.database.AbstractDatabaseManager.start(AbstractDatabaseManager.java:106)
        ... 12 more
Caused by: java.lang.IllegalStateException: Index clean jobs can't be added after collector start. | GB+Tree[file:/var/lib/neo4j/data/databases/system/schema/index/native-btree-1.0/1/index-1, layout:GenericLayout[version:0.5, identifier:5643935348791705601, fixedSize:false], generation:6/8]
        at org.neo4j.util.Preconditions.checkState(Preconditions.java:173)
        at org.neo4j.index.internal.gbptree.GroupingRecoveryCleanupWorkCollector.add(GroupingRecoveryCleanupWorkCollector.java:60)
        at org.neo4j.index.internal.gbptree.GBPTree.createCleanupJob(GBPTree.java:1336)
        at org.neo4j.index.internal.gbptree.GBPTree.<init>(GBPTree.java:541)
        at org.neo4j.kernel.impl.index.schema.NativeIndex.instantiateTree(NativeIndex.java:69)
        at org.neo4j.kernel.impl.index.schema.GenericNativeIndexAccessor.<init>(GenericNativeIndexAccessor.java:60)
        at org.neo4j.kernel.impl.index.schema.GenericNativeIndexProvider.newIndexAccessor(GenericNativeIndexProvider.java:185)
        at org.neo4j.kernel.impl.index.schema.GenericNativeIndexProvider.newIndexAccessor(GenericNativeIndexProvider.java:105)
        at org.neo4j.kernel.impl.index.schema.NativeIndexProvider.getOnlineAccessor(NativeIndexProvider.java:104)
        at org.neo4j.kernel.impl.index.schema.GenericNativeIndexProvider.getOnlineAccessor(GenericNativeIndexProvider.java:105)
        at org.neo4j.kernel.impl.api.index.IndexProxyCreator.onlineAccessorFromProvider(IndexProxyCreator.java:159)
        at org.neo4j.kernel.impl.api.index.IndexProxyCreator.createOnlineIndexProxy(IndexProxyCreator.java:106)
        at org.neo4j.kernel.impl.api.index.IndexingService.lambda$init$1(IndexingService.java:259)
        at org.neo4j.kernel.impl.api.index.IndexMapReference.modify(IndexMapReference.java:53)
        at org.neo4j.kernel.impl.api.index.IndexingService.init(IndexingService.java:241)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:405)

Given you have a backup copy, try to recursively remove the schema subfolder. Don't know for sure but this might force a full index rebuild.

Thank you very much !
I have deleted the index in the system folder and it seemed to have fixed the issue.

Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

On November 16 and 17 for 24 hours across all timezones, you’ll learn about best practices for beginners and experts alike.