Illegal addition ID state for range

Hi guys,

Technical description :

  • neo4j Community Edition version 5.20 and 5.19
  • neo4j java driver version 5.20
  • our java service, read data from kafka and merge/create index/node/relationship
  • exemple of our cypher:
UNWIND $batch as row
MERGE (entity:kif { idRef: row.id })
    ON CREATE
    SET
        entity.createdAt = localdatetime(row.createdAt),
        entity.updatedAt = localdatetime()

WITH row, entity,hash
    
// remove old labels. They will be replaced by the new
CALL apoc.create.removeLabels(entity, labels(entity)) YIELD node as removedLabels
CALL apoc.create.addLabels(entity, row.labels) YIELD node as addedLabels

// remove and replace with new properties and labels
SET entity = {
    idRef: row.id,
    type: row.type,
    raw: row.raw,
    topic: row.topic,
    app: row.app,
    version: row.version,
    domain: row.domain,
    createdAt : entity.createdAt,
    lastSyncAt: localdatetime(row.lastSyncAt),
    updatedAt: localdatetime()
}

// add dependencies from
CALL apoc.do.when(row.dependencies is not null and size(row.dependencies) > 0, "
    UNWIND row.dependencies as dependencies

    call {
        WITH row, entity, dependencies

        // add dependencies
        MERGE (entityDependency:kif { idRef: dependencies.id })
            ON CREATE SET
                entityDependency:unknown:kif,
                entityDependency.createdAt = localdatetime(row.createdAt),
                entityDependency.lastSyncAt = localdatetime(row.lastSyncAt),
                entityDependency.updatedAt = localdatetime()

        WITH row, entity, dependencies, entityDependency

        // add relation between node and dependency
        CALL apoc.merge.relationship(
            entity,
            dependencies.type,
            null,
            {createdAt:localdatetime(row.createdAt),lastSyncAt:localdatetime(row.lastSyncAt), updatedAt:localdatetime(), raw: dependencies.raw},
            entityDependency,
            {lastSyncAt:localdatetime(row.lastSyncAt),updatedAt:localdatetime(), raw: dependencies.raw}
        ) YIELD rel
        
        RETURN 0 as ignored

    } return *
", "", {row: row, entity: entity}) yield value as fromValue

....

Error Description :

After millions of ingestion, we encountered, randomly, this error

024-05-31 01:14:10.638+0000 ERROR [o.n.b.f.StateMachineImpl] Client triggered an unexpected error [Neo.DatabaseError.Transaction.TransactionStartFailed]: The database has encountered a critical er

ror, and needs to be restarted. Please see database logs for more details., reference 3d261b51-8666-4ceb-b28f-02a40a944955.

2024-05-31 01:14:10.638+0000 WARN [o.n.r.QueryRouter] Could not eagerly create kernel transaction due to: org.neo4j.graphdb.TransactionFailureException: The database has encountered a critical err

or, and needs to be restarted. Please see database logs for more details.

2024-05-31 01:14:10.638+0000 WARN [o.n.r.QueryRouter] Could not eagerly create kernel transaction due to: org.neo4j.graphdb.TransactionFailureException: The database has encountered a critical err

or, and needs to be restarted. Please see database logs for more details.

2024-05-31 01:14:10.631+0000 ERROR [o.n.b.f.StateMachineImpl] Client triggered an unexpected error [Neo.DatabaseError.Transaction.TransactionStartFailed]: The database has encountered a critical er

ror, and needs to be restarted. Please see database logs for more details., reference 66f56a69-4b42-4762-84a9-946602c7a226.

2024-05-31 01:14:10.631+0000 WARN [o.n.r.QueryRouter] Could not eagerly create kernel transaction due to: org.neo4j.graphdb.TransactionFailureException: The database has encountered a critical err

or, and needs to be restarted. Please see database logs for more details.

2024-05-31 01:14:10.630+0000 ERROR [o.n.b.f.StateMachineImpl] Client triggered an unexpected error [Neo.DatabaseError.Transaction.TransactionStartFailed]: The database has encountered a critical er

ror, and needs to be restarted. Please see database logs for more details., reference 9feea19e-cb2a-43b0-949b-02b3b0a2d9df.

org.neo4j.bolt.protocol.common.fsm.error.TransactionStateTransitionException: The database has encountered a critical error, and needs to be restarted. Please see database logs for more details.

Caused by: org.neo4j.internal.id.indexed.IllegalIdTransitionException: Illegal addition ID state for range: 33554431 (IDs 4294967168-4294967231) transition
into: 00000000 00000000 00000000 00000000 00000000 00011111 11111111 11111111
from: 00000000 00000000 00000000 00000000 00000000 00011111 11110000 00000000
        at org.neo4j.internal.id.indexed.IdRange.verifyMerge(IdRange.java:231) ~[neo4j-id-generator-5.20.0.jar:5.20.0]
        at org.neo4j.internal.id.indexed.IdRange.mergeFrom(IdRange.java:158) ~[neo4j-id-generator-5.20.0.jar:5.20.0]
....
        at org.neo4j.bolt.protocol.common.fsm.transition.transaction.streaming.StreamingStateTransition.process(StreamingStateTransition.java:62) ~[neo4j-bolt-5.20.0.jar:5.20.0]
        at org.neo4j.bolt.protocol.common.fsm.transition.transaction.streaming.AutocommitStateTransition.process(AutocommitStateTransition.java:45) ~[neo4j-bolt-5.20.0.jar:5.20.0]
        at org.neo4j.bolt.protocol.common.fsm.transition.transaction.streaming.StreamingStateTransition.process(StreamingStateTransition.java:55) ~[neo4j-bolt-5.20.0.jar:5.20.0]
        at org.neo4j.bolt.protocol.common.fsm.transition.transaction.streaming.StreamingStateTransition.process(StreamingStateTransition.java:34) ~[neo4j-bolt-5.20.0.jar:5.20.0]
        at org.neo4j.bolt.protocol.common.fsm.transition.transaction.TransactionalStateTransition.process(TransactionalStateTransition.java:42) ~[neo4j-bolt-5.20.0.jar:5.20.0]
        ... 11 more
        Suppressed: java.lang.Exception: Illegal addition ID state for range: 33554431 (IDs 4294967168-4294967231) transition
into: 00000000 00000000 00000000 00000000 00000000 00011111 11111111 11111111
from: 00000000 00000000 00000000 00000000 00000000 00011111 11110000 00000000 | GB+Tree[file:/app/neo4j/data/databases/neo4j/neostore.propertystore.db.strings.id, layout:IdRangeLayout[version:1.2,
identifier:3735929182, fixedSize:true], generation:13385/13386]

Thanks for your help.

Any idea guys ??

Aldready ran it, consistency check is OK.