Neo4j Randomly Terminating Connection With Spring Service

ga.me.rvit78 · August 22, 2024, 4:39am

While the service that maintains Neo4j as it's database is running, the database unexpectedly terminates the connection to the service. I haven't really been able to replicate the error on my own as this termination occurs completely randomly but they generally follow after the service tries to write data into the db.

These are the logs from the debug.logs file at the time of error.

2024-08-21 06:52:33.116+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=8902, gcTime=9101, gcCount=2}
2024-08-21 06:52:42.755+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=9439, gcTime=9636, gcCount=2}
2024-08-21 06:52:52.610+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=9654, gcTime=9850, gcCount=2}
2024-08-21 06:53:02.251+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=9441, gcTime=9641, gcCount=2}
2024-08-21 06:53:13.151+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=10699, gcTime=10896, gcCount=2}
2024-08-21 06:54:55.081+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=27996, gcTime=101911, gcCount=22}
2024-08-21 06:53:51.152+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0xc1ae3c32, L:/172.31.19.69:7687 ! R:/172.31.59.78:40758]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-3465' ([id: 0xc1ae3c32, L:/172.31.19.69:7687 ! R:/172.31.59.78:40758]) as the client failed to authenticate within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:84) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-08-21 06:55:02.349+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=80801, gcTime=7268, gcCount=2}
2024-08-21 06:55:21.105+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0xda6cd895, L:/172.31.19.69:7687 ! R:/172.31.59.78:53230]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-3468' ([id: 0xda6cd895, L:/172.31.19.69:7687 ! R:/172.31.59.78:53230]) as the server failed to handle an authentication request within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:77) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-08-21 06:55:38.485+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.channel.StacklessClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-08-21 06:55:38.485+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.channel.StacklessClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-08-21 06:55:38.485+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.channel.StacklessClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-08-21 06:55:38.485+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.channel.StacklessClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]

This is the stack trace from the neo4j.logs file:

2024-08-21 07:09:52.144+0000 ERROR [bolt-3503] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:306) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-08-21 07:27:02.904+0000 ERROR Client triggered an unexpected error [Neo.DatabaseError.General.UnknownError]: org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler, reference c9b9e113-5815-4e3f-a26d-c64f5e125faf.
2024-08-21 07:27:02.904+0000 ERROR [bolt-3545] Terminating connection due to unexpected error
org.neo4j.bolt.runtime.BoltConnectionFatality: Failed to process a bolt message
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.handleFailure(AbstractStateMachine.java:293) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.StateMachineContextImpl.handleFailure(StateMachineContextImpl.java:92) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.v40.fsm.ConnectedState.processAuthentication(ConnectedState.java:106) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.v40.fsm.ConnectedState.process(ConnectedState.java:57) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.nextState(AbstractStateMachine.java:136) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:96) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: java.util.NoSuchElementException: org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler
	at io.netty.channel.DefaultChannelPipeline.getContextOrDie(DefaultChannelPipeline.java:1082) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:417) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at org.neo4j.bolt.protocol.common.connector.connection.listener.AuthenticationTimeoutConnectionListener.onAuthenticated(AuthenticationTimeoutConnectionListener.java:85) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AbstractConnection.lambda$authenticate$2(AbstractConnection.java:355) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807) ~[?:?]
	at org.neo4j.bolt.protocol.common.connector.connection.AbstractConnection.notifyListeners(AbstractConnection.java:173) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AbstractConnection.authenticate(AbstractConnection.java:355) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.v40.fsm.ConnectedState.processAuthentication(ConnectedState.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	... 12 more
2024-08-21 07:55:54.691+0000 ERROR [bolt-3606] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]

This is the stack trace in the logs of the Spring application when the application was trying to insert data into the db. However, after restarting the database and trying to insert the same data it didn't throw any error.

at   org.springframework.data.neo4j.core.transaction.Neo4jTransactionManager.doBegin(Neo4jTransactionManager.java:313)
at   org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)
at   org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)
at   org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:595)
at   org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:382)
at   org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at   org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at   org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
at   org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at   org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at   org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at   org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at   com.sun.proxy.$Proxy118.findById(Unknown Source)
at   com.ew.neo4jtransactionprocessor.repository.nodes.TransactionRepository.getOrCreateNode(TransactionRepository.java:45)
at   com.ew.neo4jtransactionprocessor.services.Neo4jTransactionWriterService.createOrUpdateTransactionNodeAndVersion(Neo4jTransactionWriterService.java:33)
at   com.ew.neo4jtransactionprocessor.services.Neo4jTransactionWriterService.createOrUpdateTransaction(Neo4jTransactionWriterService.java:107)
at   com.ew.neo4jtransactionprocessor.controllers.IFTTTController.nodeCreateOrUpdate(IFTTTController.java:89)
at   jdk.internal.reflect.GeneratedMethodAccessor143.invoke(Unknown Source)
at   java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at   java.base/java.lang.reflect.Method.invoke(Method.java:566)
at   org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at   org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at   org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at   org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at   org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at   org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at   org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071)
at   org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964)
at   org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at   org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at   javax.servlet.http.HttpServlet.service(HttpServlet.java:696)
at   org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at   javax.servlet.http.HttpServlet.service(HttpServlet.java:779)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.zalando.logbook.servlet.LogbookFilter.doFilter(LogbookFilter.java:71)
at   org.zalando.logbook.servlet.HttpFilter.doFilter(HttpFilter.java:31)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at   org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)
at   org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
at   org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at   org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at   org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.zalando.logbook.servlet.LogbookFilter.doFilter(LogbookFilter.java:71)
at   org.zalando.logbook.servlet.SecureLogbookFilter.doFilter(SecureLogbookFilter.java:32)
at   org.zalando.logbook.servlet.HttpFilter.doFilter(HttpFilter.java:31)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at   org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at   org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at   org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at   org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
at   org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at   org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at   org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
at   org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at   org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at   org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
at   org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
at   org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at   org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
at   org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
at   org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at   org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at   org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at   org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at   java.base/java.lang.Thread.run(Thread.java:829)
Caused by:   org.neo4j.driver.exceptions.ServiceUnavailableException: Unable to establish   connection in 30000ms
at   org.neo4j.driver.internal.util.Futures.blockingGet(Futures.java:111)
at   org.neo4j.driver.internal.InternalSession.beginTransaction(InternalSession.java:90)
at   org.springframework.data.neo4j.core.transaction.Neo4jTransactionManager.doBegin(Neo4jTransactionManager.java:304)
... 80 more
Suppressed:   org.neo4j.driver.internal.util.ErrorUtil$InternalExceptionCause
at   org.neo4j.driver.internal.async.inbound.ConnectTimeoutHandler.unableToConnectError(ConnectTimeoutHandler.java:49)
at   org.neo4j.driver.internal.async.inbound.ConnectTimeoutHandler.readTimedOut(ConnectTimeoutHandler.java:44)
at   org.neo4j.driver.internal.shaded.io.netty.handler.timeout.ReadTimeoutHandler.channelIdle(ReadTimeoutHandler.java:90)
at   org.neo4j.driver.internal.shaded.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503)
at   org.neo4j.driver.internal.shaded.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at   org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at   org.neo4j.driver.internal.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at   org.neo4j.driver.internal.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 more

Setup

Neo4j Version: 5.3.0 Community Edition
Operating System: Centos 7 on AWS EC2
API: Neo4j driver for Java

The database contains approximately 1.2 million nodes and 5.1 million relationships. If this information is of any help.

Expected behavior

Shouldn't crash randomly while insertion of data.

Actual behavior

Unable to create connections to the database, from both the driver/browser.

cdavide8 · August 22, 2024, 8:35am

Hello @ga.me.rvit78 .

I see the server used is Netty, typically devoted to reactive programming. I see in the logs of Spring that the driver is using the function blockingGet, which is a blocking call.
Could you please ensure that you're using the reactive facilities of the driver?

Also, since you're using Spring rather than plain Java, I suggest this reading: Neo4jClient :: Spring Data Neo4j where it is explained to prefer using the Neo4j-client when using Spring instead of the plain driver. The client indeed can manage application-level transactions, which may be related to your connection issues.

Let me know

ga.me.rvit78 · August 22, 2024, 9:03am

Thanks for the reply @cdavide8.

I'm using the built in methods of Spring JPA to retrieve and save the nodes, i.e. findByID and save and according to you, they might be the cause for the unexpected termination of the connection with the database? I don't quite understand why this might be the case. I'll really appreciate if you can explain it in some detail.

As for using the Neo4jClient , I am aware of it's capabilities, but I can't use it for the current project because the project is currently being used in production and would require a major rewrite.

cdavide8 · August 22, 2024, 9:36am

Ok, first things first.

Does this error happen only when doing that specific findById call, or for every call you perform in Spring toward neo4j? This is useful to understand what may be the issue.

Does this error happen also if you try to perform read-only queries from the browser?

If the DBMS is working, open the browser and run the command :sysinfo, in the same textbox you typically run queries. This command will output the status of the databases in the DBMS.

I wanted to also add an observation: it is advisable to use the Enterprise Edition of neo4j in production environments. This edition for instance allows query logging, as per documentation: Logging - Operations Manual
Moreover, you should keep a testing environment that mimics the production environment, so that in case of issues you can playground in the test neo4j instance by trying to replicate the issue and try different solutions.

Regarding reactive programming, I would not mix blocking calls with a reactive environment like Netty, although Spring Boot probably wraps blocking calls in virtual threads to simulate reactive calls. So this may not be the issue.

Let me know about my two questions above

ga.me.rvit78 · August 22, 2024, 10:58am

This error happens completely on random while trying to save the data(I have to generally lookup the data, make some updates and then save). And when I try to reinsert the data after restarting the database it shows no error.

No, this have never occurred while trying read-only queries in the web UI.

When this error does occur, we're unable to access the web UI and the browser shows This site can't be reached and the database has to be restarted manually from the command line.

I agree with your advice on keeping a testing environment but that is not a call for me to make as it's in the hands of the testing team.

cdavide8 · August 22, 2024, 1:50pm

Ok.

So as far as I've understood, the problem doesn't happen in a specific scenario, it doesn't happen in testing environment and it doesn't happen for every write operation, because it appears to be random. Am I right?

We can focus on the Detected VM stop-the-world pause message, right before the error. That warning takes place because the JVM heap memory is going to get filled up. Indeed the gc time increases as the log proceeds. Increase the jvm max heap size by setting the property server.memory.heap.max_size to a greater value, as per documentation: https://neo4j.com/docs/operations-manual/current/docker/configuration/

ga.me.rvit78 · August 23, 2024, 5:49am

Thanks for the suggestion, I checked the conf file and found out that these properties were actually commented out and the GC was trying to dynamically allocate the heap size might have been unable to keep up.

I've configured these values and will monitor if the problem occurs again or not.

cdavide8 · August 23, 2024, 8:53am

Very good @ga.me.rvit78 . Let me know

ga.me.rvit78 · September 4, 2024, 8:24am

@cdavide8 We implemented the changes you suggested and did not encounter the problem for the past week, however, it has begun to occur again.

cdavide8 · September 4, 2024, 9:39am

Hello @ga.me.rvit78, nice to hear you back.

Did you try to increase the heap size again?

Can you please report the new logs?

ga.me.rvit78 · September 4, 2024, 10:07am

Yeah, we increased the size of the heap up to 8GB and have kept the initial size as 512MB.
These are the logs:

neo4j.log

2024-08-26 04:46:44.187+0000 INFO  Started.
2024-09-03 03:57:23.001+0000 ERROR [bolt-2856] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 03:58:55.247+0000 ERROR [bolt-2863] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 04:20:50.967+0000 ERROR [bolt-2915] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 04:22:00.055+0000 ERROR [bolt-2912] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 04:28:00.552+0000 INFO  Neo4j Server shutdown initiated by request
2024-09-03 04:28:00.552+0000 INFO  Stopping...

debug.log

2024-09-03 03:19:21.492+0000 INFO  [o.n.k.i.t.l.c.CheckPointerImpl] [neo4j/bdbbf70d] Checkpoint triggered by "Scheduled checkpoint for every 15 minutes threshold" @ txId: 7341761 checkpoint completed in 44ms. Checkpoint flushed 269 pages (0% of total available pages), in 269 IOs. Checkpoint performed with IO limit: unlimited, paused in total 0 times( 0 millis).
2024-09-03 03:19:21.493+0000 INFO  [o.n.k.i.t.l.p.LogPruningImpl] [neo4j/bdbbf70d] No log version pruned. The strategy used was '2 days'. 
2024-09-03 03:43:31.036+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=258, gcTime=280, gcCount=1}
2024-09-03 03:47:29.832+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4684, gcTime=4878, gcCount=4}
2024-09-03 03:49:20.229+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4229, gcTime=4420, gcCount=4}
2024-09-03 03:49:42.157+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4708, gcTime=4823, gcCount=23}
2024-09-03 03:49:46.578+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=3805, gcTime=3997, gcCount=2}
2024-09-03 03:49:56.786+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4461, gcTime=4571, gcCount=20}
2024-09-03 03:50:04.575+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4134, gcTime=4192, gcCount=12}
2024-09-03 03:50:12.305+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=3876, gcTime=4031, gcCount=9}

.
.
.

2024-09-03 03:57:18.141+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4205, gcTime=34657, gcCount=17}
2024-09-03 03:57:18.142+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0xd9863860, L:/172.31.19.69:7687 ! R:/172.31.60.134:58186]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-2856' ([id: 0xd9863860, L:/172.31.19.69:7687 ! R:/172.31.60.134:58186]) as the server failed to handle an authentication request within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:77) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-09-03 03:57:27.437+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=39364, gcTime=9290, gcCount=5}
2024-09-03 03:57:27.438+0000 ERROR [o.n.b.p.c.c.c.AtomicSchedulingConnection] [bolt-2856] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 03:57:32.612+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4974, gcTime=5160, gcCount=2}
2024-09-03 03:58:55.246+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0xc9fd7eaa, L:/172.31.19.69:7687 ! R:/172.31.60.134:46382]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-2863' ([id: 0xc9fd7eaa, L:/172.31.19.69:7687 ! R:/172.31.60.134:46382]) as the server failed to handle an authentication request within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:77) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-09-03 03:58:55.248+0000 ERROR [o.n.b.p.c.c.c.AtomicSchedulingConnection] [bolt-2863] Terminating connection due to unexpected error
org.neo4j.bolt.protocol.error.streaming.BoltStreamingWriteException: Failed to finalize batch: Cannot write result response
	at org.neo4j.bolt.protocol.common.transaction.result.ResultHandler.onFinish(ResultHandler.java:116) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.after(AbstractStateMachine.java:126) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.fsm.AbstractStateMachine.process(AbstractStateMachine.java:99) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.lambda$submit$4(AtomicSchedulingConnection.java:113) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJob(AtomicSchedulingConnection.java:336) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.doExecuteJobs(AtomicSchedulingConnection.java:270) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at org.neo4j.bolt.protocol.common.connector.connection.AtomicSchedulingConnection.executeJobs(AtomicSchedulingConnection.java:212) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 03:58:30.425+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0x16ce767c, L:/172.31.19.69:7687 ! R:/172.31.60.134:36174]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-2857' ([id: 0x16ce767c, L:/172.31.19.69:7687 ! R:/172.31.60.134:36174]) as the server failed to handle an authentication request within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:77) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:406) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-09-03 03:59:09.980+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4547, gcTime=97284, gcCount=49}
2024-09-03 03:59:09.983+0000 WARN  [o.n.b.p.c.t.r.ResultHandler] Client /172.31.60.134:48278 disconnected while query was running. Session has been cleaned up. This can be caused by temporary network problems, but if you see this often, ensure your applications are properly waiting for operations to complete before exiting.
2024-09-03 03:59:09.983+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0xf1133abb, L:/172.31.19.69:7687 ! R:/172.31.60.134:51708]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-2860' ([id: 0xf1133abb, L:/172.31.19.69:7687 ! R:/172.31.60.134:51708]) as the server failed to handle an authentication request within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:77) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-09-03 03:59:14.391+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=96832, gcTime=4405, gcCount=2}
2024-09-03 03:59:39.440+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4158, gcTime=25024, gcCount=14}
2024-09-03 03:59:43.776+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=24826, gcTime=4333, gcCount=2}
2024-09-03 03:59:48.161+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=4184, gcTime=4383, gcCount=2}
2024-09-03 04:00:03.247+0000 WARN  [o.n.b.p.c.t.r.ResultHandler] Client /172.31.60.134:53744 disconnected while query was running. Session has been cleaned up. This can be caused by temporary network problems, but if you see this often, ensure your applications are properly waiting for operations to complete before exiting.
2024-09-03 04:00:08.281+0000 WARN  [o.n.b.p.c.t.r.ResultHandler] Client /172.31.60.134:36176 disconnected while query was running. Session has been cleaned up. This can be caused by temporary network problems, but if you see this often, ensure your applications are properly waiting for operations to complete before exiting.
2024-09-03 04:00:59.085+0000 WARN  [o.n.b.p.c.t.r.ResultHandler] Client /172.31.60.134:54582 disconnected while query was running. Session has been cleaned up. This can be caused by temporary network problems, but if you see this often, ensure your applications are properly waiting for operations to complete before exiting.
2024-09-03 04:01:12.309+0000 ERROR [o.n.b.p.c.h.HouseKeeperHandler] Fatal error occurred when handling a client connection: [id: 0x3b14627d, L:/172.31.19.69:7687 ! R:/172.31.60.134:37380]
org.neo4j.bolt.runtime.BoltConnectionFatality: Terminated connection 'bolt-2872' ([id: 0x3b14627d, L:/172.31.19.69:7687 ! R:/172.31.60.134:37380]) as the server failed to handle an authentication request within 30000 ms.
	at org.neo4j.bolt.protocol.common.handler.AuthenticationTimeoutHandler.channelIdle(AuthenticationTimeoutHandler.java:77) ~[neo4j-bolt-5.3.0.jar:5.3.0]
	at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403) ~[netty-transport-classes-epoll-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.84.Final.jar:4.1.84.Final]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2024-09-03 04:01:03.202+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=3840, gcTime=74987, gcCount=36}
2024-09-03 04:01:30.859+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=98458, gcTime=27636, gcCount=14}
2024-09-03 04:01:30.859+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.channel.StacklessClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 04:01:30.860+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.channel.StacklessClosedChannelException: null
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
2024-09-03 04:01:30.860+0000 WARN  [i.n.c.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

cdavide8 · September 5, 2024, 9:12am

Wondering whether the machine where the server is running has a capability of 8GB. The memory consumption depends also on what operation the engine is doing with a graph of 1.2 million nodes and 5.1 million relationships. Did the trigger of the issue change? Or it does still happen randomly?

One strategy is to set up a 16GB max heap memory threshold, and make sure that the machine under the hood has such memory capacity provisioned.

You may try to increase the page cache size as well through the configuration server.memory.pagecache.size.

ga.me.rvit78 · September 9, 2024, 4:22am

Sorry for the late reply. The issue still occurs completely on random while writing/updating data into the database through Spring Data. For example, some 1000 nodes were updated in 1 hour the past week and there were no issues. But randomly on one day when there was a request to update a single node, the database terminated the connection. We restarted the database, and the same issue occurred the next day after which we increased the heap size from 4GB to 8GB.

The page cache size is currently 512MB. My senior told me this should be enough for this variable.

cdavide8 · September 9, 2024, 5:57am

Well, the last log is identical to the former one, correct me if I'm wrong: the gc is overwhelmed, the connection is interrupted and the transaction is rolled back. This make me think the issue may still lay on the heap maximum size.

Concerning the query you use to perform the operation, did you manage to decrease memory consumption? If it is a custom query, consider rewriting it.

Another point on my side. I thought you were using plain java with the driver, while now the operation leverages spring data (I think spring-data-neo4j). So, what's the environment? A clear view of the environment would help I guess.

Cheers.

ga.me.rvit78 · September 10, 2024, 2:37pm

Sorry if I didn't mention it earlier but yes we're using spring-data-neo4j and we're making use of the built-in findById and save methods to find the required node to update and then save it respectively. The issue is being created without using a custom query to update the data.
Version of spring being used: 2.7.5
Neo4j Version: 5.3.0 Community Edition

cdavide8 · September 11, 2024, 6:37am

Ok. It seems you're using quite a old version of spring-data-neo4j (SDN), which is 2.7.5. I used to work with that, it should include the OGM library. The OGM essentially maps java objects in neo4j objects and viceversa.
In older versions of SDN, the OGM built internal data structures to speed up the mapping process. As a drawback, these inner data structures need memory.
In the newer versions of SDN, the OGM is no longer available. So the mapping process is slower but less memory-demanding.

I therefore suggest you to switch to a newer version of SDN, the latest as of now is 7.3.3; keep in mind your java version when deciding a new version of the SDN.

You may also try to write a custom query to save data by statically specifying the properties and relationships you actually need to save.

ga.me.rvit78 · September 12, 2024, 2:29pm

Sure, that's what I'm actually doing in the rewrite of the service. I'm directly using the Neo4j driver for java to save the data.
On a side note, after increasing the heap memory from 4GB to 8GB the database hasn't really terminated the connection so far (it's been a week now). I'm hoping this is the end of the issue. If the same issue occurs again, I'll try to talk them into increasing the heap size up to 12GB or 16GB which I think would be more than enough, and if it would still happen, then I think manually restarting the database every time there is a connection termination would be the only solution.

cdavide8 · September 12, 2024, 5:51pm

Fine @ga.me.rvit78 , let me know

Topic		Replies	Views
Unexpected "stop-the-world" pauses while Neo4J is inactive? Neo4j Graph Platform migrated	2	333	August 23, 2024
Neo4j periodically becoming unresponsive Neo4j Graph Platform performance	2	2266	November 11, 2019
Spring Command Line application doesn't terminate after Neo4j query Java spring	3	514	April 15, 2021
Fatal error occurred when handling a client connection causes crash Operations linux , community	5	6254	August 9, 2019
Connection timed out Neo4j Graph Platform	0	1997	November 28, 2018

Neo4j Randomly Terminating Connection With Spring Service

Setup

Expected behavior

Actual behavior

Related topics