I am doing load testing of neo4j with JMeter, for 200 concurrent connections neo4j is working fine. But whenever I am increasing the load over server by increasing the number of concurrent connections from 200 to 300 or 400 or 600 in that case neo4j is throwing error “ Unable to acquire connection from the pool within configured maximum time of 60000ms”. Even I have checked within neo4j.properties file to change default connection timeout from 60s to more. But there is no such timeout attribute within neo4j.properties file which default value is 60s.
Can someone please help me out here how to fix this timeout error.
Neo4j version : 4.1.0 enterprise
debug log message
io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection timed out
ERROR [o.n.b.t.p.HouseKeeper] Fatal error occurred when handling a client connection: [id: 0SDFRDSdb3] readAddress(..) failed: Connection timed out
What are you trying to accomplish? Load testing is meant to find the limit of performance and concurrent requests, so you can plan and implement and infrastructure that can handle the traffic you're expecting. You found the limit. Now what?
Infrastructure
Now that you know the "limit" of concurrent connections, you can use sharding, or casual-clusters, or a load-balanacer, to add instances to support up to the number you're expecting.
Improve load test results
Alternatively, you can fine-tune your environment and configuration to increase the number of concurrent connections before a failure. The simplest way is more cores, and allocating more cores to the JVM and Neo4j config.
As long as you're load testing, you'll always eventually get an error. The whole point of load testing is to find when and why you hit a limit. What you do about that limit is up to you.
How to improve the results
The "pool" referenced in that message is the Java thread pool. You've probably exceeded the max number of concurrent threads.
You'll need to experiment with some of the more granular configuration options:
Depending on your environment, there are many things that can cause Neo4j to max-out a pool.
Here's a couple examples to get you started, but the list is long.
In casual clusters, the *.xxx_parallelism determines the max number of concurrent threads for that service.