C# driver on GCP Aura: Failed to acquire a connection from connection pool for server

Drivers & Stacks
,
#1

Hi team,

we are facing connection issues with the C# driver connecting to GCP Aura. Once in a while, we do see the following error in the app logs. Thanks for helping out!

Setup:

  • Microservice developed in ASP.NET 5
  • Neo4J Aura on GCP

Error:

Failed to acquire a connection from connection pool for server with URI `neo4j://GCP_ID.production-orch-0001.neo4j.io:7687/` as this server has already been removed from routing table. Please retry your query again and you should be routed with a different server from the new routing table. You should not see this error persistently.

Driver:

public Neo4JRepository(ILogger<Neo4JRepository> logger, IConfiguration configuration)
		{
			var username = configuration["Neo4j:Username"];
			var password = configuration["Neo4j:Password"];
			var uri = configuration["Neo4j:Uri"];

			var authToken = AuthTokens.None;

			if (!string.IsNullOrEmpty(password) && !string.IsNullOrEmpty(username))
				authToken = AuthTokens.Basic(username, password);

			_driver = GraphDatabase.Driver(uri, authToken);
			_logger = logger;
		}

		public async Task CreateRootNode()
		{
			var session = _driver.AsyncSession();
			try
			{
				// Write transactions allow the driver to handle retries and transient error
				await session.WriteTransactionAsync(async tx =>
				{
					// The real work is done here.
				});
			}
			catch (Neo4jException ex)
			{
				_logger.LogError(ex, "Error occurred by creating .....");
				throw;
			}
			finally
			{
				await session.CloseAsync();
			}
		}
#2

We are also facing the same exact issue sometimes intermittently, Help will be appreciated.

#3

@chandarmk
This was linked to network configurations on TCP level. Setting MaxConnectionLifetime==8 minutes should help.

#4

@marian.zoll

We are facing a similar issue with a causal cluster though.
Did the above setting help is resolving the issue?

#5

I guess yes, cause this is affecting the C# driver and not directly any cluster related setting.

#6

Hey, @marian.zoll Thanks for your answer and sorry for the late reply,

After some period of inactivity, I am getting two kinds of errors one is "Failed to acquire a connection" and another is "Failed after retried for 6 times in 30000 ms. Make sure that your database is online and retry again"

I solved the problem by watching the ServiceUnavailableException exception, if there is such an exception then I reestablished the connection

I also included the following today to see whether the occurrence of such exceptions comes down. I will watch it over and update it.

Driver = GraphDatabase.Driver(Url, AuthTokens.Basic(UserName, Password),
o =>
{
o.WithLogger(new LogWrapper())
.WithSocketKeepAliveEnabled(true)
.WithMaxConnectionLifetime(TimeSpan.MaxValue);
});