Hi, I get the following error every couple hours or so when connecting to my neo4j aura DB instance.
Failed to establish connection to ResolvedIPv4Address(('104.197.20.211', 7687)) (reason [Errno 113] No route to host))
My network connection is strong and while this is happening, my nmap to the neo4j host is a success, so I know this is not an issue with my network:
nmap -p 7687 8db8a329.databases.neo4j.io
Starting Nmap 7.92 ( https://nmap.org ) at 2021-10-16 06:38 EDT
Nmap scan report for 211.20.197.104.bc.googleusercontent.com (104.197.20.211)
Host is up (0.015s latency).
PORT STATE SERVICE
7687/tcp open bolt
Here is a minimal test harness to reproduce the error
driver = GraphDatabase.driver(uri, auth=(user, password), max_transaction_retry_time=1.0)
def workload(tx):
return tx.run("RETURN 1 as n").data()
with watch("neo4j"): # enable logging
with driver.session() as session:
session.write_transaction(workload)
driver.close()
During failure the following log is produced:
2021-10-16 10:38:25,232 [#0000] C: <ROUTING> Checking table freshness (readonly=False)
2021-10-16 10:38:25,236 [#0000] C: <ROUTING> Table expired=True
2021-10-16 10:38:25,239 [#0000] C: <ROUTING> Table routers={IPv4Address(('8db8a329.databases.neo4j.io', 7687))}
2021-10-16 10:38:25,244 [#0000] C: <ROUTING> Table has_server_for_mode=False
2021-10-16 10:38:25,248 [#0000] C: <ROUTING> Checking table freshness (readonly=False)
2021-10-16 10:38:25,250 [#0000] C: <ROUTING> Table expired=True
2021-10-16 10:38:25,252 [#0000] C: <ROUTING> Table routers={IPv4Address(('8db8a329.databases.neo4j.io', 7687))}
2021-10-16 10:38:25,254 [#0000] C: <ROUTING> Table has_server_for_mode=False
2021-10-16 10:38:25,258 Attempting to update routing table from IPv4Address(('8db8a329.databases.neo4j.io', 7687))
2021-10-16 10:38:25,260 [#0000] C: <RESOLVE> 8db8a329.databases.neo4j.io:7687
2021-10-16 10:38:25,328 [#0000] C: <OPEN> 104.197.20.211:7687
2021-10-16 10:38:28,352 [#0000] C: <ERROR> OSError 113 'No route to host'
2021-10-16 10:38:28,353 [#0000] C: <CLOSE> 104.197.20.211:7687
2021-10-16 10:38:28,355 Routing is unavailable (Couldn't connect to 104.197.20.211:7687 (resolved to ('104.197.20.211:7687',)):
Failed to establish connection to ResolvedIPv4Address(('104.197.20.211', 7687)) (reason [Errno 113] No route to host))
2021-10-16 10:38:28,389 [#0000] C: <ROUTING> Deactivating address ResolvedIPv4Address(('104.197.20.211', 7687))
2021-10-16 10:38:28,392 [#0000] C: <ROUTING> table={None: RoutingTable(database=None routers={IPv4Address(('8db8a329.databases.neo4j.io', 7687))}, readers={}, writers={}, last_updated_time=177329.678501131, ttl=0)}
2021-10-16 10:38:28,397 Failed to fetch routing info 104.197.20.211:7687
2021-10-16 10:38:28,400 [#0000] C: <ROUTING> Deactivating address IPv4Address(('8db8a329.databases.neo4j.io', 7687))
2021-10-16 10:38:28,404 [#0000] C: <ROUTING> table={None: RoutingTable(database=None routers={}, readers={}, writers={}, last_updated_time=177329.678501131, ttl=0)}
2021-10-16 10:38:28,409 Attempting to update routing table from
2021-10-16 10:38:28,414 Unable to retrieve routing information
2021-10-16 10:38:28,418 Transaction failed and will be retried in 0.9467342865805752s (Unable to retrieve routing information)
The issue resolves itself after an hour or so and produces a successful result
2021-10-16 10:48:30,613 [#0000] C: <ROUTING> Checking table freshness (readonly=False)
2021-10-16 10:48:30,615 [#0000] C: <ROUTING> Table expired=True
2021-10-16 10:48:30,616 [#0000] C: <ROUTING> Table routers={IPv4Address(('8db8a329.databases.neo4j.io', 7687))}
2021-10-16 10:48:30,619 [#0000] C: <ROUTING> Table has_server_for_mode=False
2021-10-16 10:48:30,621 [#0000] C: <ROUTING> Checking table freshness (readonly=False)
2021-10-16 10:48:30,623 [#0000] C: <ROUTING> Table expired=True
2021-10-16 10:48:30,626 [#0000] C: <ROUTING> Table routers={IPv4Address(('8db8a329.databases.neo4j.io', 7687))}
2021-10-16 10:48:30,630 [#0000] C: <ROUTING> Table has_server_for_mode=False
2021-10-16 10:48:30,632 Attempting to update routing table from IPv4Address(('8db8a329.databases.neo4j.io', 7687))
2021-10-16 10:48:30,634 [#0000] C: <RESOLVE> 8db8a329.databases.neo4j.io:7687
2021-10-16 10:48:30,713 [#0000] C: <OPEN> 104.197.20.211:7687
2021-10-16 10:48:30,766 [#B84A] C: <SECURE> 8db8a329.databases.neo4j.io
2021-10-16 10:48:30,851 [#B84A] C: <MAGIC> 0x6060B017
2021-10-16 10:48:30,853 [#B84A] C: <HANDSHAKE> 0x00030304 0x00000104 0x00000004 0x00000003
2021-10-16 10:48:30,898 [#B84A] S: <HANDSHAKE> 0x00000304
2021-10-16 10:48:30,901 [#B84A] C: HELLO {'user_agent': 'neo4j-python/4.3.6 Python/3.8.12-final-0 (linux)', 'routing': {'address': '8db8a329.databases.neo4j.io:7687'}, 'scheme': 'basic', 'principal': 'neo4j', 'credentials': '*******'}
2021-10-16 10:48:30,949 [#B84A] S: SUCCESS {'server': 'Neo4j/4.3-aura', 'connection_id': 'bolt-63832', 'hints': {'connection.recv_timeout_seconds': 60}}
2021-10-16 10:48:30,952 [#B84A] C: ROUTE {'address': '8db8a329.databases.neo4j.io:7687'} None
2021-10-16 10:48:31,003 [#B84A] S: SUCCESS {'rt': {'servers': [{'addresses': ['neo4j-core-8db8a329-9.production-orch-0002.neo4j.io:7687'], 'role': 'WRITE'}, {'addresses': ['neo4j-core-8db8a329-7.production-orch-0002.neo4j.io:7687', 'neo4j-core-8db8a329-8.production-orch-0002.neo4j.io:7687'], 'role': 'READ'}, {'addresses': ['neo4j-core-8db8a329-7.production-orch-0002.neo4j.io:7687', 'neo4j-core-8db8a329-8.production-orch-0002.neo4j.io:7687', 'neo4j-core-8db8a329-9.production-orch-0002.neo4j.io:7687'], 'role': 'ROUTE'}], 'ttl': 10}}
2021-10-16 10:48:31,006 [#0000] S: <ROUTING> table=RoutingTable(database=None routers={IPv4Address(('neo4j-core-8db8a329-7.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-8.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))}, readers={IPv4Address(('neo4j-core-8db8a329-7.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-8.production-orch-0002.neo4j.io', 7687))}, writers={IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))}, last_updated_time=177936.169131931, ttl=10)
2021-10-16 10:48:31,011 [#0000] C: <UPDATE ROUTING TABLE> address=ResolvedIPv4Address(('104.197.20.211', 7687)) (RoutingTable(database=None routers={IPv4Address(('neo4j-core-8db8a329-7.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-8.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))}, readers={IPv4Address(('neo4j-core-8db8a329-7.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-8.production-orch-0002.neo4j.io', 7687))}, writers={IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))}, last_updated_time=177936.169131931, ttl=10))
2021-10-16 10:48:31,013 [#B84A] C: GOODBYE
2021-10-16 10:48:31,015 [#B84A] C: <CLOSE>
2021-10-16 10:48:31,016 [#0000] C: <ROUTING AGED> database=None
2021-10-16 10:48:31,018 [#0000] C: <ROUTING AGED> last_updated_time=177936.169131931 perf_time=177936.181331931
2021-10-16 10:48:31,019 [#0000] C: <ROUTING TABLE ENSURE FRESH> {None: RoutingTable(database=None routers={IPv4Address(('neo4j-core-8db8a329-7.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-8.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))}, readers={IPv4Address(('neo4j-core-8db8a329-7.production-orch-0002.neo4j.io', 7687)), IPv4Address(('neo4j-core-8db8a329-8.production-orch-0002.neo4j.io', 7687))}, writers={IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))}, last_updated_time=177936.169131931, ttl=10)}
2021-10-16 10:48:31,022 [#0000] C: <ACQUIRE ADDRESS> database=None address=IPv4Address(('neo4j-core-8db8a329-9.production-orch-0002.neo4j.io', 7687))
2021-10-16 10:48:31,023 [#0000] C: <RESOLVE> neo4j-core-8db8a329-9.production-orch-0002.neo4j.io:7687
2021-10-16 10:48:31,682 [#0000] C: <OPEN> 104.197.20.211:7687
2021-10-16 10:48:31,731 [#B84C] C: <SECURE> neo4j-core-8db8a329-9.production-orch-0002.neo4j.io
2021-10-16 10:48:31,779 [#B84C] C: <MAGIC> 0x6060B017
2021-10-16 10:48:31,780 [#B84C] C: <HANDSHAKE> 0x00030304 0x00000104 0x00000004 0x00000003
2021-10-16 10:48:31,827 [#B84C] S: <HANDSHAKE> 0x00000304
2021-10-16 10:48:31,830 [#B84C] C: HELLO {'user_agent': 'neo4j-python/4.3.6 Python/3.8.12-final-0 (linux)', 'routing': {'address': '8db8a329.databases.neo4j.io:7687'}, 'scheme': 'basic', 'principal': 'neo4j', 'credentials': '*******'}
2021-10-16 10:48:31,878 [#B84C] S: SUCCESS {'server': 'Neo4j/4.3-aura', 'connection_id': 'bolt-183988', 'hints': {'connection.recv_timeout_seconds': 60}}
2021-10-16 10:48:31,881 [#B84C] C: BEGIN {}
2021-10-16 10:48:31,885 [#B84C] C: RUN 'RETURN 1 as n' {} {}
2021-10-16 10:48:31,887 [#B84C] C: PULL {'n': 1000}
2021-10-16 10:48:31,937 [#B84C] S: SUCCESS {}
2021-10-16 10:48:31,940 [#B84C] S: SUCCESS {'t_first': 4, 'fields': ['n'], 'qid': 0}
2021-10-16 10:48:31,943 [#B84C] S: RECORD * 1
2021-10-16 10:48:31,946 [#B84C] S: SUCCESS {'type': 'r', 't_last': 1, 'db': 'neo4j'}
2021-10-16 10:48:31,947 [#B84C] C: COMMIT
2021-10-16 10:48:31,991 [#B84C] S: SUCCESS {'bookmark': 'bde05149-1df2-4e10-a084-a9e7ea98bc7f:1821279'}
2021-10-16 10:48:31,993 [#B84C] C: RESET
2021-10-16 10:48:32,036 [#B84C] S: SUCCESS {}
The same failure repeats after a couple hours running my program. Is aura blocking my connection? How do I get around this?