The connection speed of the Python driver is very slow. Is this normal?

Drivers & Stacks
1

The official Neo4j Python driver takes 20 seconds to establish a database connection, which is excessive. In the browser, it only takes 2 ms, and py2neo works fine. I'm using Python version 3.11 and have also tried other versions, 3.9 and 3.7. The Neo4j database has been used with three versions: latest, 5.12, and 5.13. The database was set up using Docker, and I've tried three different versions. All of them are empty databases. Do you have any solutions for this?
The time is in the lower left corner of the picture, the same query, py2neo runs in less than 1s

Clip_2024-10-15_19-33-03
Clip_2024-10-15_19-33-03770×552 19.2 KB

docker-compose.yaml

services:
  database-neo4j:
    image: neo4j:5.13
    container_name: neo4j-513
    network_mode: bridge
    ports:
      - 7687:7687
      - 7474:7474
    volumes:
      - ./conf:/var/lib/neo4j/conf
      - ./data:/data
      - ./logs:/logs
      - ./plugins:/plugins
      - ./import:/import
    environment:
      - NEO4J_AUTH=neo4j/dwsSap
      - NEO4J_PLUGINS=["apoc"]
      - NEO4J_dbms_security_procedures_unrestricted=apoc.*
      - NEO4J_server_memory_heap_initial__size=4G
      - NEO4J_server_memory_heap_max__size=8G
      - NEO4J_dbms_memory_transaction_total_max=8G
      - dbms_default__database=neo4j
    restart: unless-stopped
2

Not sure how you are measuring the time it takes to establish a database connection? My guess is:

from neo4j import GraphDatabase takes a bit of time (when I run it in a new kernel it takes 2 seconds the fist time, 0 if I run it a second time)

3

I set three times. It takes a very long time when first query. In addition, the second query is 0s.
the print output:

connect_time: 0.0
session_time: 0.0
query_time: 21.06687641143799

from neo4j import GraphDatabase
from dotenv import load_dotenv
import os
import time
load_dotenv()

class Neo4jConnector:
    def __init__(self):
        start_time = time.time()
        self.driver = GraphDatabase.driver(
            os.getenv("NEO4J_URI"), auth=("neo4j", os.getenv("NEO4J_PASSWORD"))
        )
        end_time = time.time()
        time1 = end_time - start_time
        print(f'connect_time: {time1}')

    def query(self, query, params=None):
        start_time = time.time()
        with self.driver.session() as session:
            end_time = time.time()
            time2 = end_time - start_time
            print(f'session_time: {time2}')
            start_time = time.time()
            result = session.run(query, params)
            end_time = time.time()
            time3 = end_time - start_time
            print(f'query_time: {time3}')
            return result
        
neo4j = Neo4jConnector()
results = neo4j.query("RETURN 'Hello World' as message")
print(results)
4

I ran you code and got:

connect_time: 0.0001049041748046875
session_time: 4.6253204345703125e-05
query_time: 0.06154322624206543
<neo4j._sync.work.result.Result object at 0x1194e8760>

My neo4j python driver version is 5.17.0 (and my database version 5.24.1)

The query time goes down if I also set the database in the session:

with self.driver.session(database='neo4j') as session:

But I would start by checking that you are actually on a recent version of the python driver.

5

I guess you can always double check so there is not something completely off with the docker bridge with net cat or similar tool:

time nc -z localhost 7687
Connection to localhost port 7687 [tcp/*] succeeded!
nc -z localhost 7687  0.01s user 0.00s system 39% cpu 0.024 total
6

I ran this code on my other computer, and the result was the same as yours. Both computers are using WSL, one with Debian and the other with Ubuntu (which is working fine). I'm curious why the problematic session doesn't consume time.

connect_time: 0.00012636184692382812
session_time: 3.3855438232421875e-05
query_time: 0.013840913772583008

I'll check the package versions and look into the WSL Debian issue later. If I figure it out, I'll update you here. Thanks for your help!

1 Like
7

You can turn on driver logging (it will be quite verbose, but it may help you identify exactly what is taking time) API Documentation — Neo4j Python Driver 5.25

But considering docker networking + wsl networking is at play, I would widen the investigation and also look at what may be causing issues there. It does not hurt to double check layer by layer how fast you can reach the container running the db.

8

It is indeed a network issue; it points localhost to [::1], but Docker does not have IPv6 enabled. I tried to fix this problem, but after removing [::1] from the hosts file, localhost still points to [::1]. I resolved it by setting Bolt to 127.0.0.1.

9

except C:\Windows\System32\drivers\etc\hosts
Is there something else that would affect the resolution of localhost？

10

My windows skills are not what they used to be. I would assume it has gotten more complicated than just the hosts file these days. I think there are many ways of running docker in wsl as well. Hopefully you can find bits and pieces in windows forums or by searching the web.