Issue with Python Driver and consume(), profile to return db hits and time

Hi Neo4j community,

I have now been trying for some time to figure out why using the Neo4j Python Driver getting the dbhits and times for some queries does not seem to work and I cannot figure out why.

I have created a Database class in Python:

class gdbms_test:


    def __init__(self, uri, user, password):
        self.driver = GraphDatabase.driver(uri, auth=(user, password))

    def close(self):
        self.driver.close()

    def reset(self):
        with self.driver.session() as session:
            session.run("MATCH (m) DETACH DELETE m")

    def execute_query(self, query):
        with self.driver.session() as session:
            session.run(query)

    def execute_query_with_output(self, query):
        with self.driver.session() as session:
            record = session.run(query)
        return record


    def execute_query_with_output_result(self, query):
        with self.driver.session() as session:
            record = session.run(query)
        return [dict(i) for i in record]

and get the total db hits / time as follows:

def sum_db_hits(profile):
    return (profile.get("dbHits", 0) + sum(map(sum_db_hits, profile.get("children", []))))


# getting execution time

def sum_time(profile):
    return (profile.get("time", 0) + sum(map(sum_time, profile.get("children", []))))


def show_query_details(database, query):

    
    new_db = database
    
    result = new_db.execute_query_with_output(query)

    summary = result.consume().profile

    return  (sum_db_hits(summary), sum_time(summary))

Later on in my code I do the following:

update = "PROFILE MATCH (ad:Actor:Director) WHERE ad.name IS NOT NULL AND ad.bornIn IS NOT NULL AND ad.name = 'Larry David' SET ad.languages = 'English'"
            current_db_hits, current_time = show_query_details(new_db, update)

This works perfectly fine and I get the respective total db hits and total time.

However, doing the following (query with result instead of an update)

query = "PROFILE MATCH (ad:Actor:Director) WHERE ad.name IS NOT NULL AND ad.bornIn IS NOT NULL RETURN ad.name"
            current_db_hits, current_time = show_query_details(new_db, query)

this does not seem to work.

I did a fair bit of trial and error, looked here, Stackoverflow, etc. as well as Python Driver documentation and cannot seem to find where things go wrong.

In the code snippet above once I perform the query instead of the update before the line

summary = result.consume().profile

everything is fine before that line and after the .profile returns in this case a None object.

Having split it up into

summary = result.consume()

and then performing the .profile yielded the following:

<neo4j.work.result.Result object at 0x000001F146A96C70>
<class 'neo4j.work.result.Result'>
<class 'neo4j.work.summary.ResultSummary'>
None
<class 'NoneType'>

When performing the update instead of the query I get

<neo4j.work.summary.ResultSummary object at 0x0000023CBB2C9EE0>
<class 'neo4j.work.summary.ResultSummary'>
{'args': {'GlobalMemory': 192, 'planner-impl':..........}
<class 'dict'>

and everything works fine.

Even more weird is the following:

I got all of this running within a loop and one time I executed I believe the first time everything worked fine for the query and I got a dictionary and at the second run I got a None object again. This is why I thought, maybe some code is not executed at a certain point to proceed further and I put in some sleep to delay things but this also did not work.

I am glad for any help and any pointers you might be able to provide.

Thanks,
Philipp

Hi Philipp.

I tried to reproduce the issue but couldn't. Please share a little mor info:

  • What driver version are you using?
  • What server version are you using?
  • How is the DBMS hosted? Is it running locally for testing, on Aura, single instance or a cluster, ...?

Here's what I ran against a freshly created Aura free instance:

from __future__ import annotations

import neo4j


URI = "neo4j+s://***.databases.neo4j.io"
AUTH = ("neo4j", "***")
DB = "neo4j"


def sum_db_hits(profile: dict):
    return (profile.get("dbHits", 0)
            + sum(map(sum_db_hits, profile.get("children", []))))


def sum_time(profile: dict):
    return (profile.get("time", 0)
            + sum(map(sum_time, profile.get("children", []))))


def show_query_details(summary: neo4j.ResultSummary):
    profile = summary.profile
    return sum_db_hits(profile), sum_time(profile)


def main():
    with neo4j.GraphDatabase.driver(URI, auth=AUTH) as driver:
        # wipe the whole database and populate it with sample data
        driver.execute_query("MATCH (n) DETACH DELETE n", database_=DB)
        driver.execute_query(
            "CREATE (ad:Actor:Director $props)",
            props={
                "name": "Larry David",
                "bornIn": 1947,
                "languages": "Testing",
            },
            database_=DB,
        )
        # now testing the profiles
        res = driver.execute_query(
            "PROFILE MATCH (ad:Actor:Director) "
            "WHERE ad.name IS NOT NULL "
            "AND ad.bornIn IS NOT NULL "
            "AND ad.name = 'Larry David' "
            "SET ad.languages = 'English'",
            database_=DB,
        )
        print(res)
        print(show_query_details(res.summary))

        res = driver.execute_query(
            "PROFILE MATCH (ad:Actor:Director) "
            "WHERE ad.name IS NOT NULL "
            "AND ad.bornIn IS NOT NULL "
            "RETURN ad.name",
            database_=DB,
        )
        print(res)
        print(show_query_details(res.summary))


if __name__ == "__main__":
    main()

And the result:

EagerResult(records=[], summary=<neo4j._work.summary.ResultSummary object at 0x7f33879f61e0>, keys=[])
(6, 0)
EagerResult(records=[<Record ad.name='Larry David'>], summary=<neo4j._work.summary.ResultSummary object at 0x7f33879fe900>, keys=['ad.name'])
(5, 0)

No issues with the profile being None :thinking:

Hi Rouven,

Thanks for getting back. Good to know that it seems to be an issue on my end and not a general issue.

I am hosting the db locally, single instance using Neo4j Desktop 1.5.9

and

You are running

Neo4j Browser version: [5.15.0]

Neo4j Server version: [4.4.8]

Also I am using Python Driver 5.17.0

Also, I tried a little bit around and now after seeing what my output looks like at a certain time I got the following error:

I hope this might help.

Please let me know in case you have any ideas or if you may need any further information. Much appreciated.

Cheers,
Philipp