Python - generator already executing

Hi,
We are using Neo4j driver for python(version 4.2.1).
We have a live system that writes/read from the neo4j cluster all the time, And once in a while we are getting this exception:

Traceback (most recent call last):
  File "main.py", line 85, in on_new_snapshot
    response = customers_list[customer_id]['customer_model_handler_obj']\
  File "/ms_topology_center/../ms_topology_center/customer_model_handler.py", line 130, in on_new_snapshot
    return self.load_last_snapshot_to_graphdb(timestamp=timestamp,
  File "/ms_topology_center/../ms_topology_center/customer_model_handler.py", line 119, in load_last_snapshot_to_graphdb
    self.modeler.create_services('agnostic', region_snapshot_data['created'], account_id, region, timestamp)
  File "/ms_topology_center/../ms_topology_center/modeler.py", line 115, in create_services
    self.graph_db.commit_transaction()
  File "/ms_topology_center/../shared/graph_db.py", line 135, in commit_transaction
    self._transaction.commit()
  File "/usr/local/lib/python3.9/site-packages/neo4j/work/transaction.py", line 134, in commit
    self._connection.fetch_all()
  File "/usr/local/lib/python3.9/site-packages/neo4j/io/_bolt4.py", line 429, in fetch_all
    detail_delta, summary_delta = self.fetch_message()
  File "/usr/local/lib/python3.9/site-packages/neo4j/io/_bolt4.py", line 334, in fetch_message
    details, summary_signature, summary_metadata = next(self.inbox)
  File "/usr/local/lib/python3.9/site-packages/neo4j/io/_common.py", line 79, in __next__
    tag, fields = self.pop()
  File "/usr/local/lib/python3.9/site-packages/neo4j/io/_common.py", line 73, in pop
    return next(self._messages)
ValueError: generator already executing

After this exception appers we are getting several other very strange exceptions:

  • Failed to read from defunct connection IPv4Address(('10.0.201.154', 7687)) (IPv4Address(('10.0.201.154', 7687)))

  • No data

  • Internal server error (Failed to write to closed connection IPv4Address(('10.0.201.154', 7687)) (IPv4Address(('10.0.201.154', 7687))))

File "/ms_topology_center/../shared/graph_db.py", line 135, in commit_transaction
    self._transaction.commit()
  File "/usr/local/lib/python3.9/site-packages/neo4j/work/transaction.py", line 133, in commit
    self._connection.send_all()
  File "/usr/local/lib/python3.9/site-packages/neo4j/io/_bolt4.py", line 296, in send_all
    raise ServiceUnavailable("Failed to write to closed connection {!r} ({!r})".format(
neo4j.exceptions.ServiceUnavailable: Failed to write to closed connection IPv4Address(('10.0.201.154', 7687)) (IPv4Address(('10.0.201.154', 7687)))
  • [Errno 9] Bad file descriptor

After I restart the pod everything back to normal.

Any idea why it's happened to us and how can we avoid this kind of errors?

Thanks ,
Dan Yazerski

1 Like

Hi @dan3 , Did you find a solution to this problem?

I created a GH issue for it:

This sounds like a concurrency issue outside the driver. However, without seeing a code sample that is capable of reproducing this bug, I won't be able to say for sure. ValueError: generator already executing is raised when multiple threads try to consume from one generator. Note that the driver object is thread safe, the session objection isn't. You have to spawn (at least) one session per thread.