# Workflow for threads and writing to Neo4j?

**URL:** <https://community.neo4j.com/t/workflow-for-threads-and-writing-to-neo4j/20347>\
**Category:** Python\
**Created:** [June 20, 2020, 3:54pm UTC](https://community.neo4j.com/t/workflow-for-threads-and-writing-to-neo4j/20347 "2020-06-20T15:54:19Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rogie](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/rogie/32/3913_2.png) [@Rogie](https://community.neo4j.com/u/Rogie)\
**Post date:** [June 20, 2020, 3:54pm UTC](https://community.neo4j.com/t/workflow-for-threads-and-writing-to-neo4j/20347/1 "2020-06-20T15:54:19Z")

</div>

I have code that reads data from the web, cleans it up, and stores it in Neo4j. I'm wondering how to "parallelize" this process, since getting the data from web can be slow sometimes. My current setup is something like this:

In `config.py` :

```auto
from neo4j import GraphDatabase

class cfg_holder():
    ''' Container for global variables.'''
    def __init__ (self, params):
        self.params = params
        self.uri = "bolt://localhost:7687"
        self.driver = GraphDatabase.driver(self.uri, auth=("user", "pass"))
        self.db = self.driver.session()

def init(param):
    return cfg_holder(param)

```

In `main.py` :

```auto
import concurrent.futures
import config

def func(h):
    # get data
    # build queries
    # when enough data has been collected:
    with h.db.begin_transaction() as tx:
        tx.run(q)
        tx.success = True

if __name__ == " __main__":
    holders = []
    for i in [10, 20]:
        holders.append(config.init(i))
    with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
        executor.map(func, holders)

```

So each `cfg_holder` has access to its own db connection. I'm not sure this is the correct way to set things up.

It's possible that my design pattern is entirely off here. What's the right way to set this kind of thing up? Do I need to be locking the threads somewhere? Are threads even the right way to go about this? Looking for some general advice here...

---

<div class="post-metadata">

**Author:** ![jggomez](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/jggomez/32/7793_2.png) [@jggomez](https://community.neo4j.com/u/jggomez)\
**Post date:** [June 20, 2020, 8:45pm UTC](https://community.neo4j.com/t/workflow-for-threads-and-writing-to-neo4j/20347/2 "2020-06-20T20:45:04Z")

</div>

Hi, I tried too but I tried with multiprocessing with python and I had problem and doesn't work. The reason because the driver doesn't support that, I don't know if the new driver for neo4j 4.0 supports that.

---

<div class="post-metadata">

**Author:** ![Rogie](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/rogie/32/3913_2.png) [@Rogie](https://community.neo4j.com/u/Rogie)\
**Post date:** [June 21, 2020, 12:50am UTC](https://community.neo4j.com/t/workflow-for-threads-and-writing-to-neo4j/20347/3 "2020-06-21T00:50:57Z")

</div>

Hello, can you please tell me what it is exactly that the driver doesn't support?

---

<div class="post-metadata">

**Author:** ![jggomez](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/jggomez/32/7793_2.png) [@jggomez](https://community.neo4j.com/u/jggomez)\
**Post date:** [June 23, 2020, 2:37am UTC](https://community.neo4j.com/t/workflow-for-threads-and-writing-to-neo4j/20347/4 "2020-06-23T02:37:03Z")

</div>

Multiprocessing and threads in python