Crashing regularly using serverless setup with next.js

Drivers & Stacks Javascript
1

Every day or so the connection so my neo4j server crashes. This is probably due to the pool size and connection limits. The app and connection works like a charm for a day and then it just stops.

I have tried increasing memory limits and pool size limits and tweaked timeouts, but nothing works. Almost like clockwork the connection crashes.

I don't know what to do and would really like some help.

Here's a gist of my driver initialization: neo4j-driver.ts · GitHub

Errors I'm seeing in the logs:

[Error: Connection acquisition timed out in 60000 ms. Pool status: Active conn count = 0, Idle conn count = 0.]

[Error]: Could not perform discovery. No routing servers available. Known routing table: RoutingTable[database=default database, expirationTime=0, currentTime=1733988069189, routers=[], readers=[], writers=[]]
    at d (/app/.next/server/chunks/192.js:5:779)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async o (/app/.next/server/chunks/192.js:1:9163)
    at async o (/app/.next/server/app/(public)/login/page.js:1:13638) {
  errorCode: 'QUERY_FAILED',
  digest: '3375610673'
}

neo4j-driver: 5.27.0
nextjs: 14.2.15 - running in standalone mode in a docker container.
neo4j: 5.19 - running in docker container

2

Disclaimer upfront: I am no JavaScript (driver) expert.

It looks to me that you initialise a new Driver instance on every executeQuery call. (neo4j-driver.ts · GitHub)
Just read up on this and it is the same as with the other drivers, you have to close those instances. Even more, you should just have one instance in place for the lifetime of your application. The only reason to close this would be a graceful shutdown of the application.

Since I am here, it would also be a good idea to close the sessions after use.

I assume that you are running out of connections because the driver itself will bring its own pool of connections and with every new instance of the driver, you will create more and more connection pools.

This would at least explain to me why you're observing this within ~one day with average usage.

3

I expect this is the reason as well. But I don't know what to do about it. Next.js is a serverless app and thus I can't have a single pool, or single driver instance.

This is why I'm asking for help. I don't know how to pair next.js with neo4j.

Is there a way in the neo4j db to auto-close pools or connections after a certain time? Regardless of what the client does?

4

If this is really the case, your approach is correct. But it might be that those instances might not get cleaned up on the server-side after the request.

To see if this is the root cause of your problem at least, I would advise to do the close call on the database object after processing and before returning the data.

5

Thanks. I was thinking this as well, but I'm not sure where to make this call in next.js. Due to the serverless nature and a lot of abstractions i'm unsure when and where to actually close the driver. This probably would require some next.js expert...

6

Also, do you know where I could find more help regarding this issue, or pay for help? Does neo4j offer this type of support?

7

Sorry I am not aware of a ad-hoc support offering.

What I can help you with is to ask some of our more JavaScript/TypeScript focussed colleagues to have a look at this thread.

Now we are entering real thin ice for me:
But wouldn't it be possible to change the signature of the executeQuery queryFunction from

queryFunction: (session: Session) => Promise<T[]>

to

queryFunction: (driver:Driver) => Promise<T[]>

This way you would provide the driver instance from the outside and call the close method in the async chain's finally, or?

I assume that it might also be true when adding the driver close after the session close in the finally of the try/catch/finally part. But this is where my knowledge about async javascript comes to a hard stop ;)

8

Hey @fredrikburmester, are you deploying the application in a serverless environment or running it as a server?

In my next.js applications, I have read and write functions similar to yours that call a singleton instance of the driver running in the node process for that thread. But then I don't really run the application serverless-ly.

If you call driver.close() rather than session.close() on line 75, all resources will be released and the process will be free to close.

There will be a cold-start and the driver will have to establish a connection the next time a serverless process fires up, but it'll be a solution to this problem at least.

9

Hi, thank you for your answer. I run next.js in standalone mode (hosted on my own servers with docker). And to be honest I don't really know how standalone differs from Vercel deployments under the hood. Does that somewhat answer your question?

I'll try the driver.close() and see if that works.

10

OK great, in that case try closing the driver after each query. The only downside is there will be a few milliseconds wait each time while the driver connects and gets routing information.

If you still experience problems, use a singleton instance of the driver:

let _driver

function createDriver() {
  if (!_driver) {
      const { NEO4J_URI, NEO4J_USERNAME, NEO4J_PASSWORD } = neo4jEnv();
     _driver = driver(NEO4J_URI, auth.basic(NEO4J_USERNAME, NEO4J_PASSWORD), {
      maxTransactionRetryTime: 10000,
      disableLosslessIntegers: true,
      maxConnectionLifetime: 60 * 1000, // 1 minutes
      connectionTimeout: 30 * 1000, // 30 seconds
      maxConnectionPoolSize: 300,
    });
  }

  return _driver
}