Neo4j itself (the database) doesn't get packaged with AWS Lambda functions, but the python driver to talk to Neo4j does.
AWS has documentation on how to package any python dependency together with the python code for your lambda function, which you can find here:
The short version is that you create a ZIP file, and what you push to the cloud to be your lambda function isn't your python code, but rather a ZIP archive containing your python code and other things, including dependencies.
I would really recommend a python virtual environment setup like venv, poetry, etc. This can help with the packaging process
They have a full worked example here: Deploy Python Lambda functions with .zip file archives - AWS Lambda
Finally -- with Neo4j/python, all you really need is the neo4j driver
pip install neo4j as a dependency. Everything else is the same, you can follow those docs above and there aren't any Neo4j specific differences.