I am trying to ingest large data (around a billion nodes and relations) using the neo4j import tool but the ingestion gradually slows down with time. It quickly ingests the first 100 million nodes after which it becomes increasingly slow. I've found people mentioning that they have ingested around 2-3 billion nodes in around 2 hours so can someone point out what I am doing wrong. I have taken care of deduplication as well. Following is the import setup:
Command to ingest data (25 files for nodes and 26 files for relations):
neo4j-admin import --database=neo4j --nodes=... --relationships=... --force=true --delimiter="\t" --skip-duplicate-nodes=true --skip-bad-relationships=true --high-io=true --cache-on-heap=true