Need to limit the record size of the database while using NEO4J ETL TOOL

Neo4j Graph Platform Import / Export
1

we are trying to test some limited records from database to neo4j desktop but it is having billions of records in our end and while importing it is taking more time to import the data.

Is there any option to limit the records while migrating for testing purpose?

2

Sorry, I may not fully understand. If you are importing data you are in full control of your data to import. What is it you want to limit?

Do you have a particular query you are using ?

3

Hi,
i am having billions of records for one particular table so if i want to limit for example if i need only 1000 records to import in neo4j desktop, how to limit the records from my database to neo4j?
is there any options to limit the records before importing into neo4j?

4

I think you are stating you have another database with billions of records and you are asking how to limit the import?

How are you extracting the data from your other database and importing into neo4j? Are you dumping it into a csv file?

5

No, from ETL tool i am migrating my graph database to neo4j tool but the records are huge, so when mapping and importing the data into neo4j it is taking so many hours to complete.
For the testing purpose do we have any option to limit the records in the graph database?
for example: I am having 1000000 rows in my database and i require to process only 1000 records in neo4j, How can we achieve this instead of transferring full database into neo4j?
Can you suggest any idea?

6

Now I understand a little better. What ETL tool are you using

7

Neo4j ETL tool with JDBC connection only

8

I don’t see a means with that tool to extract a subset of the records.

Suggestions:

Can you export a subset of the records to a csv file?
Can you create a view in your database, so you can extract from the view instead of the table. You can create a view form a select statement that limits the number of rows.
Can you copy a subset of the records to new table for the purpose of extracting from the new table, which will have a small subset of the records.
Use the apoc.load.jdbc procedure to load a set of records based on a select statement.

9

sure will try. Thank you so much for your suggestions.

1 Like