cancel
Showing results for 
Search instead for 
Did you mean: 

Best practices for transforming rows of data in nodes & relationships

Nodeynode
Node

Are there articles, code snippets or architectural diagrams that shows how best to transform data from a traditional database or siem into neo4j?

I'm thinking actual design, the code, the schema and gotchas for how to logic through the transformation. The first thing that comes to mind is how do you prevent sending millions of duplicate rows to the db over API CREATE statements.

This seems like the most difficult part of making this product useful. Once the data is in, cypher query is great for most of our needs.

2 REPLIES 2

glilienfield
Ninja
Ninja

This is a good resource to start with.

https://neo4j.com/developer/guide-importing-data-and-etl/

Thank you. I did see that and should have been a little more clear.

What about from a Kafka or kinesis streaming standpoint? Is the deduping done before the MERGEs or do people just slam the db with one call per event?

When are the relationships built? As events are streamed in? Or some kind of batch job that runs over the neo4j db after the streaming creates the nodes?