I am trying to do a project which will effectively process data from the mysql binlogs and reflect all the changes in my sql Db to the graph in real time.I want to know if real time processing is feasible for the following process. I have a stored all the parent and child relations present in my Sql database in my graph.The model is as follows: Each table row is represented as a row.The child rows to parent rows relations in mysql are represented as relationships in neo4j. Mysql binlogs do not show the child tables that get deleted when a parent row gets deleted.So my aim is to track the changes in the graph and return the records or nodes that are also deleted in the graph along with its primary key value to a java program which then pushes it to a kakfa topic. This above concept is part of a bigger architecture that has to ensure real time processing of data events from the mysql binlogs to kafka topics. Any suggestions on how to maintain real time data processing will be appreciated.Thanks!
The following image describes the graph model denoting the foreign key constraints of the sql tables as relationships.
