I am trying to create knowledge graph from RDF triples produced from an Open IE tool. The issue is that it crashes for large number of requests. So I am trying to scale it. One solution I could think is using Apache Spark.
After creating individual graphs for each request, I would like to merge them into a sinlge large-scale knowledge graph. How is this possible? Can I use MapReduce for this? How will the pipeline look like?
My ultimate goal will be to query upon large-scale knowledge graphs produced. How can I achieve this? Do I need Neo4j for this?