Creating a level 3 graph using JSON Data

I want to create an graph using ne04j which gives me a graph of the terms included in javascript array along with relative terms pointed to terms, and no duplicates if 2 terms have a common relative term it should point to a single relative term

{
 "JavaScript": [
    {
      "termName": "asm.js",
      "link": "/terms/javascript/asmjs",
      "info": "A subset of JavaScript. Effectively describes a safe virtual machine for memory-unsafe languages like C or C++. It can be used as a low-level, efficient target language for compilers.",
      "relatedTerms": [
        {
          "name": "JavaScript",
          "link": "/terms/go_to_term/549"
        },
        {
          "name": "C",
          "link": "/terms/go_to_term/470"
        },
        {
          "name": "C++",
          "link": "/terms/go_to_term/860"
        },
        {
          "name": "compilers",
          "link": "/terms/go_to_term/390"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Cycle.js",
      "link": "/terms/javascript/cyclejs",
      "info": "JavaScript framework for a predictable code. Builds on functional and reactive streams. Describes an app as a simple function taking an event stream as input and outputting an event stream.",
      "relatedTerms": [
        {
          "name": "JavaScript",
          "link": "/terms/go_to_term/549"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Derby",
      "link": "/terms/javascript/derby",
      "info": "Server component which is written on top of Node.js. It's is a server-side application with rich real-time data synchronization technology such used in google docs.",
      "relatedTerms": [
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Express.js",
      "link": "/terms/javascript/expressjs",
      "info": "Lightweight, efficient middleware and routing framework. Express.js is best known as another quarter of the MEAN (MongoDB, Express, AngularJS and Node) software stack, and is the most popular Node.js framework.",
      "relatedTerms": [
        {
          "name": "MEAN",
          "link": "/terms/go_to_term/361"
        },
        {
          "name": "MongoDB",
          "link": "/terms/go_to_term/868"
        },
        {
          "name": "AngularJS",
          "link": "/terms/go_to_term/47"
        },
        {
          "name": "Node",
          "link": "/terms/go_to_term/595"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Flow",
      "link": "/terms/javascript/flow",
      "info": "A static type checker for JavaScript. Integrates with the code editor by checking changes and analyzing correctness of the code. Provides context as to what is wrong with the code.",
      "relatedTerms": [
        {
          "name": "JavaScript",
          "link": "/terms/go_to_term/549"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Hapi",
      "link": "/terms/javascript/hapi",
      "info": "Web framework for building web applications, APIs and services. Hapi was created around the idea that configuration is better than code, that business logic must be isolated from the transport layer.",
      "relatedTerms": [
        {
          "name": "APIs",
          "link": "/terms/go_to_term/206"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Koa",
      "link": "/terms/javascript/koa",
      "info": "Callback-less, lightweight middleware framework for Node.js to build efficient web applications and APIs. Koa.js efficiently uses generators to deal with callbacks and increase error-handling capabilities.",
      "relatedTerms": [
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        },
        {
          "name": "API",
          "link": "/terms/go_to_term/206"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "MEAN",
      "link": "/terms/javascript/mean",
      "info": "A full stack JavaScript solution that refers to the first letters of the four components of a solution for building dynamic websites: MongoDB, a NoSQL database; Express.js, a web applications framework; Angular.js, a JavaScript MVC framework for web apps; Node.js, a software platform for scalable server-side and networking applications.",
      "relatedTerms": [
        {
          "name": "JavaScript",
          "link": "/terms/go_to_term/549"
        },
        {
          "name": "MongoDB",
          "link": "/terms/go_to_term/868"
        },
        {
          "name": "NoSQL",
          "link": "/terms/go_to_term/951"
        },
        {
          "name": "Express.js",
          "link": "/terms/go_to_term/392"
        },
        {
          "name": "Angular.js",
          "link": "/terms/go_to_term/47"
        },
        {
          "name": "MVC",
          "link": "/terms/go_to_term/201"
        },
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Meteor",
      "link": "/terms/javascript/meteor",
      "info": "JavaScript framework built on top of Node.js with MongoDB as its database driver. With Node.js and Meteor, you’re using JavaScript everywhere, and data is synced through the use of WebSockets, a protocol that allows an interactive communication between server and browser.",
      "relatedTerms": [
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        },
        {
          "name": "MongoDB",
          "link": "/terms/go_to_term/868"
        },
        {
          "name": "JavaScript",
          "link": "/terms/go_to_term/549"
        },
        {
          "name": "WebSockets",
          "link": "/terms/go_to_term/1046"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Node.js",
      "link": "/terms/javascript/nodejs",
      "info": "One of the most popular JavaScript frameworks which eases the work of building web applications. By far the fastest growing language in use. Runs on Windows, Linux, and Mac OS. The use of Node.js is mainly for full stack, front-end and back-end.",
      "relatedTerms": [
        {
          "name": "JavaScript",
          "link": "/terms/go_to_term/549"
        },
        {
          "name": "Windows",
          "link": "/terms/go_to_term/1373"
        },
        {
          "name": "Linux",
          "link": "/terms/go_to_term/558"
        },
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Oracle JET",
      "link": "/terms/javascript/oracle_jet",
      "info": "Oracle JavaScript Extension Toolkit. A modular toolkit based on modern JavaScript, CSS3 and HTML5 design and development principles. Created for developers who work on applications that consume and interact with Oracle products and services.",
      "relatedTerms": [

      ],
      "category": "JavaScript"
    },
    {
      "termName": "Restify",
      "link": "/terms/javascript/restify",
      "info": "A Node.js module built specifically to enable to build correct REST web services. It borrows heavily from express as that is more or less the de facto API for writing web applications on top of Node.js.",
      "relatedTerms": [
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        },
        {
          "name": "REST",
          "link": "/terms/go_to_term/524"
        },
        {
          "name": "API",
          "link": "/terms/go_to_term/206"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Sails",
      "link": "/terms/javascript/sails",
      "info": "Sails.js wraps express and provides another higher level for doing things like connecting to the DB, auto-generating input pages, etc. Especially suitable for creating chat apps or multiplayer games.",
      "relatedTerms": [
        {
          "name": "DB",
          "link": "/terms/go_to_term/415"
        }
      ],
      "category": "JavaScript"
    },
    {
      "termName": "Seneca",
      "link": "/terms/javascript/seneca",
      "info": "Microservices toolkit for Node.js. Helps to write an organized code that can be scaled and deployed at any time. Organizes the business logic of an app.",
      "relatedTerms": [
        {
          "name": "Node.js",
          "link": "/terms/go_to_term/595"
        }
      ],
      "category": "JavaScript"
    }
  ]
  }

Above is the JSON My Expected Result Should look like: [Expected](Screenshot from 2020-01-28 15-42-32 )

Here is the Actual Result: [Actual]

Following is the Query which returns the Result

CALL apoc.load.json("file:///js.json") YIELD value
UNWIND value.JavaScript AS item
FOREACH(js IN item |
CREATE(cat:Category{name:js.termName})
FOREACH(rt in item.relatedTerms|
CREATE(rtt:RelatedTerms{name:rt.name})
MERGE (cat)-[r:RELATESTO]->(rtt)))

The switching the CREATE statements for MERGE statements. In Cypher, MERGE will check to see if a pattern exists and create it if it does not. CREATE will create a new pattern every time, even if a pattern like it already exists in the graph.

CALL apoc.load.json("file:///js.json") YIELD value
UNWIND value.JavaScript AS item
MERGE (cat:Category {name:item.termName})
WITH cat, item
UNWIND item.relatedTerms AS subitem 
MERGE (rt:RelatedTerm {name:subitem.name})
MERGE (cat)-[:RELATES_TO]->(rt)
RETURN *
1 Like

Screenshot from 2020-01-28 18-20-20
I am getting this result but it is not as i expected can you suggest me any logical change if any.

I updated my answer to use two unwind statements instead of a foreach. I think that will give you a better result.

1 Like

Thank You but how can i remove the duplicates?

Try this:

CALL apoc.load.json("file:///js.json") YIELD value
UNWIND value.JavaScript AS item
MERGE (cat:Category {name:item.termName})
WITH cat, item
UNWIND item.relatedTerms AS subitem

WITH distinct subitem as subitems, cat, item

MERGE (rt:RelatedTerm {name:subitems.name})
MERGE (cat)-[:RELATES_TO]->(rt)
RETURN *

1 Like

hi @dhiresh,
may I suggest to use the apoc.graph.fromDocument procedure?
Please look at this article where is used in combination with apoc.mongo.get but you should easily combine it with the apoc.load.json

(disclaimer: I wrote that article)

2 Likes

@ameyasoft Thanks for the help, but even this didn't work i am getting duplicates and also some terms are missing which have no related Terms.

hello @conker84, Thank you for the suggestion will try to implement the procedure using the article.

@conker84
CALL apoc.load.json("file:///js.json") YIELD value
UNWIND value.JavaScript AS item
CALL apoc.graph.fromDocument({item},
{
write: false,
mappings: {
$: 'JavaScript:Term{*}',
$.relativeTerms: 'related{!link, name}'
}
}) yield graph as g
return g;

I ran the following query but i am getting the Following Error

Neo.ClientError.Statement.ParameterMissing
Expected parameter(s): item

Any Further Instructions?

Thank You @nsmith_piano my db was populated that is why i was getting the duplicates i created a fresh DB and ran the Query which helped me out.

1 Like

@ameyasoft Thank you, Even your query returned the correct output

1 Like

This should work:

CALL apoc.load.json("file:///js.json") YIELD value
UNWIND value.JavaScript AS item
CALL apoc.graph.fromDocument(item,
{
write: false,
mappings: {
$: 'JavaScript:Term{*}',
$.relativeTerms: 'related{!link, name}'
}
}) yield graph as g
return g;
2 Likes

Thank You, but the graph has some duplicates running through this query and my problem statement is solved by the above solution.Thanks for the alternative solution.

@dhiresh can you share your full file?