Retrieving list of nodes & creating relationships

Drivers & Stacks Python
1

Hey

py2neo for the moment, But trying to build most of the stuff via queries. In any case, I'm struggling with this code... How can I properly and efficiently create 100k+ relationships?

# My getter to return list of nodes
def getNodesByType(type, limit=-1):
    if limit > 0:
        a = graph.run('''
        MATCH (n:{type}) 
        RETURN n 
        LIMIT {limit}'''.format(type=type, limit=limit))
        return a
    return graph.run('''
    MATCH (n:{type}}
    RETURN n'''.format(type=type))

def makeConnection(source, key, dest):
    tx = graph.begin()
    tx.run('''
    UNWIND $arrSource AS mSrc
    CALL apoc.create.relationship(mSrc,{k}, {d})    
    YIELD node
    RETURN node
                 '''
           , arrSource=source, k=key, d=dest)
     tx.commit()

nodeA = getNodesByType("Pets", 10).to_ndarray()
nodeB= getNodesByType("Objects", 3).to_ndarray()

makeConnection(nodeA,"livesIn",nodeB[0])
makeConnection(nodeA[0:5000],"livesNextTo",nodeB[1])
makeConnection(nodeA[50000:99999],"livedIn",nodeB[2])

I keep on getting

TypeError: Neo4j does not support JSON parameters of type ndarray

I'm lost, I mean I'm passing Node object to it, so maybe thats wrong? How should I execute this task ?

Regards
Dariusz

Edit now that I look at it, I'm not sure if I can use the apoc as I need CREATE UNIQUE as I want to only create a relationship if it's missing.

3

Hey
I'm starting at it & trying to wrap my head around, thank so much for fast help!

I take your Node has "iid" as "key" ? So Node(iid:"someid") ?
so the iid:row.id1/id2 standas for dics entries inside row mhmhmhmhmhmmm ok I see ! So I should not be passing directly Node form py2neo but rather grab node["myId"] instead and pass that. kkk gacia!.

I think I got it, will test it. As to other thing, how can I make sure that the connection is Unique ? I mean, I want to create relationship only if its missing here.

I don't see apoc.createRelationship.unique()

Regards
Dariusz

4

No problem, to be honest I never used py2neo, I prefer to use Neo4j Python driver (Neo4j Python Driver 5.4 — Neo4j Python Driver 5.4)

I don't know if I'm the good guy to help you with the py2neo thing but I can give you some advices:)

You can check if the relationship exists before to create it thanks to EXISTS (Predicate functions - Cypher Manual)

5

Hey
I'm not dead set on py2neo, it just was a lib that was proposed on neo youtube channel > Graph Databases for Python Users - YouTube
So I can move to neo4j too.

As to the issue...
I've written my command as follow:

    cmd = '''
        MATCH (dest:{destTag}{{icId:"{destId}"}})
        UNWIND $arrSource AS row
        MATCH (source:{srcType}{{icId:row.id}})
        CALL apoc.create.relationship(source, "{k}", {{relation:"LINKED_TO"}}, dest)
        YIELD rel
        RETURN rel
                         '''.format(destId=dest[0]["icId"], destTag=destType, srcType=sourceType, k=key)
    print(cmd)
    c = tx.run(cmd, arrSource=data)

As far as I understand UNWIND, its a "loop", so I decided to match dest outside it(I guess?) as dest is constant, and perform the func above. It does work, which is great but it does not handle existing connections as it should.

I started reading the docs but I'm lost, the 2 examples they show are not enough and other stuff I read is just uhhh.... where is "if else...".... :- (((( :- )

"big brain time" :- ) Lets see if I can crack it ^^ brb.

6

For the if-else, in Cypher it's called CASE Expressions - Cypher Manual

And if you want to check if the relation exists, you can do something in this style after the second MATCH:

WHERE NOT EXISTS((dest)-[:LINKED_TO]->(source))
7

Hello

Nope, I can't get through that if statement system. It is making my head hurt :- )

This is as far as I got >

MATCH (ct:Pets{icId:"800a1031-1102-49c4-abdd-f294ae867821"})
MATCH (gt:Objects{name:"geo_98608"})
CALL apoc.when([NOT EXISTS((gt)-[:LINKED_TO]->(ct)),
'RETURN ct'],
'RETURN gt')
YIELD VALUE
RETURN VALUE

I was testing if I can just "test" and return either object depending on test, I think I need apoc.do.when for creating relationship, but for now its just read.

Anyway all I get is
image

Regards
Dariusz

1 Like
8 
cmd = '''
         MATCH (dest:{destTag}{{icId:"{destId}"}})
         UNWIND $arrSource AS row
         MATCH (source:{srcType}{{icId:row.id}})
         WHERE NOT EXISTS((dest)-[:LINKED_TO]->(source))
         CALL apoc.create.relationship(source, "{k}", {{relation:"LINKED_TO"}}, dest)
         YIELD rel
         RETURN rel
     '''.format(destId=dest[0]["icId"], destTag=destType, srcType=sourceType, k=key)
    c = tx.run(cmd, arrSource=data)
9

That make no sense at all :- )

So you do WHERE NOT EXISTS((dest)-[:LINKED_TO]->(source))
And next statement is being executed if True, what statement happen if it exist? How would the else statement look like?

I'm going over as many if/else infos as I can but as far as I can tell its this language is "precious"...

10

But if you will always get the same relation type, for example LINKED_TO, you can use MERGE or CREATE UNIQUE to create the relation:)

11

Yeh I was going to go with unique once I understand the if/else/case/where/when syntaxes...

12

You don't need ifelse, you just want to create the relation when it does not exist and it's what is doing the cypher request.

13

On this doc, the two examples are pretty clear I think:) There are 2 syntaxes for CASE:
https://neo4j.com/docs/cypher-refcard/current/

14

Nope not getting the CASE example to work at all.

MATCH (ct:Pets{icId:"800a1031-1102-49c4-abdd-f294ae867821"})
MATCH (gt:Objects{name:"geo_98608"})
CASE
WHEN NOT EXISTS((gt)-[:LINKED_TO]->(ct)) THEN ct
WHEN EXISTS ((gt)-[:LINKED_TO]->(ct)) THEN gt
ELSE THEN ct, gt
END

just gives me

Invalid input 'S': expected 'l/L' (line 3, column 3 (offset: 101))
"CASE"
   ^

:/

15

You don't need CASE, this request is good:

cmd = '''
         MATCH (dest:{destTag}{{icId:"{destId}"}})
         UNWIND $arrSource AS row
         MATCH (source:{srcType}{{icId:row.id}})
         WHERE NOT EXISTS((dest)-[:LINKED_TO]->(source))
         CALL apoc.create.relationship(source, "{k}", {{relation:"LINKED_TO"}}, dest)
         YIELD rel
         RETURN rel
     '''.format(destId=dest[0]["icId"], destTag=destType, srcType=sourceType, k=key)
    c = tx.run(cmd, arrSource=data)
16

Yes I know, I'm just trying to understand how to use CASE at this point. To build more complex logic in future. :- )

17 
MATCH (ct:Pets{icId:"800a1031-1102-49c4-abdd-f294ae867821"})
MATCH (gt:Objects{name:"geo_98608"})
RETURN
CASE
WHEN NOT EXISTS((gt)-[:LINKED_TO]->(ct))
THEN ct
ELSE ct, gt
END,
CASE WHEN EXISTS ((gt)-[:LINKED_TO]->(ct))
THEN gt
ELSE ct, gt
END
18

So I can wrap this around in my head...

MATCH (ct:Pets{icId:"800a1031-1102-49c4-abdd-f294ae867821"}) << variable allocator, allocate ct
MATCH (gt:Objects{name:"geo_98608"}) < variable allocator, allocate gt
RETURN { < return objects from function below
    CASE < if statement begin
        WHEN NOT EXISTS((gt)-[:LINKED_TO]->(ct)) < case 0
            THEN ct  < returns ct from this function meaning the "RETURN ct" above returns ct then
        ELSE ct, gt, < direct return to above, "RETURN ct, gt"
        , < why comma? what does it do ?
        "CASE WHEN EXISTS ((gt)-[:LINKED_TO]->(ct)) could be written as >"
        CASE < case 1
            WHEN EXISTS ((gt)-[:LINKED_TO]->(ct)) - check statement, case 1?
                THEN gt same return idea as case 0
            ELSE ct, gt same return idea as case 0
    END - end case? Which case ? we made 2 cases  don't we need 2x END ?

This is how I see it, but I don't get it. We made 2 cases, with 2 if else statements. Are they additive? If both cases are true, and case 0 return ct and case 1 return gt, does the return is then ct, gt?

As to the code above, I get this >

Invalid input ',': expected an identifier character, whitespace, '{', node labels, a property map, a relationship pattern, '.', '(', '[', '^', '*', '/', '%', '+', '-', "=~", IN, STARTS, ENDS, CONTAINS, IS, '=', '~', "<>", "!=", '<', '>', "<=", ">=", AND, XOR, OR or END (line 7, column 8 (offset: 164))
"ELSE ct, gt"

Apology for "silly" questions, but I'm just lost with that language so hard... : / I come from c++/Python but this is just black magic :- )

21

You can see this like a switch case:)
You should try to understand the example on the doc first before to apply it to your code:)

CASE n.eyes
WHEN 'blue' THEN 1
WHEN 'brown' THEN 2
ELSE 3
END

CASE
WHEN n.eyes = 'blue' THEN 1
WHEN n.age < 40 THEN 2
ELSE 3
END
22

Hello

Apology for slow reply. But as I'm new user apparently I'm limited to how much I can post :- )

Right so I had a "fun" day of if else tests. In any case I ended up running some tests but I hit a bit of a snag...

While I tried to connect 100k nodes 1 in the loop above, even without the WHERE NOT EXISTS test It was unable to perform the task ...

I'm not quite sure how to bite it at this point. Could neo not be the tool for me ? I'm looking to have millions of nodes with relationships to each other that could be updated/changed at a 50-100k per user request.

Regards
Dariusz

23

Hello,

I think Neo4j is the tool you need but it requires time to learn how to use it correctly:)

You can ask your database to give you the nodes that have no relationships and after create a relation between them like this you don't need to check at the creation, that's anoter solution:)