How to remove connected components less than x nodes?

Neo4j Graph Platform Cypher
41

Thank you for the suggestion, I gonna try that.
Best regards !

42

Found that :

CALL gds.wcc.stream({
nodeProjection: "Library",
relationshipProjection: "DEPENDS_ON"
})
YIELD nodeId, componentId
RETURN componentId, collect(gds.util.asNode(nodeId).id) AS libraries
ORDER BY size(libraries) DESC;

would that be a good starting point ?

43

Yeah, good start! :slight_smile:

If you want to do everything in one time (maybe you have to change the nodeProjection and the relationship Projection). In my query, it will delete communities which have less than 6 nodes.

CALL gds.wcc.stream({
    nodeProjection: "Item",
    relationshipProjection: "BELONGS_TO"
})
YIELD nodeId, componentId
WITH componentId, collect(gds.util.asNode(nodeId).id) AS libraries
WITH size(libraries) AS size, libraries
WHERE size < 6
WITH apoc.coll.flatten(collect(libraries)) AS nodes_list
CALL apoc.periodic.iterate('
    MATCH (n)
    WHERE n.id IN $nodes_list
    RETURN n
    ', '
    DETACH DELETE n
    ', {batchSize:1000, params:{nodes_list:nodes_list}}) YIELD batch, operations
RETURN 1

If you want to do it in two times:

  • Save the size of the community in a property:
CALL gds.wcc.stream({
    nodeProjection: "Item",
    relationshipProjection: "BELONGS_TO"
})
YIELD nodeId, componentId
WITH componentId, collect(gds.util.asNode(nodeId).id) AS libraries
WITH size(libraries) AS size, libraries
WITH size, apoc.coll.flatten(collect(libraries)) AS nodes_list
CALL apoc.periodic.iterate('
    MATCH (n)
    WHERE n.id IN $nodes_list
    RETURN n
    ', '
    SET n.community_id = $community_id
    ', {batchSize:1000, params:{nodes_list:nodes_list, community_id:size}}) YIELD batch, operations
RETURN 1
  • Next, to delete, for example connected components that have less than 6 nodes:
CALL apoc.periodic.iterate('MATCH (n) WHERE n.community_id < $community_id RETURN n', 'DETACH DELETE n', {batchSize:1000, params:{community_id:6}})

Regards,
Cobra

44

Hi,

I tried to delete all the nodes to do another import with the following command :

match (a) -[r] -> () delete a, r

And after a while, I got this error message :

Neo.DatabaseError.Transaction.TransactionCommitFailed

Makes me think to a db settings issue (Maybe to root cause of the issue with the query no eonding ?)
My settings :
dbms.directories.import=import
dbms.security.auth_enabled=true
dbms.memory.heap.initial_size=512m
dbms.memory.heap.max_size=4G
dbms.memory.pagecache.size=2G
dbms.tx_state.memory_allocation=ON_HEAP
dbms.connector.bolt.enabled=true
dbms.connector.http.enabled=true
dbms.connector.https.enabled=false
dbms.security.procedures.unrestricted=apoc.*
dbms.jvm.additional=-XX:+UseG1GC
dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow
dbms.jvm.additional=-XX:+AlwaysPreTouch
dbms.jvm.additional=-XX:+UnlockExperimentalVMOptions
dbms.jvm.additional=-XX:+TrustFinalNonStaticFields
dbms.jvm.additional=-XX:+DisableExplicitGC
dbms.jvm.additional=-XX:MaxInlineLevel=15
dbms.jvm.additional=-Djdk.nio.maxCachedBufferSize=262144
dbms.jvm.additional=-Dio.netty.tryReflectionSetAccessible=true
dbms.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048
dbms.jvm.additional=-Djdk.tls.rejectClientInitiatedRenegotiation=true
dbms.jvm.additional=-XX:FlightRecorderOptions=stackdepth=256
dbms.jvm.additional=-XX:+UnlockDiagnosticVMOptions
dbms.jvm.additional=-XX:+DebugNonSafepoints
dbms.windows_service_name=neo4j

Seems ok to you ?

Have a great day !

45

Tried another time and got :

Neo.DatabaseError.Statement.ExecutionFailed
Java heap space

46

To delete everything in the database, you should use:

CALL apoc.periodic.iterate('MATCH (n) RETURN n', 'DETACH DELETE n', {batchSize:1000})
47

Thank you !
BTW, I increased dbms.memory.heap.max_size to 16G and the delete query have been executed

48

It's another way :slight_smile: but it's always better to use the query I gave you :slight_smile:

49

Hi Cobra,

The Query is sucesseful but it seems no nodes are deleted. The query terminates very fast too :

image
image1462×422 29.5 KB

Query used :


CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD nodeId, componentId
WITH componentId, collect(gds.util.asNode(nodeId).id) AS libraries
WITH size(libraries) AS size, libraries
WHERE size < 16
WITH apoc.coll.flatten(collect(libraries)) AS nodes_list
CALL apoc.periodic.iterate('
    MATCH (n)
    WHERE n.id IN $nodes_list
    RETURN n
    ', '
    DETACH DELETE n
    ', {batchSize:1000, params:{nodes_list:nodes_list}}) YIELD batch, operations
RETURN 1
50

Can you show me what is returned by:

CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD nodeId, componentId
WITH componentId, collect(gds.util.asNode(nodeId).id) AS libraries
WITH size(libraries) AS size, libraries
RETURN *
51

Sure. Here's the result :

image
image1102×424 13.7 KB

52

Can you show me your properties on the right please? And tell me the one which is unique please :slight_smile:

53

Sure. Here you go :

image
image1155×540 33.4 KB

Here's the headers of my csv file :
Entity:ID,description:LABEL

Entity and ID is the same data . I added a unique constrainte on "Entity" even if I guess it is done automatically because Entity is used as ID.

Best regards

54

What is returned by:

CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD nodeId, componentId
WITH componentId, collect(gds.util.asNode(nodeId).Entity) AS libraries
WITH size(libraries) AS size, libraries
RETURN *
55

Same result :

image
image802×392 19.7 KB

56

Try this:

CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD nodeId, componentId
WITH componentId, collect(gds.util.asNode(nodeId)) AS libraries
RETURN *
57

Same result, no data returned :-)

58

There is something weird...
And?

CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD nodeId, componentId
RETURN *
59

Same.
I tried also :

CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD nodeId
RETURN *

and

CALL gds.wcc.stream({
    nodeProjection: "Entity",
    relationshipProjection: "DEPENDS"
})
YIELD componentId
RETURN *

but same : no output

60

I have no idea...
You made it wotk before and now it's not working anymore...
Is GDS still installed?

Anyway, these queries in my message should solve the main issue but maybe you will have to change the property depending of the one which is unique :slight_smile:

Regards,
Maxime