APOC load json with exception

[OnlyXYL]

Question

I import data with apoc.load.json from local file. and something wrong with id ,like this:

2023-04-02 02:40:12.919+0000 ERROR [o.n.b.t.p.ProtocolHandshaker] Fatal error occurred during protocol handshaking: [id: 0x3e68e0da, L:/172.17.0.2:7687 - R:/123.118.74.36:3061]
java.lang.NullPointerException: null
        at io.netty.buffer.Unpooled.copiedBuffer(Unpooled.java:363) ~[netty-buffer-4.1.86.Final.jar:4.1.86.Final]
        at org.neo4j.bolt.transport.DiscoveryResponseHandler.channelRead(DiscoveryResponseHandler.java:60) ~[neo4j-bolt-4.4.17.jar:4.4.17]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.handlerRemoved(ByteToMessageDecoder.java:266) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:536) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) [netty-codec-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at org.neo4j.bolt.transport.pipeline.AuthenticationTimeoutHandler.channelRead(AuthenticationTimeoutHandler.java:58) [neo4j-bolt-4.4.17.jar:4.4.17]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) [netty-transport-classes-epoll-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) [netty-transport-classes-epoll-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) [netty-transport-classes-epoll-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.86.Final.jar:4.1.86.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.86.Final.jar:4.1.86.Final]
        at java.lang.Thread.run(Thread.java:829) [?:?]
2023-04-02 02:40:21.554+0000 INFO  [o.n.c.i.ExecutionEngine] [neo4j/695a545d] Discarded stale query from the query cache after 454 seconds. Reason: NodesWithLabelCardinality(Some(LabelId(4))) changed from 2000.0 to 8114.0, which is a divergence of 0.7535124476213951 which is greater than threshold 0.6727608334791857. Query id: 8604
2023-04-02 02:40:58.225+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=245, gcTime=0, gcCount=0}
2023-04-02 02:41:11.336+0000 WARN  [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=5071, gcTime=0, gcCount=0}

Version

   neo4j-java-driver:  5.4.0
   jdk: 17
   neo4j-server : 4.4.17

Cypher

call apoc.load.json("file:///medical_relation_rule.json") yield value 
match (source:实体:中药) 
match (target:实体:方剂) 
where value.rule 
CALL apoc.merge.relationship(source,value.type,{},{},target,{}) yield rel 
RETURN count(id(rel)) as relations

and I already have nodes called source and target. the file called medical_relation_rule.json is just the relation rules , Instances that match the relations rules to create relationships like this:

[
  {
    "source": "实体:中药",
    "target": "实体:方剂",
    "type": "HAS_PART",
    "rules": "source.药名 is not null and target.名称 is not null and (split(target.药材,';') contains source.药名  or widdo.list.intersects(split(source.别名,'、'),split(target.药材,';')))"
  }
]

and the function called widdo.list.intersects(list1,list2) is defined by me to know whether set 1 and set 2 have intersection

somebody help !

only_xyl

[glilienfield (Gary Lilienfield)]

I put your json in a file and was able to import it using neo4j desktop, so I don't believe it is the file nor apoc procedure..

The error looks like it is trying to network communications. Are you executing this via the java driver? If so, can you post the java code?

As a note, your query is referencing 'value.rule', while your data has property 'rules'. Finally, after changing it to 'where value.rules', I got an error 'complaining it did not know how to treat the string 'value.rules' as a boolean. You need to fix this. If you intent is to only proceed if the value.rule value is present, then change it to 'where value.rule is not null'

[OnlyXYL]

hi, @glilienfield

i used java-driver

code

there is my code

 public WisdomResult buildInstanceRelationsFromJson(Map<String, Object> params) {

        try {
            final String source = params.get("source").toString();

            final String target = params.get("target").toString();

            String relationRuleFile = "medical_relation_rule.json";

            /*
                注意：加载文件需要对neo4j进行配置。
                apoc.import.file.enabled=true
                dbms.directories.import=文件目录。容器方式启动的话，为容器内容的目录。需要挂载主机目录。apoc.load加载的文件需要放到对应的目录下
             */
      /*      final String cypher = String.format("""
                    call apoc.load.json("file:///%s") yield value\s
                    match (source:%s)\s
                    match (target:%s)\s
                    where value.rule\s
                    CALL apoc.merge.relationship(source,value.type,{},{},target,{}) yield rel\s
                    RETURN count(id(rel)) as relations
                    """, relationRuleFile, source, target);*/

/*            final String cypher = String.format("""
                    CALL apoc.periodic.iterate("
                    match (source:实体:中药)\s
                    match (target:实体:方剂)\s
                    where (split(target.`药材`,';') contains source.`药名` or widdo.list.intersects(split(source.`别名`,'、'),split(target.`药材`,';'))) return source,target"
                    ,"CALL apoc.merge.relationship(source,'HAS_PART',{},{},target,{})", {batchSize:500,parallel:true}) yield batches, total return {total:total}""", relationRuleFile, source, target);*/

            final String cypher = String.format("""
                    CALL apoc.periodic.iterate("
                    call apoc.load.json("file:///%s") yield values \\s
                    unwind values as value
                    match (source:%s)\s
                    match (target:%s)\s
                    where value.rule return source,target"
                    ,"CALL apoc.merge.relationship(source,'HAS_PART',{},{},target,{})", {batchSize:500,parallel:true}) yield batches, total return {total:total}""", relationRuleFile, source, target);


            final Result<List<Map<String, Value>>> result = neo4jHelper.write(cypher);

            return WisdomResultInterface.NEO4j.ALL.wrapper(result);
        } catch (Exception e) {
            throw new BaseException(IResultInterface.Neo4jResultEnum.FAIL, e.getMessage());
        }

medical_relation_rule.json

The content of the file called medical_relation_rule.json is

[
  {
    "source": "实体:中药",
    "target": "实体:方剂",
    "type": "HAS_PART",
    "rules": "source.药名 is not null and target.名称 is not null and (split(target.药材,';') contains source.药名  or widdo.list.intersects(split(source.别名,'、'),split(target.药材,';')))"
  }
]

Data

The source called 实体:中药 like this:

And the target called 实体:方剂 like this :

Thank you for your answer

only_xyl