Neo4j apoc.periodic.repeat doesn't work with streams.publish

Does anyone ever used Neo4j apoc.periodic.repeat function to publish message to kafka? Here is what I have now:

I have a working query to send message to my kafka topic:

CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")

However, when I used apoc repeat function:

CALL apoc.periodic.repeat('kafka-only',
    'CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")', 5)

Nothing happens..

I have checked the log/debug log and can't get any hint. Can anyone help me?

This is my configuration:

kafka.zookeeper.connect=localhost:2181
kafka.bootstrap.servers=localhost:9092
streams.procedures.enabled=true
streams.source.enabled=true
streams.source.schema.polling.interval=10000
dbms.security.procedures.unrestricted=apoc.*

Please provide the following information if you ran into a more serious issue:

  • neo4j version: neo4j-community-3.4.17
  • desktop version : MacOS 10.12.6
  • which plugins / extensions / procedures do you use : apoc-3.4.0.7-all.jar and neo4j-streams-3.4.4.jar
  • neo4j.log and debug.log : No exceptions and no logs
1 Like

Can you be more specific on nothing happens? You don't see the messages going to Kafka, or you don't get any results from APOC?

What happens if you call streams.publish() by itself without the APOC?

When I use APOC procedure then I don't see the messages getting publish to Kafka topic.
If i call streams.publish() without APOC in that case message is getting published in Kafka topic.

Could you try to reproduce with a 3.5.12 setup and appropriate more current apoc + streams versions?

1 Like

Yes, I tried to reproduce with 3.5.12 version and it has same issue.
streams.publish without apoc producer is publishing message in Kafka topic.

CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")

Below query with apoc job management procedure is not publishing message in Kafka topic.

CALL apoc.periodic.repeat('kafka-only',
    'CALL streams.publish("my-neo4j-topic", "Hello World from Neo4j!")', 5)

Thanks @kavitakjava please try with the following jar:

I look forward to your feedback.
Thanks a lot
Andrea

1 Like

Hi Andrea @conker84,

The jar which you provided worked with neo4j-community-3.5.13 and i can see messages published every 5 seconds in the Kafka topic !!

The jar from https://github.com/neo4j-contrib/neo4j-streams/releases/tag/3.5.4 did not worked.

Seems like you have some fixes in ur jar.
Can this be available on github?

Your fix worked! Thank you so much for your help!

Best,
Kavita

Yes I fixed the bug in that jar. I'm preparing a PR so it'll be available with the next release.
Thanks a lot for pointing this out!

1 Like

Awesome!
I'm glad that my use case helped to brought up this issue !

Will this fix available for previous releases e.g. 3.4.17 so that we might now have to upgrade the version in production?

2 Likes