KAFKA Connect, $lastCheck field does nothing

Archive
,
1

Hello.

I have the kafka connect source running with the following properties

3X_e_d_ed7f68283772951fb8ef0e82a6fed907895f8e42.png
3X_e_d_ed7f68283772951fb8ef0e82a6fed907895f8e42.png756×999 105 KB

the query is:

MATCH (n:Note) WHERE n.createdAt > $lastCheck RETURN id(n) as id, n.id AS nid, n.createdAt AS createdAt

This never publishes data to my topic. If I change the CYPHER to remove the $lastCheck, something like:

MATCH (n:Note) RETURN n.id as nid, n.createdAt as createdAt

it works fine.

Does anyone have experience getting the kafka connect source query to work using $lastCheck? I can't poll my data otherwise making this whole thing totally useless.

2

Hello @jonathan ,

Maybe you can try by changing the 'neo4j.streaming.properties: NOW' to

LAST_COMMITTED . because the $lastCheck of LAST_COMMITTED < $lastCheck of Now, that mean with 'NOW' you can't not found your createdAt > NOW($lastCheck) .

3

@jonathan Hi Jonathan, did you solve this issue? having the exact same problem, documentation is not very clear on how $lastCheck works, I also noticed that datetime is not converted to Timestamp type on the schema registry, it shows as a string. Thanks, Victor

4

Dates used with $lastCheck comparison have to be in epochMillis:

where datetime({datetime: s.ModifiedDate}).epochMillis > $lastCheck