apoc.load.jsonParams within apoc.periodic.commit for API retrieval

Neo4j Graph Platform Procedures & APOC
1

Using Desktop 1.4.1 with a 4.2.0 database...

I am trying to get paginated results from an API using apoc procedures.
I have seen the Zendesk example from @dana_canzano as well as some other threads but I just can't seem to get the right code to work. Below is my code using a temp node to store the "next url" to paginate to similar to the Zendesk example.

WITH {Authorization: 'key ' + apoc.static.get('api.key') } AS headers
CALL apoc.periodic.commit('
   **3** MATCH (tn:TempNode {id: 1})
    WITH tn, headers
    CALL apoc.load.jsonParams(tn.url, headers, null) 
    YIELD value as value
    WITH tn, headers, value, value.next as next_url
        FOREACH (result in value.results |
        MERGE (s:Set {id:result.set_num})
        SET s.url = result.set_url,
    **11**       s.img = result.set_img_url)
    WITH next_url, headers
    CASE WHEN next_url is NULL then 0 ELSE 1 END AS count
    FOREACH(_ in CASE WHEN count = 0 THEN [] ELSE[1] END | 
           MERGE (tn:TempNode {id:1})
           SET tn.url = next_url
           )
    RETURN count(*) '
    , null);

The logic in lines 3 thru 11 will work fine if i don't CALL apoc.periodic.commit. My headers also work fine. So something seems to happen when trying to reset the tn.url parameter to get the next url to begin the process again for the next 1000 sets. I have tried many iterations to get it to work. As of now, the code errors out with a " Query cannot conclude with CALL (must be RETURN or an update clause) " Yet I do not see any other RETURN lines in Zendesk or other examples. So, do I have a typo somewhere? Has the 4.2.0 update changed something? I have spent all day trying to get this to work without success. Any help would be appreciated by a fresh set of eyes. Thanks.

2

trying to get this to work with 4.2.3 and having issues

    WITH next_url, headers
    CASE WHEN next_url is NULL then 0 ELSE 1 END AS count

should be

    WITH next_url, headers, 
    CASE WHEN next_url is NULL then 0 ELSE 1 END AS count

note the trailing , after headers

further I did just to see if I could get it to progress changed

    RETURN count(*) '
    , null)

to

    RETURN count(*) '
    , null)
yield updates return count(*);

and then it complained that it didnt like with tn, headers from line 4 and that headers was not defined. I then tried to move line 1 to the 1st line of apoc.periodic.comit and then had further errors.

maybe the above will get you closer

yield updates return count(*);

3

Well, it is refreshing, I guess to see you struggle with it as well. I tried your suggestions and few more things....switching FOREACH to UNWIND, etc. but I still can't find the right mixture. Maybe it is a 4.* issue and I should stop for the time being.