I am using
apoc.load.html() procedure to do scraping. So i have stored list of html links on
( u:Url) node with property
urls so that it will access the urls from
And It works fine for links with status 200 (that are available) the problem is when it got link/url that is removed /deleted the process terminate with error like this :
To solve it i tried to replace the url with new one (with working link) manually but this is not feasible.
So my question is, is there any
apoc procedure or
Cypher code that can allow the process to proceed even if some of the links are unavailable(404).
I am thinking of maybe something to catch the failed link and processed getting the data for the working link one.