I found I had to uninstall, then remove anything related to neo4j in the appdata folder on my pc. And re-install, selecting "only I can use this program" when asked.
How to completely and cleanly reinstall NEO4J? - #5 by elaine.rosenberg - Has a thread in a similar vein. My suggestion is once removing the application, via uninstall, seek out the remnenants of neo4j in the appdata folder under your profile folder.