Dear Angelo,
thank you very much for your suggestions. Yes, I made the mistake and deleted Neo4j directly without uninstaller. Then I proceeded to delete everything related to Neo4j in AppData, including the files you suggested + ".Neo4jDesktop".
Following your mail I checked again that these files were indeed deleted. I don't know what changed in the meantime, however I was now able to finish the new setup.
I will check again next time if the issue reoccurs.
Kind regards
Julian
PS: BTW, when I install Neo4jDesktop for a local user, and then try to update it when a new version is available, it asks me for admin credentials, and otherwise aborts the update, so that you have to delete and reinstall from scratch. This is what led me to the entire trouble at first. Do you have any idea on this?