Looking at the APOC documentation, it seems that apoc.algo.dijkstra is missing for 4.1 only.
I think the only way is to upgrade Neo4j to 4.2 or later or use the gds library.
4.1
https://neo4j.com/labs/apoc/4.1/overview/apoc.algo/
4.4
https://neo4j.com/labs/apoc/4.4/overview/apoc.algo/apoc.algo.dijkstra/