Neogit: filesystem snapshots in Neo4j as a queryable graph

Projects & Collaboration
1

Sharing with the community a tool I wrote a few years ago: Neogit :rocket: !

What it's about ?

neogit takes content-addressed Merkle snapshots of a filesystem and stores the structure
(commits, trees, blobs and their edges) in Neo4j, while the bytes live in pluggable object
storage keyed by SHA-1. The result is filesystem history you can query in Cypher, and, more
importantly, enrich.

I think of it as a content-addressed temporal graph: Git's content-addressing and commit
DAG (the time axis), expressed as a graph so history is queryable instead of only walkable.

neogit-commit-demo

Why a graph, and not just Git?

Git's object graph is forward-only: commit → tree → blob. Answering anything that needs the
reverse direction "which commits contain this object?" requires walking all of history,
because there is no back-edge. Git has no query language; you get git log and pickaxe, not
arbitrary traversal.

Neo4j flips that: put the same content-addressed objects in a graph and the reverse question is a single Cypher traversal. Any relationship direction works, aggregation is first-class, and queries
compose.

The second reason is enrichment. The graph is open-ended, so you can attach your own nodes (registry keys, binary symbols, PE metadata) to the same commit graph. History queries then work identically for every enrichment type without extra tooling.

Where it's used ?

Neogit is one of the foundational library behind the OSWatcher project:

The main goal is to capture and follow OS evolution by extracting and mapping their shared characteristics.

Our goal was to be able to asnwer questions like

  • "On which Windows builds have we seen this registry key ?"
  • "Which Windows builds changed this kernel struct field over the years ?"

Demos

Advanced git log over Registry key:

git_log_reg

Advanced git log over EProcess struct field:

git_log_eprocess

Feedback

The server is built on neo4j-graphql library, and use a custom procedure to speedup the diff.

I'm planning to open-source those as well.

Contributions, feeback and ideas are welcome ! :clap:

Special thanks to @michael.hunger and @darrellwarde for your help building this project over the last few years ! :100: