Using multilevel projection interfaces to load data including pagination

Drivers & Stacks Spring Data Neo4j & Neo4j-OGM
1

I have built quite a complex project with a lot of relations. When I started to feed data into the database I quickly came to the point, where I would run out of memory, because querying one entity would load a huge chunk of the database. So now I need to control which relations should be followed for different use cases.

I'm having problems understanding how to instantiate my objects from those interfaces. I'm using multilevel projection interfaces and paginated queries.
I tried multiple different things, but ran into different problems casting the query results to those objects that implemented the interfaces, or got the reduced objects, when I needed the complete objects etc.

I find the problem hard to describe and I can't post code examples because the project is too complex already.
Is there an example project somewhere of a "real life like" application?

2

There are several points to start with:
Loading all (what feels like or might be) the whole graph is what you tell SDN to do if you have bidirectional or self/same-type-referencing relationships. It will just follow each possible path and map the entities accordingly.

When using projections, you do not have to care about assigning a concrete type to them. They are proxy objects. From the documentation:

The query execution engine creates proxy instances of that interface at runtime for each element returned and forwards calls to the exposed methods to the target object.

You would need to find the right way between loading too much and creating projections for every use-case, if you have plenty of them.

One problem, I don't understand is: Why do you think, that you need to cast anything?
You would either do something in your repository like:

List<MyProjection> findByName(String name);

Or use the Neo4jTemplate:

List<MyProjection> projections = neo4jTemplate.find(Entity.class).as(MyProjection.class).all();
3

You're right, I don't need to cast - that was just one of the things I tried, because nothing else worked.

I want two projections, one without related objects (EventList) and one with the first level of relations (EventFull)

So one of the things I tried was your first example:

public interface EventRepository extends Neo4jRepository<Event, Long>, RepositorySortableFilterable<EventList> {

    Optional<EventFull> findEventFullById(Long id);

    @Query(value = "MATCH (evt:Event)" +
            "WHERE toLower(evt.title) CONTAINS toLower($searchTerm) " +
            "RETURN evt ORDER BY evt.fullDate ASC SKIP $skip LIMIT $limit",
            countQuery = "MATCH (evt:Event) " +
                    "WHERE toLower(evt.title) CONTAINS toLower($searchTerm) " +
                    "RETURN count(evt)")
    Page<EventList> searchPaginated(String searchTerm, Pageable page);

For getting the list, I do something like this:

List<EventList> = this.eventRepository.searchPaginated(searchTitle, PageRequest.of(query.getPage(), query.getLimit())).stream());

which works just fine.

But when I want to go to the details of one of the events, like this:

Optional<Event> result = eventRepository.findById(idParameter));
this.event = result.orElseThrow(() -> new NoSuchRecordException("Could not find event."));

I only get the event with one-way relations. All other relation-sets are empty.

The same is true if I use the interface EventFull:

@Node("EventFull")
public interface EventFull extends EventProperties {

    Set<PersonList> getInvolved();

    Set<LocationList> getLocations();

    Set<TagList> getTags();
    
}

Involved and locations are empty, tags are set.

By the way, I also tried this for the list projection:

neo4jTemplate.find(Event.class).as(EventList.class).all().stream().skip(query.getOffset()).limit(query.getPageSize())

This has extremely bad performance.

4

Ok, I solved the problem, I think! The bidirectional relations were not displayed, because in the entity classes of the related entities I had the list interface defined like this:

@Node
public class Location extends BaseNode implements LocationList 
{
...
}

I don't exactly know, why this influences how entities are loaded, but removing it solved the problem.

@Node
public class Location extends BaseNode
{
...
}

Now I have a bunch of new problems, because I have the related objects in Drag-and-Drop components, that have been working with the entities before (like "Location") that now have to work with the interfaces ("LocationList" or maybe "LocationFull"), which was, why I had "... implements LocationList" there in the first place IIRC. (I'm using Vaadin, not a seperate frontend framework).

5

After countless refactoring of the whole project, I decided to use neither DTOs nor Interfaces.
Maybe I just don't understand the concept of projections well enough, but this gives me more transparency about what happens and this finally made my project work again.

For flexibility the repository interfaces look like this:

    @Query("MATCH (person:Person) WHERE ID(person) = $id " +
            "OPTIONAL MATCH (person)-[tw:TAGGED_WITH]->(t:Tag) " +
            "RETURN collect(tw) as tags, collect(t) as tag, " +
            "person")
    <T> Optional<T> findById(Long id, Class<T> type);

Then there is a PersonService, that does the actual interaction with the rest of the code, and can do other things, like build queries for more complex searches:

    public Optional<Person> findById(Long id) {
        return repository.findById(id, Person.class);
    }

    public Stream<Person> searchPaginated(String searchFirstname, String searchLastname, Integer searchAge,
                                          PageRequest pageRequest, AccessRights accessRights) {
        Node evt = Cypher.node("Person").named("p");
        // Build the query based on which parameters are set
        return repository.findAll(statement).stream();
    }