Spring Data Neo4j 7.1.2 - findByID and findAll() causing infinite loop/crash

Drivers & Stacks Spring Data Neo4j & Neo4j-OGM
,
1

Hi,

I've implemented a Spring Data Neo4jRepository. Please see Code snippet below:

@Repository
public interface BookRepository extends Neo4jRepository<Book, String> {

    List<Book> findAll();    

    Optional<Book> findOneByTitle(String title);

    boolean existsByTitle(String title);

    @Query("MATCH (book:Book) WHERE book.title = $title RETURN book")
    Book findABookManual(@Param("title") String title);


import lombok.*;
import org.springframework.data.neo4j.core.schema.Id;
import org.springframework.data.neo4j.core.schema.Node;
import org.springframework.data.neo4j.core.schema.Property;
import org.springframework.data.neo4j.core.schema.Relationship;

import java.util.*;

import static org.springframework.data.neo4j.core.schema.Relationship.Direction.INCOMING;
import static org.springframework.data.neo4j.core.schema.Relationship.Direction.OUTGOING;


@Data
@NoArgsConstructor
@ToString
@Node("Book")

public class Book {

    @Id
    private String title;
    @Property("characters")
    private List<String>characters;
    @Property("genres")
    private List<String> genres;
    @Property("description")
    private String description;
    @Property("bookFormat")
    private String bookFormat;
    @Property("loan")
    private boolean loan;

    @ToString.Include
    @Relationship(type = "published", direction = INCOMING)
    private Set<Publisher> publishers = Collections.emptySet();

    @ToString.Include
    @Relationship(type = "hasFormat", direction = OUTGOING)
    private Set<BookFormat> bookFormats = Collections.emptySet();

    @ToString.Include
    @Relationship(type = "writtenBy", direction = OUTGOING)
    private Set<Author> authors = Collections.emptySet();

    @ToString.Include
    @Relationship(type = "publishedIn", direction = OUTGOING)
    private Set<PublishDate> publishDates = Collections.emptySet();

    @ToString.Include
    @Relationship(type = "hasBorrowed", direction = INCOMING)
    private Set<LibraryUser> Borrowers = Collections.emptySet();



    public Book (String title, List<String> characters, List<String> genres, String description, String bookFormat, boolean loan) {
        this.title = title;
        this.characters = characters;
        this.genres = genres;
        this.description = description;
        this.bookFormat = bookFormat;
        this.loan = loan;

    }

When I call the findByTitle() or the findAll() method, the app just goes into a loop until it crashes because the database (Aura Basic) refuses the connection. When I look at the logs, it seems to check for every relationship in the whole database (ca. 75k nodes and 130k relationships). The manual cypher query works perfect.
Adding a low depth number to the queries does not help. I don't really need the findAll() method, but if I could get the findByTitle() working it would be nice.

Thanks a lot ;)

2

Do any of the Publisher, Author, or LibraryUser entities have relationships to books that can then reference back to the book you are trying to find?

3

I don't think so. Apart from a second Relationship from User to Book ("currentlyBorrowed" - I forgot to put it into the class there - that will be deleted by a query when the Book is returned) all relationships are listed in the class above.

Book --> PublishDate
Book --> Author

LibraryUser --> Book
LibraryUser --> Book

Publisher --> Book

These are all relationships that exist in the database

4

Would it make sense to make all relationships as outgoing from a book node?

5

The directions should have no affect.

Your model shows LibraryUser has a relationship to books. This means books are related to books two levels deep. It would then be possible for a book related to itself.

Can you try commenting out the book attributes in the each of the related entities to see if the query works?

6

Ok, the problem here seemed to be that I established the opposite relationships on every node.
So

    @ToString.Include
    @Relationship(type = "hasBorrowed", direction = INCOMING)
    private Set<LibraryUser> Borrowers;

in the Book class

and

@ToString.Include
@Relationship(type = "hasBorrowed", direction = OUTGOING)
private Set<Book> borrowedBooks;

in the LibraryUser class

wont work. As soon as there is one of these constructs active it will go into an infinite loop (the same goes for all other nodes/relationships, so it not just affects the User/Book combo). So I have to decide where to put the relationship declarations, or is there another possibility (of course it would be nice for example to access the borrowed books from "both sides", so from the user side and the book side (so it would be nice to know which users borrowed a specific book, but also which books a specific user borrowed)
If I comment out all opposite relations also adding relations by simple adding Node classes to the sets works now. No cypher query required, although it's slower then using a short custom query.

7

Your Book entity is related to Publishers, Authors, and PublishDates. Each of these is related to Books. As such, you could have a book related to an author related to a book, which could be the same book you started with. Can you try commenting out these attributes to see if you get a response back.

8

The error occurs as soon as there is any relationship defined in the Target node and the starting node. No matter what sort of node it is. Or why "attributes" do you mean?

9

Comment out the publisher property

//@ToString.Include
    //@Relationship(type = "published", direction = INCOMING)
    //private Set<Publisher> publishers = Collections.emptySet();

And the other ones that are related to books. Then try using the find methods. This doesn’t solve the problem but identifies it.

Btw, how did you get your initial data in the database that you are testing with?

10

It's a list of Books and so on from GoodReads. I used the Import tool of neo4j aura

11

The problem are really the double declarations. Even if I keep only one relation and comment out all the other stuff it won't work. I tried to comment out all relations apart from the Book -> Author. As soon as I declare a reference to this in the Author Class and the Book Class (outgoing and incoming), it won't work. Only when I declare it in one of those it will work. Perhaps I will get rid of a few relationships that are not really needed for recommending Book nodes

12

That makes sense. Your author in your book has a reference to books itself. Given a specific book, the book has a reference to all its authors. Each of those authors has a reference to all books they wrote. In every case each author would have a reference to the given book we started with. This means you have a loop in your data, which seems to explain your symptoms.

I don’t think you can use SDN for your use case. Remember, when you save a Book it must have all its descendant nodes otherwise SDN will remove any relationships to nodes not included in your entity you are saving.

How did you create your data in the first place that you are testing?

13

I used the import tool on neo4j AuraDB and imported a CSV file there.
If you look up the official documentation/tutorials of SDN, all the examples are with Movies/Actors (and should therefore have the same problem, as a Movie can have many actors and all actors are again connected to all movies in which they acted), so in my opinion quite a similar dataset as mine... Therefore I was quite confident that I can do it with that tools.

14

In their example, the movie has a relationship to actors. The actors don’t have relationships to movies. So you can read an actor and get all their movies. The query stops there.

IMG_4068
IMG_40681170×2532 391 KB

IMG_4069
IMG_40691170×2532 321 KB

https://docs.spring.io/spring-data/neo4j/docs/current/reference/html/#projections

15

Hello! You can define relationships in both places. The key is to set an annotation on the property to ignore certain fields when it traverses back to the other entity, so that you don't end up in an infinite loop.

I have an example of that in my repository here: https://github.com/JMHReif/neo4j-code-wars/blob/master/src/main/java/com/jmhreif/neo4jcodewars/OrderedProduct.java

16

Hi, thanks a lot for your answer. My Relationships are without properties. When I try to do a relationship property, it goes into an infinite loop again (the same as when I try to declare on both sides). So if I understand it correctly, the json ignore annotation does the trick? I will try that.

17

ok, this seems to work at least in one direction. If i create a Relationship Author -> Book, I declare Book as a target node. I'm therefore able to get the Book name out of the relationships (for example which books an author has written). Is it possible to go the other way round? the relationship properties class only has the target node stored. Is it possible to declare the starting one was well? The relationship is declared in both node classes with INCOMING and OUTGOING, and it is a Set of the "RelationshipProperties" class.

18

You don't have to have relationship properties to use this. And, yes, you can use this in either or both directions! So you would have something like the following code:

public class Book {
    //fields for id, characters, genres, etc

    @JsonIgnoreProperties({"books"})
    @Relationship(type = "writtenBy", direction = OUTGOING)
    private Set<Author> authors;
}

public class Author {
    //fields for this class

    @JsonIgnoreProperties({"authors"})
    @Relationship(type = "writtenBy", direction = INCOMING)
    private Set<Book> books;
}

And then you can use the same syntax with the other connected classes, as well. Also, you don't need to use the @Property annotation unless your field name in the Java class is different from the property name in the database. Hope this helps!

19

I did this on pair of nodes and instantly gives me a stack overflow error :frowning:
Did it exactly the same way as described by you

20

Could you send the text of the error? Also, is it possible to send me our project (Github, zip, etc)? I can take a look and try to figure out what's missing.