# Data ingestion from scratch (via auto api) **URL:** \ **Category:** Neo4j Graph Platform\ **Tags:** cypher, import\ **Created:** [December 1, 2024, 2:14pm UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705 "2024-12-01T14:14:02Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![f1-outsourcing](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/f1-outsourcing/32/29443_2.png) [@f1-outsourcing](https://community.neo4j.com/u/f1-outsourcing)\ **Post date:** [December 1, 2024, 2:14pm UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/1 "2024-12-01T14:14:02Z")
I am doubting a bit what to do with populating a neo4j database from scratch. Mostly I have been focussed on how I could represent neo4j data visually. Most subjects here are about ingesting data from already existing sources and then it makes sense importing some csv file. However I don't have anything so I practically need to select text and paste it into the database. So my first thought would be to add some auto rest/crud api in front of the database. If I would be working with mariadb/postgres I would get some api like this[1] and see if I can integrate it with some client to automate input. It looks like this has been discussed earlier[3], and this sofa[4] looks interesting. I was just wondering if there are any new or better ways than suggested there? Basically I am going to ingest nodes and their relationships, and I need to have something that registers which user added them. (Don't have yet a db schema) The other issue I am doubting is that it would be easier for me to implement this in a regular rdbms and then export this data to neo4j, but disadvantage is then I can't view added data real time. So I was wondering if someone is doing something similar, adding data 'manually' and what they decided to implement. [1] > **[GitHub - mevdschee/php-crud-api: Single file PHP script that adds a REST API to a...](https://github.com/mevdschee/php-crud-api)** > > Single file PHP script that adds a REST API to a SQL database [2] > **[GitHub - dbohdan/automatic-api: A list of software that turns your database into a...](https://github.com/dbohdan/automatic-api?tab=readme-ov-file)** > > A list of software that turns your database into a REST/GraphQL API [3] > [@Automatic creation of REST-API endpoints based on GraphQL endpoint and introspection](https://community.neo4j.com/t/automatic-creation-of-rest-api-endpoints-based-on-graphql-endpoint-and-introspection/58759/2): > > The SOFA project you linked looks interesting. I haven't seen it used with the Neo4j GraphQL Library, but looking through the docs I don't see any reasons it shouldn't work. Did you try it? Curious if it worked out for you? Another thing to consider is the [Neo4j GraphQL Library's OGM package](https://neo4j.com/docs/graphql-manual/current/ogm/). While it won't autogenerate a REST API, it can be used with the same GraphQL type definitions as used for the GraphQL API to define your models. It then exposes a programmatic API that can be used when de… [4] > **[GitHub - Urigo/SOFA: The best way to create REST APIs - Generate...](https://github.com/Urigo/SOFA)** > > The best way to create REST APIs - Generate RESTful APIs from your GraphQL Server ---
**Author:** ![f1-outsourcing](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/f1-outsourcing/32/29443_2.png) [@f1-outsourcing](https://community.neo4j.com/u/f1-outsourcing)\ **Post date:** [March 20, 2025, 10:41am UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/2 "2025-03-20T10:41:42Z")
Better think about trying with SOFA api could be a waste of your time. Seems buggy. I hardly tested. > > > I am using this /api/docs > > curl -s -X POST 'http://localhost:4000/api/delete-pe…ople?name=p3' > {"nodesDeleted":6,"relationshipsDeleted":10} > > I was expecting node with name=p3 would be deleted > > > When clicking a bit through this ui I am getting this error. I was wondering if …my 'default' setup needs to be changed for this. > > "message": "Variable \\"$sort\\" got invalid value \[{}\] at \\"sort\[0\]\\"; Field \\"0\\" is not defined by type \\"UserSort\\"." > > \`\`\` > type Query { > databaseStatus: String > actors: \[Person\] > } > > schema { > query: Query > } > > type Person @node { > name: String > created: BigInt > } > > type User @node { > created: BigInt > username: String > enabled: Boolean > password: String > name: String > apikey: String > } > \`\`\` > **[post request with parameters? · graphql-hive/SOFA · Discussion #1711](https://github.com/graphql-hive/SOFA/discussions/1711)** > > This looks also buggy. This is comes from when enabling the swagger ui from the example. Not that I have worked that much with apis But what is with this stuff after ? POST reguest to https://xxxxx... > **[query example is not correct, it does not create the expect result ·...](https://github.com/graphql-hive/SOFA/discussions/1710)** > > This is actors is not working according to your example\[1\] why? Besides I don't really get the need to 'translate' Person to People. It will probably do more harm than good. type Query { actors: \[P... ---
**Author:** ![glilienfield](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/glilienfield/32/27534_2.png) [@glilienfield](https://community.neo4j.com/u/glilienfield)\ **Post date:** [March 21, 2025, 2:42am UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/3 "2025-03-21T02:42:43Z")
I don’t know your tech stack constraints, but you could build a REST api to perform CRUD operations and other specific updates/queries using spring boot and java api. ---
**Author:** ![f1-outsourcing](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/f1-outsourcing/32/29443_2.png) [@f1-outsourcing](https://community.neo4j.com/u/f1-outsourcing)\ **Post date:** [March 21, 2025, 9:57am UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/4 "2025-03-21T09:57:26Z")
Yes I maybe have to look into that. Currently I am operating mostly on the lower end of the market where eg java developers are charging me 16 hours for a downloaded stomp ws example and still don't know the difference between topics and queues and how to correctly implement queues ---
**Author:** ![joshcornejo](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/joshcornejo/32/34171_2.png) [@joshcornejo](https://community.neo4j.com/u/joshcornejo)\ **Post date:** [March 21, 2025, 10:16am UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/5 "2025-03-21T10:16:18Z")
I don't get what is your objective. If you are going to be doing ad-hoc ingestion, your pipeline could look like this: ```auto source -> python code doing ETL -> Neo4J ``` If you want to get updates regularly or have more dynamic types of ingestion, you will do something like this: ```auto source -> Extract component -> GraphQL API -> Neo4J ``` Where your GraphQL will take the shape of whatever "transform-\>load" you are doing. But so far in my experience, what I ended up doing? ```auto 1. source convert CSV or JSON 2. move to import folder 3. apoc.load.csv() 4. cleanse the nodes ``` And I have in my sight [https://debezium.io](https://debezium.io) as a source for CDC (change data capture) if required. ---
**Author:** ![f1-outsourcing](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/f1-outsourcing/32/29443_2.png) [@f1-outsourcing](https://community.neo4j.com/u/f1-outsourcing)\ **Post date:** [March 21, 2025, 10:42am UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/6 "2025-03-21T10:42:58Z")
I was thinking of updating a node or properties of a node via an api. In some cases there will be an property added. I want to keep the client as simple as possible so in the middle ware and/or in Neo4j plugin I will do some modifications. ---
**Author:** ![f1-outsourcing](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/f1-outsourcing/32/29443_2.png) [@f1-outsourcing](https://community.neo4j.com/u/f1-outsourcing)\ **Post date:** [March 25, 2025, 8:41pm UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/7 "2025-03-25T20:41:29Z")
I am currently testing a bit with spring-boot-starter-data-neo4j and spring-boot-starter-data-rest and switching from the movies database on bolt+s://demo.neo4jlabs.com:7687 to my own, and getting this error 2025-03-25T20:05:26.976Z WARN 2827 --- [nio-8090-exec-1] o.s.data.neo4j.cypher.unrecognized : Neo.ClientNotification.Statement.UnknownPropertyKeyWarning: The provided property key is not in the database MATCH (movie:`Movie`) RETURN movie{.released, .tagline, .title, **nodeLabels** : labels(movie), **elementId** : id(movie)} this is because properties are missing. But what I don't understand is, that this is quite common for nodes in any database. AI is telling me the solution is to start overriding queries, but I have the impression that there must be a more 'auto' solution than to start overriding queries? ---
**Author:** ![glilienfield](https://sea1.discourse-cdn.com/flex021/user_avatar/community.neo4j.com/glilienfield/32/27534_2.png) [@glilienfield](https://community.neo4j.com/u/glilienfield)\ **Post date:** [March 26, 2025, 12:49am UTC](https://community.neo4j.com/t/data-ingestion-from-scratch-via-auto-api/71705/8 "2025-03-26T00:49:13Z")
It’s just a warning, so no impact.