Aura db import does not create nodes

Neo4j Graph Platform Import / Export
1

Hello,
I'm very new to Neo4j.
I loaded a set of *.csv files on an AuraDB free instance and create a very simple graph with two nodes and 1 relation.

I came into troubles at import time. Let's focus on my first issue.

For my First node, let's say Command, I have 4 different properties m_uid_0, m_uid_1, m_uid_2, m_uid_3 as keys. In my understanding, the "uniqueness of the key" means that the {m_uid_0, m_uid_1, m_uid_2, m_uid_3} combination is unique for each node.
When I look at the cypher code generated for import, I see the following:

UNWIND $nodeRecords AS nodeRecord
WITH *
WHERE NOT nodeRecord.`m_uid_0` IN $idsToSkip AND NOT toInteger(trim(nodeRecord.`m_uid_0`)) IS NULL AND NOT nodeRecord.`m_uid_1` IN $idsToSkip AND NOT toInteger(trim(nodeRecord.`m_uid_1`)) IS NULL AND NOT nodeRecord.`m_uid_2` IN $idsToSkip AND NOT toInteger(trim(nodeRecord.`m_uid_2`)) IS NULL AND NOT nodeRecord.`m_uid_3` IN $idsToSkip AND NOT toInteger(trim(nodeRecord.`m_uid_3`)) IS NULL
MERGE (n: `Command` { `m_uid_0`: toInteger(trim(nodeRecord.`m_uid_0`)), `m_uid_1`: toInteger(trim(nodeRecord.`m_1`)), `m_uid_2`: toInteger(trim(nodeRecord.`m_uid_2`)), `m_uid_3`: toInteger(trim(nodeRecord.`m_uid_3`)) })

It looks like the generated code expect that each properties is unique. This is not what I expected.

Could you help me to find a way to make the import tool understand that the uniqueness is on the properties combination and not on each proprerty alone?

Thanks a lot.

2

This is not a behaviour I'd expect to see for the Import tool, would you be able to share with us your configuration downloaded from the tool?

3

Hy @gregory_king .
Sure I can share with you the model exported from AURAdb before import.
Here it is:

{
  "version": "0.12.0-beta.0",
  "graph": {
    "nodes": [
      {
        "id": "n16",
        "position": {
          "x": -21.742633708099618,
          "y": 259.33612559318027
        },
        "caption": "Command"
      },
      {
        "id": "n19",
        "position": {
          "x": 225.55738608731693,
          "y": 259.33612559318027
        },
        "caption": "Result"
      }
    ],
    "relationships": [
      {
        "id": "n8",
        "type": "trigger",
        "fromId": "n16",
        "toId": "n19"
      }
    ]
  },
  "dataModel": {
    "configurations": {
      "idsToIgnore": []
    },
    "fileModel": {
      "fileSchemas": {
        "hCommandTopicview.csv": {
          "expanded": true,
          "fields": [
            {
              "name": "g_TimestampAsReal",
              "type": "float",
              "sample": "26.741283641",
              "include": true
            },
            {
              "name": "IsValid",
              "type": "boolean",
              "sample": "1",
              "include": true
            },
            {
              "name": "TopicName",
              "type": "string",
              "sample": "hCommandTopic",
              "include": true
            },
            {
              "name": "partition",
              "type": "string",
              "sample": "SN22261D/TXL_INTERNAL",
              "include": true
            },
            {
              "name": "sequenceNumber",
              "type": "integer",
              "sample": "1",
              "include": true
            },
            {
              "name": "transactionId",
              "type": "integer",
              "sample": "0",
              "include": true
            },
            {
              "name": "writer_SID",
              "type": "integer",
              "sample": "1801750009",
              "include": true
            },
            {
              "name": "writer_LID",
              "type": "integer",
              "sample": "1346",
              "include": true
            },
            {
              "name": "writer_serial",
              "type": "integer",
              "sample": "3",
              "include": true
            },
            {
              "name": "m_uid_m_systemId",
              "type": "integer",
              "sample": "1801750009",
              "include": true
            },
            {
              "name": "m_uid_m_localId",
              "type": "integer",
              "sample": "1346",
              "include": true
            },
            {
              "name": "m_uid_m_majorUid",
              "type": "integer",
              "sample": "26741",
              "include": true
            },
            {
              "name": "m_uid_m_minorUid",
              "type": "integer",
              "sample": "741148969",
              "include": true
            },
            {
              "name": "m_commandId",
              "type": "integer",
              "sample": "22",
              "include": true
            },
            {
              "name": "m_commandParams",
              "type": "string",
              "sample": "message.userData.m_commandParams:[0]=2$[1]=0$",
              "include": true
            }
          ]
        },
        "hResultTopicview.csv": {
          "expanded": true,
          "fields": [
            {
              "name": "g_TimestampAsReal",
              "type": "float",
              "sample": "29.196168379",
              "include": true
            },
            {
              "name": "IsValid",
              "type": "boolean",
              "sample": "1",
              "include": true
            },
            {
              "name": "TopicName",
              "type": "string",
              "sample": "hResultTopic",
              "include": true
            },
            {
              "name": "partition",
              "type": "string",
              "sample": "HOST/SN22261D/TXL_COMMAND",
              "include": true
            },
            {
              "name": "sequenceNumber",
              "type": "integer",
              "sample": "1",
              "include": true
            },
            {
              "name": "transactionId",
              "type": "integer",
              "sample": "0",
              "include": true
            },
            {
              "name": "writer_SID",
              "type": "integer",
              "sample": "1801750009",
              "include": true
            },
            {
              "name": "writer_LID",
              "type": "integer",
              "sample": "2142",
              "include": true
            },
            {
              "name": "writer_serial",
              "type": "integer",
              "sample": "6",
              "include": true
            },
            {
              "name": "m_componentId",
              "type": "string",
              "sample": "HardwareMonitor",
              "include": true
            },
            {
              "name": "m_requestUid_m_systemId",
              "type": "integer",
              "sample": "1801750009",
              "include": true
            },
            {
              "name": "m_requestUid_m_localId",
              "type": "integer",
              "sample": "1346",
              "include": true
            },
            {
              "name": "m_requestUid_m_majorUid",
              "type": "integer",
              "sample": "26741",
              "include": true
            },
            {
              "name": "m_requestUid_m_minorUid",
              "type": "integer",
              "sample": "741148969",
              "include": true
            },
            {
              "name": "m_requestStatus",
              "type": "boolean",
              "sample": "0",
              "include": true
            }
          ]
        }
      }
    },
    "graphModel": {
      "nodeSchemas": {
        "n16": {
          "label": "Command",
          "additionLabels": [],
          "labelProperties": [],
          "properties": [
            {
              "property": "g_TimestampAsReal",
              "type": "float",
              "identifier": "6RL1LQ1uQoLasK7M-dO6e"
            },
            {
              "property": "IsValid",
              "type": "boolean",
              "identifier": "HKZQ_3zrFhdxDZqPqk7Vr"
            },
            {
              "property": "TopicName",
              "type": "string",
              "identifier": "-T6-YER1uqoV890_08XMw"
            },
            {
              "property": "partition",
              "type": "string",
              "identifier": "bFwQqoc2lYYkYGLHGbp6u"
            },
            {
              "property": "sequenceNumber",
              "type": "integer",
              "identifier": "H0_OUBIl40sHFSUeXG_jN"
            },
            {
              "property": "transactionId",
              "type": "integer",
              "identifier": "Z77ixnp3iiZZ3PXwRi-kJ"
            },
            {
              "property": "writer_SID",
              "type": "integer",
              "identifier": "9Oa2FPkElKtFTzSFrgk4W"
            },
            {
              "property": "writer_LID",
              "type": "integer",
              "identifier": "x7qev7otEVMQdTVva-Yz5"
            },
            {
              "property": "writer_serial",
              "type": "integer",
              "identifier": "gBFIDIqo6smhpZDZHpsS7"
            },
            {
              "property": "m_uid_m_systemId",
              "type": "integer",
              "identifier": "6QXQVcwguo-V4fL4ZfhZi"
            },
            {
              "property": "m_uid_m_localId",
              "type": "integer",
              "identifier": "OG8Cso91NEDvE_YitxUGQ"
            },
            {
              "property": "m_uid_m_majorUid",
              "type": "integer",
              "identifier": "fypSbkMA3KwRf3DtFnJOy"
            },
            {
              "property": "m_uid_m_minorUid",
              "type": "integer",
              "identifier": "Hkjkgyy87JDBSySl9zPzg"
            },
            {
              "property": "m_commandId",
              "type": "integer",
              "identifier": "lQh6qJr5uIOn6iFIQnUuo"
            },
            {
              "property": "m_commandParams",
              "type": "string",
              "identifier": "nPNOHI2ewljNkYsBOSMiL"
            }
          ],
          "key": {
            "properties": [
              "6QXQVcwguo-V4fL4ZfhZi",
              "OG8Cso91NEDvE_YitxUGQ",
              "fypSbkMA3KwRf3DtFnJOy",
              "Hkjkgyy87JDBSySl9zPzg"
            ]
          }
        },
        "n19": {
          "label": "Result",
          "additionLabels": [],
          "labelProperties": [],
          "properties": [
            {
              "property": "g_TimestampAsReal",
              "type": "float",
              "identifier": "MevERjCbTXI-Wo_nPH4JN"
            },
            {
              "property": "IsValid",
              "type": "boolean",
              "identifier": "awk68Fiu3GQzF2jsD5QvM"
            },
            {
              "property": "TopicName",
              "type": "string",
              "identifier": "zAULQjzV081HwdacUi1dB"
            },
            {
              "property": "partition",
              "type": "string",
              "identifier": "CJkRZg_wxXM04XNwf3Jzi"
            },
            {
              "property": "sequenceNumber",
              "type": "integer",
              "identifier": "ra-przN8g5dWBSDdbSo_h"
            },
            {
              "property": "transactionId",
              "type": "integer",
              "identifier": "TMTPjbemOn6c9hNhBM1U-"
            },
            {
              "property": "writer_SID",
              "type": "integer",
              "identifier": "xTFCqTUHQo9I8UfOpz4Kv"
            },
            {
              "property": "writer_LID",
              "type": "integer",
              "identifier": "DQpKKPhtyvvq6HmxtXyth"
            },
            {
              "property": "writer_serial",
              "type": "integer",
              "identifier": "upX5g4jotDzWcmm5hilUB"
            },
            {
              "property": "m_componentId",
              "type": "string",
              "identifier": "ILCX10VSiITkDtGtOAbvG"
            },
            {
              "property": "m_requestUid_m_systemId",
              "type": "integer",
              "identifier": "W8RHbECsfPrbCW-H5k0nF"
            },
            {
              "property": "m_requestUid_m_localId",
              "type": "integer",
              "identifier": "jb1OA1sjy5nbKpytHcc8v"
            },
            {
              "property": "m_requestUid_m_majorUid",
              "type": "integer",
              "identifier": "nCYrmfsR4wbhw3jorvEHv"
            },
            {
              "property": "m_requestUid_m_minorUid",
              "type": "integer",
              "identifier": "pEBUWIg0WISCm9hTjfIXA"
            },
            {
              "property": "m_requestStatus",
              "type": "boolean",
              "identifier": "dSFzHiUl1IiM4bDlCQvNv"
            }
          ],
          "key": {
            "properties": [
              "W8RHbECsfPrbCW-H5k0nF",
              "jb1OA1sjy5nbKpytHcc8v",
              "nCYrmfsR4wbhw3jorvEHv",
              "pEBUWIg0WISCm9hTjfIXA"
            ]
          }
        }
      },
      "relationshipSchemas": {
        "n8": {
          "type": "trigger",
          "sourceNodeSchema": "n16",
          "targetNodeSchema": "n19",
          "properties": [
            {
              "property": "m_uid_m_systemId",
              "type": "integer",
              "identifier": "ZChENcp7Yoe2NRGo_UL0W"
            },
            {
              "property": "m_uid_m_localId",
              "type": "integer",
              "identifier": "SqeMn2xBQjh1oZiQpiUWB"
            },
            {
              "property": "m_uid_m_majorUid",
              "type": "integer",
              "identifier": "SxGJoH3sft2xqcUAo_3kl"
            },
            {
              "property": "m_uid_m_minorUid",
              "type": "integer",
              "identifier": "hx6lPVTC7oyb_HdmqXfhL"
            }
          ]
        }
      }
    },
    "mappingModel": {
      "nodeMappings": {
        "n16": {
          "nodeSchema": "n16",
          "fileSchema": "hCommandTopicview.csv",
          "mappings": [
            {
              "field": "g_TimestampAsReal"
            },
            {
              "field": "IsValid"
            },
            {
              "field": "TopicName"
            },
            {
              "field": "partition"
            },
            {
              "field": "sequenceNumber"
            },
            {
              "field": "transactionId"
            },
            {
              "field": "writer_SID"
            },
            {
              "field": "writer_LID"
            },
            {
              "field": "writer_serial"
            },
            {
              "field": "m_uid_m_systemId"
            },
            {
              "field": "m_uid_m_localId"
            },
            {
              "field": "m_uid_m_majorUid"
            },
            {
              "field": "m_uid_m_minorUid"
            },
            {
              "field": "m_commandId"
            },
            {
              "field": "m_commandParams"
            }
          ]
        },
        "n19": {
          "nodeSchema": "n19",
          "fileSchema": "hResultTopicview.csv",
          "mappings": [
            {
              "field": "g_TimestampAsReal"
            },
            {
              "field": "IsValid"
            },
            {
              "field": "TopicName"
            },
            {
              "field": "partition"
            },
            {
              "field": "sequenceNumber"
            },
            {
              "field": "transactionId"
            },
            {
              "field": "writer_SID"
            },
            {
              "field": "writer_LID"
            },
            {
              "field": "writer_serial"
            },
            {
              "field": "m_componentId"
            },
            {
              "field": "m_requestUid_m_systemId"
            },
            {
              "field": "m_requestUid_m_localId"
            },
            {
              "field": "m_requestUid_m_majorUid"
            },
            {
              "field": "m_requestUid_m_minorUid"
            },
            {
              "field": "m_requestStatus"
            }
          ]
        }
      },
      "relationshipMappings": {
        "n8": {
          "relationshipSchema": "n8",
          "mappings": [
            {
              "field": "m_uid_m_systemId"
            },
            {
              "field": "m_uid_m_localId"
            },
            {
              "field": "m_uid_m_majorUid"
            },
            {
              "field": "m_uid_m_minorUid"
            }
          ],
          "sourceMappings": [
            {
              "field": "m_uid_m_systemId"
            },
            {
              "field": "m_uid_m_localId"
            },
            {
              "field": "m_uid_m_majorUid"
            },
            {
              "field": "m_uid_m_minorUid"
            }
          ],
          "targetMappings": [
            {
              "field": "m_requestUid_m_systemId"
            },
            {
              "field": "m_requestUid_m_localId"
            },
            {
              "field": "m_requestUid_m_majorUid"
            },
            {
              "field": "m_requestUid_m_minorUid"
            }
          ],
          "fileSchema": "hCommandTopicview.csv"
        }
      }
    }
  }
}

As you can see, the properties names change a bit with regards to the given example ( intstead of m_uid_0, m_uid_1, m_uid_2, m_uid_3, you will get m_uid_m_systemId, m_uid_m_localId, m_uid_m_majorUid, m_uid_m_minorUid as IDs ).

When I launch import with this model and the associaated csv files, I get the following results:

neo4j
neo4j909×797 50.4 KB

Thanks a lot for your help.
Regards.

4

Hi @zozogrgr, thanks for sharing your config. Did you manually change the JSON configuration to have multiple keys for the ids? This part for example:

                    "key": {
                        "properties": [
                            "W8RHbECsfPrbCW-H5k0nF",
                            "jb1OA1sjy5nbKpytHcc8v",
                            "nCYrmfsR4wbhw3jorvEHv",
                            "pEBUWIg0WISCm9hTjfIXA"
                        ]
                    }

Although the json and cypher generation support multiple keys, the UI shouldn't have let you arrive at a configuration like this, so I'm curious how this happened :slight_smile:

5

Yes indeed. As the ui did not offert this feature, I édit the json manually. Did I do something wrong ? Best regards.

6

Hey,

The UI doesn't support compound IDs at the moment. Although there is partial support for compounds in the json configuration that you edited manually (and resulting Cypher generation), it does not extend to being able to map the relationships between nodes that use compound IDs. This means you'll be able to load nodes with compound IDs but not link them.

Your alternatives at the moment are to craft your own load statements using LOAD CSV, or if you prefer to retain use of the Data Importer UI, pre-process your file to add a new column that concatenates the discrete columns in a way that they can be used as a single ID.

Let me know if you need any more info. Thanks!