Error in neo4j setup in kubernetes using helm charts

Neo4j Graph Platform Installation
1

Hi All,

I am trying to setup neo4j in kubernetes using the helm charts and below is the values.yaml file I have used and modified few things like installing plugin and adding extra vars,

# Default values for Neo4j.
# This is a YAML-formatted file.
# Declare name/value pairs to be passed into your templates.
# name: value

name: "neo4j"

# Specs for the Neo4j docker image
image: "neo4j"
imageTag: "3.5.7"
imagePullPolicy: "IfNotPresent"
# imagePullSecret: registry-secret
acceptLicenseAgreement: "no"
podDisruptionBudget: {}
  # minAvailable: 2
  # maxUnavailable: 1

# Use password authentication
authEnabled: true

## Specify password for neo4j user
## Defaults to a random 10-character alphanumeric string if not set and authEnabled is true
neo4jPassword: changeme

# Specify cluster domain (used eg. as suffix in definition of NEO4J_causal__clustering_initial__discovery__members environment variable)
clusterDomain: "cluster.local"

# Specs for the images used for running tests against the Helm package
testImage: "markhneedham/k8s-kubectl"
testImageTag: "master"

# Cores
core:
  numberOfServers: 3
  persistentVolume:
    ## whether or not persistence is enabled
    ##
    enabled: true

    ## core server data Persistent Volume mount root path
    ##
    mountPath: /data

    ## core server data Persistent Volume size
    ##
    size: 10Gi

    ## core server data Persistent Volume Storage Class
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    ## storageClass: "-"

    ## Subdirectory of core server data Persistent Volume to mount
    ## Useful if the volume's root directory is not empty
    ##
    ## subPath: ""

  ## Pass extra environment variables to the Neo4j container.
  ##
  extraVars:
   - name: com.graphaware.module.ES.uri
     value: elasticsearch
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j core pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers: []
  ## init containers to run before the Neo4j core pod e.g. to install plugins

   - name: init-plugins
     image: "appropriate/curl:latest"
     imagePullPolicy: "IfNotPresent"
     volumeMounts:
     - name: plugins
       mountPath: /plugins
     command:
       - "/bin/sh"
       - "-c"
       - |
         curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.2.0.3/apoc-3.2.0.3-all.jar -O
         cp apoc-3.2.0.3-all.jar /plugins/

# Read Replicas
readReplica:
  numberOfServers: 0
  ## Pass extra environment variables to the Neo4j container.
  ##
  # extraVars:
  # - name: EXTRA_VAR_1
  #   value: extra-var-value-1
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j replica pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers: []
  ## init containers to run before the Neo4j replica pod e.g. to install plugins

  # - name: init-plugins
  #   image: "appropriate/curl:latest"
  #   imagePullPolicy: "IfNotPresent"
  #   volumeMounts:
  #   - name: plugins
  #     mountPath: /plugins
  #   command:
  #     - "/bin/sh"
  #     - "-c"
  #     - |
  #       curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.2.0.3/apoc-3.2.0.3-all.jar -O
  #       cp apoc-3.2.0.3-all.jar /plugins/

resources: {}
# limits:
#   cpu: 100m
#   memory: 512Mi
# requests:
#   cpu: 100m
#   memory: 512Mi

Below is the command i execute and getting the response,

ubuntu@:~/KubernetesCluster/elasticsearch7/neo4j$ helm install --name neo4j-community stable/neo4j -f ./test.yaml
Error: failed to parse ./test.yaml: error converting YAML to JSON: yaml: line 76: did not find expected key

I followed the default values.yaml and modified the changes, I suspect the indentation is not properly given in yaml and the error line 76 where I defined the command for install plugin in neo4j and I did tried changing the indentation inside yaml but didn't worked.

Also inside the values.yaml I defined the extra environment variable for neo4j container and these values are not passed to neo4j conf

  ## Pass extra environment variables to the Neo4j container.
  ##
  extraVars:
   - name: com.graphaware.module.ES.uri
     value: elasticsearch

Please correct me if I am doing anything wrong in the values.yaml

Regards,
Ganeshbabu R

2

It would help if you posted your full YAML and identified exactly which line is line 76. Indeed it does sound like you have a formatting issue.

It sounds like you've made more than just these changes to the kubernetes yaml because you're installing neo4j-community, and applying "test.yaml" - which is fine - but it's hard to spot what's going wrong without the full YAML.

Additionally as a debugging step I'd suggest not using "extraVars" to set environment variables, but instead to set them directly in the pod, like here:

This would help eliminate as a source of an error helm incorrectly substituting those vars into a separate template.

3

@david_allen

Below is my complete test.yaml file

# Default values for Neo4j.
# This is a YAML-formatted file.
# Declare name/value pairs to be passed into your templates.
# name: value

name: "neo4j"

# Specs for the Neo4j docker image
image: "neo4j"
imageTag: "3.5.7"
imagePullPolicy: "IfNotPresent"
# imagePullSecret: registry-secret
acceptLicenseAgreement: "no"
podDisruptionBudget: {}
  # minAvailable: 2
  # maxUnavailable: 1

# Use password authentication
authEnabled: true

## Specify password for neo4j user
## Defaults to a random 10-character alphanumeric string if not set and authEnabled is true
neo4jPassword: changeme

# Specify cluster domain (used eg. as suffix in definition of NEO4J_causal__clustering_initial__discovery__members environment variable)
clusterDomain: "cluster.local"

# Specs for the images used for running tests against the Helm package
testImage: "markhneedham/k8s-kubectl"
testImageTag: "master"

# Cores
core:
  numberOfServers: 3
  persistentVolume:
    ## whether or not persistence is enabled
    ##
    enabled: true

    ## core server data Persistent Volume mount root path
    ##
    mountPath: /data

    ## core server data Persistent Volume size
    ##
    size: 10Gi

    ## core server data Persistent Volume Storage Class
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    ## storageClass: "-"

    ## Subdirectory of core server data Persistent Volume to mount
    ## Useful if the volume's root directory is not empty
    ##
    ## subPath: ""

  ## Pass extra environment variables to the Neo4j container.
  ##
  extraVars:
   - name: com.graphaware.module.ES.uri
     value: elasticsearch
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j core pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers: []
  ## init containers to run before the Neo4j core pod e.g. to install plugins

   - name: init-plugins
     image: "appropriate/curl:latest"
     imagePullPolicy: "IfNotPresent"
     volumeMounts:
     - name: plugins
       mountPath: /plugins
     command:
       - "/bin/sh"
       - "-c"
       - |
         curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.2.0.3/apoc-3.2.0.3-all.jar -O
         cp apoc-3.2.0.3-all.jar /plugins/

# Read Replicas
readReplica:
  numberOfServers: 0
  ## Pass extra environment variables to the Neo4j container.
  ##
  # extraVars:
  # - name: EXTRA_VAR_1
  #   value: extra-var-value-1
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j replica pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers: []
  ## init containers to run before the Neo4j replica pod e.g. to install plugins

  # - name: init-plugins
  #   image: "appropriate/curl:latest"
  #   imagePullPolicy: "IfNotPresent"
  #   volumeMounts:
  #   - name: plugins
  #     mountPath: /plugins
  #   command:
  #     - "/bin/sh"
  #     - "-c"
  #     - |
  #       curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.2.0.3/apoc-3.2.0.3-all.jar -O
  #       cp apoc-3.2.0.3-all.jar /plugins/

resources: {}
# limits:
#   cpu: 100m
#   memory: 512Mi
# requests:
#   cpu: 100m
#   memory: 512Mi

Yes right I tried its worked but i wanted to test "extraVars" to pass the paramaters to neo4j conf but for me it didn't worked..

Please let me know your thoughts..

Regards,
Ganeshbabu R

4

Sadly this template needs to be fairly re-organized. What it is saying is that you wish to create 3 independent copies of neo4j community, which is not a sensible thing to do. Community does not support clustering.

I think what you're trying to do here is to deploy a SINGLE community container in kubernetes. You're better off reading the docker documentation for Neo4j and writing your own single YAML file from scratch, because the way the cluster config is being adapted for community is causing you more issues than it's solving here.

From the original cluster YAML, you've removed the StatefulSet declaration, but you didn't declare this as a separate kubernetes resource spec.

So I'm sorry as I know this is a high-effort suggestion, but I recommend you start from square 1 with an eye towards what you're trying to deploy.

1 Like
5

@david_allen
We tried changing the yaml and able to run the neo4j with helm command and below is the full yaml,

Also we are able to add parameter in extraVars to pass it to the neo4j conf and corrected the yaml formatting for install plugin in neo4j

# Default values for Neo4j.
# This is a YAML-formatted file.
# Declare name/value pairs to be passed into your templates.
# name: value

name: "neo4j"

# Specs for the Neo4j docker image
image: "neo4j"
imageTag: "3.5.7"
imagePullPolicy: "IfNotPresent"
# imagePullSecret: registry-secret
acceptLicenseAgreement: "yes"
podDisruptionBudget: {}
  # minAvailable: 2
  # maxUnavailable: 1

# Use password authentication
#authEnabled: true

## Specify password for neo4j user
## Defaults to a random 10-character alphanumeric string if not set and authEnabled is true
#neo4jPassword: ""

# Specify cluster domain (used eg. as suffix in definition of NEO4J_causal__clustering_initial__discovery__members environment variable)
clusterDomain: "cluster.local"

# Specs for the images used for running tests against the Helm package
testImage: "markhneedham/k8s-kubectl"
testImageTag: "master"

# Cores
core:
  numberOfServers: 1
  persistentVolume:
    ## whether or not persistence is enabled
    ##
    enabled: true

    ## core server data Persistent Volume mount root path
    ##
    mountPath: /data

    ## core server data Persistent Volume size
    ##
    size: 50Gi

    ## core server data Persistent Volume Storage Class
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    #storageClass: ""

    ## Subdirectory of core server data Persistent Volume to mount
    ## Useful if the volume's root directory is not empty
    ##
    ## subPath: ""

  ## Pass extra environment variables to the Neo4j container.
  ##
  extraVars:
  - name: NEO4J_dbms_unmanaged__extension__classes
    value: com.graphaware.server=/graphaware
  - name: NEO4J_com_graphaware_runtime_enabled
    value: "true"
  - name: NEO4J_com_graphaware_module_ES_2
    value: com.graphaware.module.es.ElasticSearchModuleBootstrapper
  - name: NEO4J_com_graphaware_module_ES_uri
    value: elasticsearch
  - name: NEO4J_com_graphaware_module_ES_port
    value: "9200"
  - name: NEO4J_com_graphaware_module_ES_protocol
    value: http
  - name: NEO4J_com_graphaware_module_ES_index
    value: graph
  - name: NEO4J_com_graphaware_module_ES_keyProperty
    value: uuid
  - name: NEO4J_com_graphaware_module_ES_retryOnError
    value: "false"
  - name: NEO4J_com_graphaware_module_ES_queueSize
    value: "10000"
  - name: NEO4J_com_graphaware_module_ES_reindexBatchSize
    value: "2000"
  - name: NEO4J_com_graphaware_module_ES_relationship
    value: (true)
  - name: NEO4J_com_graphaware_module_ES_bulk
    value: "false"
  - name: NEO4J_com_graphaware_module_ES_initializeUntil
    value: "2222222222222"
  - name: NEO4J_com_graphaware_module_ES_mapping
    value: com.graphaware.module.es.mapping.JsonFileMapping
  - name: NEO4J_com_graphaware_module_ES_file
    value: ../../../plugins/mapping.json
  # - name: EXTRA_VAR_1
  #   value: extra-var-value-1
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j core pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers:
  ## init containers to run before the Neo4j core pod e.g. to install plugins

  - name: init-plugins
    image: "appropriate/curl:latest"
    imagePullPolicy: "IfNotPresent"
    volumeMounts:
    - name: plugins
      mountPath: /plugins
    command:
      - "/bin/sh"
      - "-c"
      - |
        curl -L https://products.graphaware.com/download/framework-server-community/graphaware-server-community-all-3.5.4.53.jar -O
        curl -L https://products.graphaware.com/download/neo4j-to-elasticsearch/graphaware-neo4j-to-elasticsearch-3.5.4.53.11.jar -O
        cp graphaware*.jar /plugins/
        cat <<'EOF' > mapping.json
        {
          "defaults": {
            "nodes_index": "graph-node",
            "relationships_index": "graph-relationship",
            "key_property": "uuid",
            "include_remaining_properties": true
          },
          "node_mappings": [
            {
              "condition": "allNodes()",
              "type": "nodes",
              "properties": {
                "labels": "getLabels()"
              }
            }
          ],
          "relationship_mappings": [
            {
              "condition": "isType('installed')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (n) WHERE id(n) = {id} MATCH (s)-[i:installed]->(h) RETURN collect({name: type(i), start_node: labels(s), end_node: labels(h)}) AS value')"
               }
            }
          ]
        }
        EOF
        cp mapping.json /plugins/
# Read Replicas
readReplica:
  numberOfServers: 0
  ## Pass extra environment variables to the Neo4j container.
  ##
  # extraVars:
  # - name: EXTRA_VAR_1
  #   value: extra-var-value-1
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j replica pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers: []
  ## init containers to run before the Neo4j replica pod e.g. to install plugins

  # - name: init-plugins
  #   image: "appropriate/curl:latest"
  #   imagePullPolicy: "IfNotPresent"
  #   volumeMounts:
  #   - name: plugins
  #     mountPath: /plugins
  #   command:
  #     - "/bin/sh"
  #     - "-c"
  #     - |
  #       curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.2.0.3/apoc-3.2.0.3-all.jar -O
  #       cp apoc-3.2.0.3-all.jar /plugins/

resources: {}
# limits:
#   cpu: 100m
#   memory: 512Mi
# requests:
#   cpu: 100m
#   memory: 512Mi

Regards,
Ganeshbabu R

1 Like
6

Hello,

Is there a way to modify helm chart core-statefulset.yaml to use predefined PVCs vi parameters

so instead of
...
- name: datadir
mountPath: "{{ .Values.core.persistentVolume.mountPath }}"
{{- if .Values.core.persistentVolume.subPath }}
subPath: {{ .Values.core.persistentVolume.subPath }}
{{- end }}

...
volumes
- name: datadir
emptyDir: {}

use
- name: datadir-core0
emptyDir: myspecific PVC

7

the PVCs that are used by the helm chart follow predictable naming patterns. So while you can't configure what you're asking for directly, what you can do is pre-prepare the PVC that you want mounted under the right name. If the PVC already exists, neo4j will not re-create it, it'll just use whatever is there. And so by naming the PVC strategically, you can stage whatever data & volume you wish, and plug it into the helm chart.

8

Thank David,

Would you consider the change to allow PVCs names provided as variables in the next release of the chart?
I my case it would be preferable option rather that pre-build PVCs (or modify chart)?

9

Sure. Please open an issue on the repo, and we'll see if we can get it in later

10

Thanks,
Opened issue #93: "PVCs names should be defined as variables for NEO4J chart"