Error in Neo4j pod logs and failed to start via helm package

Hi All,

I am running neo4j 3.5.7 community version via helm package..

I am using stable helm charts,

helm repo add stable https://kubernetes-charts.storage.googleapis.com/

and below is the command i am using to install helm

helm install --name neo4j-community stable/neo4j -f ./neo4j-community-values.yaml

and below is the error I am getting in neo4j logs,

Directories in use:
  home:         /var/lib/neo4j
  config:       /var/lib/neo4j/conf
  logs:         /logs
  plugins:      /plugins
  import:       /var/lib/neo4j/import
   data:         /var/lib/neo4j/data
  certificates: /var/lib/neo4j/certificates
  run:          /var/lib/neo4j/run
Starting Neo4j.
2020-06-27 14:44:19.398+0000 ERROR [o.n.s.CommunityBootstrapper] Failed to start Neo4j on unknown address. Setting "dbms.connector.bolt.advertised_address" must be in the format "hostname:port" or ":port". "neo4j-community-neo4j-core-0.neo4j-community-neo4j.core.svc.cluster.local" does not conform to these formats
org.neo4j.graphdb.config.InvalidSettingException: Setting "dbms.connector.bolt.advertised_address" must be in the format "hostname:port" or ":port". "neo4j-community-neo4j-core-0.neo4j-community-neo4j.core.svc.cluster.local" does not conform to these formats
        at org.neo4j.graphdb.config.Setting.validate(Setting.java:96)
        at org.neo4j.kernel.configuration.ConnectorValidator.lambda$validate$2(ConnectorValidator.java:145)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1699)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at org.neo4j.kernel.configuration.ConnectorValidator.validate(ConnectorValidator.java:141)
        at org.neo4j.kernel.configuration.IndividualSettingsValidator.validate(IndividualSettingsValidator.java:61)
        at org.neo4j.kernel.configuration.Config.migrateAndValidateAndUpdateSettings(Config.java:807)
        at org.neo4j.kernel.configuration.Config.<init>(Config.java:420)
        at org.neo4j.kernel.configuration.Config.<init>(Config.java:84)
        at org.neo4j.kernel.configuration.Config$Builder.build(Config.java:314)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:108)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
        at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: java.lang.IllegalArgumentException: Setting "dbms.connector.bolt.advertised_address" must be in the format "hostname:port" or ":port". "neo4j-community-neo4j-core-0.neo4j-community-neo4j-core.svc.cluster.local" does not conform to these formats
        at org.neo4j.helpers.SocketAddressParser.deriveSocketAddress(SocketAddressParser.java:57)
        at org.neo4j.kernel.configuration.Settings$14.apply(Settings.java:615)
        at org.neo4j.kernel.configuration.Settings$14.apply(Settings.java:589)
        at org.neo4j.kernel.configuration.Settings$15.apply(Settings.java:653)
        at org.neo4j.kernel.configuration.Settings$15.apply(Settings.java:630)
        at org.neo4j.graphdb.config.Setting.validate(Setting.java:83)
        ... 19 more
2020-06-27 14:44:19.402+0000 INFO [o.n.s.CommunityBootstrapper] Neo4j Server shutdown initiated by request

I am running helm version,

root@graylog3:~/KubernetesCluster/# helm version
Client: &version.Version{SemVer:"v2.16.9", GitCommit:"8ad7037828e5a0fca1009dabe290130da6368e39", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.16.5", GitCommit:"89bd14c1541fa93a09492010030fd3699ca65a97", GitTreeState:"clean"}

Please let me know how to resolve this issue and previously we were not getting error this.. If anyone faced issues like this please share your thoughts it would be very helpful.

Note:- I am attaching the neo4j-community-values files.

# Default values for Neo4j.
# This is a YAML-formatted file.
# Declare name/value pairs to be passed into your templates.
# name: value

name: "neo4j"

# Specs for the Neo4j docker image
image: "neo4j"
imageTag: "3.5.7"
imagePullPolicy: "IfNotPresent"
# imagePullSecret: registry-secret
acceptLicenseAgreement: "no"
podDisruptionBudget: {}
  # minAvailable: 2
  # maxUnavailable: 1

# Use password authentication
authEnabled: true

## Specify password for neo4j user
## Defaults to a random 10-character alphanumeric string if not set and authEnabled is true
neo4jPassword: "qwertyuiop@bng1"

# Specify cluster domain (used eg. as suffix in definition of NEO4J_causal__clustering_initial__discovery__members environment variable)
clusterDomain: "cluster.local"

# Specs for the images used for running tests against the Helm package
testImage: "markhneedham/k8s-kubectl"
testImageTag: "master"

# Cores
core:
  numberOfServers: 1
  persistentVolume:
    ## whether or not persistence is enabled
    ##
    enabled: true

    ## core server data Persistent Volume mount root path
    ##
    mountPath: /data

    ## core server data Persistent Volume size
    ##
    size: 50Gi

    ## core server data Persistent Volume Storage Class
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    storageClass: "core"

    ## Subdirectory of core server data Persistent Volume to mount
    ## Useful if the volume's root directory is not empty
    ##
    ## subPath: ""

  ## Pass extra environment variables to the Neo4j container.
  ##
  extraVars:
  - name: NEO4J_dbms_unmanaged__extension__classes
    value: com.graphaware.server=/graphaware
  - name: NEO4J_com_graphaware_runtime_enabled
    value: "true"
  - name: NEO4J_com_graphaware_module_UIDM_1
    value: com.graphaware.module.uuid.UuidBootstrapper
  - name: NEO4J_com_graphaware_module_UIDM_uuidProperty
    value: uuid
  - name: NEO4J_com_graphaware_module_UIDM_relationship
    value: "true"
  - name: NEO4J_com_graphaware_module_UIDM_uuidIndex
    value: uuidIndex
  - name: NEO4J_com_graphaware_module_UIDM_initializeUntil
    value: "0"
  - name: NEO4J_com_graphaware_module_ES_2
    value: com.graphaware.module.es.ElasticSearchModuleBootstrapper
  - name: NEO4J_com_graphaware_module_ES_uri
    value: elasticsearch-master
  - name: NEO4J_com_graphaware_module_ES_port
    value: "9200"
  - name: NEO4J_com_graphaware_module_ES_protocol
    value: http
  - name: NEO4J_com_graphaware_module_ES_index
    value: graph
  - name: NEO4J_com_graphaware_module_ES_keyProperty
    value: id
  - name: NEO4J_com_graphaware_module_ES_retryOnError
    value: "false"
  - name: NEO4J_com_graphaware_module_ES_queueSize
    value: "10000"
  - name: NEO4J_com_graphaware_module_ES_reindexBatchSize
    value: "2000"
  - name: NEO4J_com_graphaware_module_ES_relationship
    value: (true)
  - name: NEO4J_com_graphaware_module_ES_bulk
    value: "false"
  - name: NEO4J_com_graphaware_module_ES_initializeUntil
    value: "2222222222222"
  - name: NEO4J_com_graphaware_module_ES_mapping
    value: com.graphaware.module.es.mapping.JsonFileMapping
  - name: NEO4J_com_graphaware_module_ES_file
    value: ../../../plugins/mapping.json
  - name: NEO4J_dbms_memory_heap_initial__size
    value: "2G"
  - name: NEO4J_dbms_memory_heap_max__size
    value: "2G"
  - name: NEO4J_dbms_memory_pagecache__size
    value: "1G"  
  - name: NEO4J_dbms_security_procedures_unrestricted
    value: apoc.\\\*
  - name: NEO4J_apoc_export_file_enabled
    value: "true"
  - name: NEO4J_apoc_import_file_enabled
    value: "true"    
  # - name: EXTRA_VAR_1
  #   value: extra-var-value-1
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j core pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers:
  ## init containers to run before the Neo4j core pod e.g. to install plugins

  - name: init-plugins
    image: "appropriate/curl:latest"
    imagePullPolicy: "IfNotPresent"
    volumeMounts:
    - name: plugins
      mountPath: /plugins
    command:
      - "/bin/sh"
      - "-c"
      - |
        curl -L https://s3.amazonaws.com/graphaware-neo4j-to-elasticsearch-3.5.4.53.11.jar -O
        curl -L https://s3.amazonaws.com/graphaware-server-community-all-3.5.4.53.jar -O
        curl -L https://s3.amazonaws.com/graphaware-uuid-3.5.4.53.17.jar -O
        curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.5.0.11/apoc-3.5.0.11-all.jar -O
        curl -L https://github.com/neo4j-contrib/neo4j-graph-algorithms/releases/download/3.5.4.0/graph-algorithms-algo-3.5.4.0.jar -O
        cp graph-algorithms-algo-3.5.4.0.jar /plugins/
        cp apoc-3.5.0.11-all.jar /plugins/        
        cp graphaware*.jar /plugins/
        cat <<'EOF' > mapping.json
        {
          "defaults": {
            "nodes_index": "graph-node",
            "relationships_index": "graph-relationship",
            "key_property": "id",
            "include_remaining_properties": true
          },
          "node_mappings": [
            {
              "condition": "allNodes()",
              "type": "nodes",
              "properties": {
                "labels": "getLabels()"
              }
            },
            {
              "condition": "hasLabel('CI')",
              "type": "nodes",
              "index": "ci",
              "properties": {
                "labels": "getLabels()"
              }
            }
          ],
          "relationship_mappings": [
            {
              "condition": "isType('HAS_ALLOCATED_ADDRESS')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[allot:HAS_ALLOCATED_ADDRESS]->(b) where id(allot) = {id} RETURN collect({name: type(allot), source_node: a.id, destination_node: b.id}) AS value')"
               }
            },
            {
              "condition": "isType('CLUSTER_MEMBER_OF')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[c:CLUSTER_MEMBER_OF]->(b) where id(c) = {id} RETURN collect({name: type(c), source_node: a.id, destination_node: b.id}) AS value')"
               }
            },
            {
              "condition": "isType('HOSTS')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[h:HOSTS]->(b) where id(h) = {id} RETURN collect({name: type(h), source_node: a.id, destination_node: b.id}) AS value')"
               }
            },
            {
              "condition": "isType('CONNECTS')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[m:CONNECTS]->(b) where id(m) = {id} RETURN collect({name: type(m), source_node: a.id, destination_node: b.id}) AS value')"
               }
            },
            {
              "condition": "isType('CRITICAL_RESOURCE_OF')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[mof:CRITICAL_RESOURCE_OF]->(b) where id(mof) = {id} RETURN collect({name: type(mof), source_node: a.id, destination_node: b.id}) AS value')"
               }
            },
            {
              "condition": "isType('NON_CRITICAL_RESOURCE_OF')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[p:NON_CRITICAL_RESOURCE_OF]->(b)  where id(p) = {id} RETURN collect({name: type(p), source_node: a.id, destination_node: b.id}) AS value')"
               }
            },
            {
              "condition": "isType('SAME_AS')",
              "type": "relationship",
              "properties": {
                "relationship": "query('MATCH (a)-[s:SAME_AS]->(b)  where id(s) = {id} RETURN collect({name: type(s), source_node: a.id, destination_node: b.id}) AS value')"
               }
            }
          ]
        }
        EOF
        cp mapping.json /plugins/
# Read Replicas
readReplica:
  numberOfServers: 0
  ## Pass extra environment variables to the Neo4j container.
  ##
  # extraVars:
  # - name: EXTRA_VAR_1
  #   value: extra-var-value-1
  # - name: EXTRA_VAR_2
  #   value: extra-var-value-2

  sidecarContainers: []
  ## Additional containers to be added to the Neo4j replica pod.
  #  - name: my-sidecar
  #    image: nginx:latest

  initContainers: []
  ## init containers to run before the Neo4j replica pod e.g. to install plugins

  # - name: init-plugins
  #   image: "appropriate/curl:latest"
  #   imagePullPolicy: "IfNotPresent"
  #   volumeMounts:
  #   - name: plugins
  #     mountPath: /plugins
  #   command:
  #     - "/bin/sh"
  #     - "-c"
  #     - |
  #       curl -L https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.2.0.3/apoc-3.2.0.3-all.jar -O
  #       cp apoc-3.2.0.3-all.jar /plugins/

resources:
limits:
  cpu: 1.5
  memory: 2Gi
requests:
  cpu: 0.5
  memory: 1Gi

Regards,
Ganeshbabu R

[quote="babu.ganesh0708, post:1, topic:20685"]

Caused by: java.lang.IllegalArgumentException: Setting "dbms.connector.bolt.advertised_address" must be in the format "hostname:port" or ":port". "neo4j-community-neo4j-core-0.neo4j-community-neo4j-core.svc.cluster.local" does not conform to these formats

Not familiar with the HELM environment but somehow that string does not gets translated correctly and neo4j is expecting a valid hostname or ip-address

Add this to extra vars

  • name: NEO4J_dbms_connector_bolt_advertised__address
    value: neo4j-community-neo4j-core-0.neo4j-community-neo4j-core.svc.cluster.local:7687

The host name can be an internal host name or external if you want to be able to access it using external browser.

1 Like