Writes being dropped in k8s cluster

We have the following values.yaml config for deploying neo4j enterprise edition on k8s:

neo4j:
  name: "neo4j-cluster"
  minimumClusterSize: 3
  edition: "enterprise"
  passwordFromSecret: "neo4j-auth"  # This should be the name of your existing secret
  config:
    dbms.memory.transaction.total.max: '0'
  acceptLicenseAgreement: "yes"
  resources:
    requests:
      cpu: "1"
      memory: "2Gi"
    limits:
      cpu: "2"
      memory: "4Gi"
env:
  NEO4J_PLUGINS: '["apoc"]'

config:
#  server.directories.plugins: "/var/lib/neo4j/labs"
  dbms.security.procedures.unrestricted: "apoc.*"
  dbms.security.procedures.allowlist: "apoc.*"

volumes:
  data:
    mode: "dynamic"
    dynamic:
      storageClassName: premium-rwo

We are deploying this via helm as server-1.values.yaml, server-2.values.yaml, server-3.values.yaml

all three servers come up and are available in sysinfo command :

But when I am writing to this cluster some writes are not getting persisted.
The moment I reduce my nodes to 1 node, all writes are getting persisted.

How can I debug this further? This is the result of show config command on current server:

[
  {
    "name": "browser.allow_outgoing_connections",
    "value": "true"
  },
  {
    "name": "browser.credential_timeout",
    "value": "0s"
  },
  {
    "name": "browser.post_connect_cmd",
    "value": ""
  },
  {
    "name": "browser.remote_content_hostname_whitelist",
    "value": "guides.neo4j.com,localhost"
  },
  {
    "name": "browser.retain_connection_credentials",
    "value": "true"
  },
  {
    "name": "browser.retain_editor_history",
    "value": "true"
  },
  {
    "name": "client.allow_telemetry",
    "value": "true"
  },
  {
    "name": "db.checkpoint",
    "value": "PERIODIC"
  },
  {
    "name": "db.checkpoint.interval.time",
    "value": "15m"
  },
  {
    "name": "db.checkpoint.interval.tx",
    "value": "100000"
  },
  {
    "name": "db.checkpoint.interval.volume",
    "value": "250.00MiB"
  },
  {
    "name": "db.checkpoint.iops.limit",
    "value": "600"
  },
  {
    "name": "db.cluster.catchup.pull_interval",
    "value": "1s"
  },
  {
    "name": "db.cluster.raft.apply.buffer.max_bytes",
    "value": "1.00GiB"
  },
  {
    "name": "db.cluster.raft.apply.buffer.max_entries",
    "value": "1024"
  },
  {
    "name": "db.cluster.raft.in_queue.batch.max_bytes",
    "value": "8.00MiB"
  },
  {
    "name": "db.cluster.raft.in_queue.max_bytes",
    "value": "2.00GiB"
  },
  {
    "name": "db.cluster.raft.leader_transfer.priority_group",
    "value": ""
  },
  {
    "name": "db.cluster.raft.leader_transfer.priority_tag",
    "value": ""
  },
  {
    "name": "db.cluster.raft.log.prune_strategy",
    "value": "1g size"
  },
  {
    "name": "db.cluster.raft.log_shipping.buffer.max_bytes",
    "value": "1.00GiB"
  },
  {
    "name": "db.cluster.raft.log_shipping.buffer.max_entries",
    "value": "1024"
  },
  {
    "name": "db.cluster.raft.so_keepalive_enabled",
    "value": "false"
  },
  {
    "name": "db.filewatcher.enabled",
    "value": "true"
  },
  {
    "name": "db.format",
    "value": "block"
  },
  {
    "name": "db.import.csv.buffer_size",
    "value": "2097152"
  },
  {
    "name": "db.import.csv.legacy_quote_escaping",
    "value": "true"
  },
  {
    "name": "db.index.fulltext.default_analyzer",
    "value": "standard-no-stop-words"
  },
  {
    "name": "db.index.fulltext.eventually_consistent",
    "value": "false"
  },
  {
    "name": "db.index.fulltext.eventually_consistent_apply_parallelism",
    "value": "1"
  },
  {
    "name": "db.index.fulltext.eventually_consistent_index_update_queue_max_length",
    "value": "10000"
  },
  {
    "name": "db.index.fulltext.eventually_consistent_refresh_interval",
    "value": "0s"
  },
  {
    "name": "db.index.fulltext.eventually_consistent_refresh_parallelism",
    "value": "1"
  },
  {
    "name": "db.index_sampling.background_enabled",
    "value": "true"
  },
  {
    "name": "db.index_sampling.sample_size_limit",
    "value": "8388608"
  },
  {
    "name": "db.index_sampling.update_percentage",
    "value": "5"
  },
  {
    "name": "db.lock.acquisition.timeout",
    "value": "0s"
  },
  {
    "name": "db.logs.query.annotation_data_as_json_enabled",
    "value": "false"
  },
  {
    "name": "db.logs.query.annotation_data_format",
    "value": "CYPHER"
  },
  {
    "name": "db.logs.query.early_raw_logging_enabled",
    "value": "false"
  },
  {
    "name": "db.logs.query.enabled",
    "value": "VERBOSE"
  },
  {
    "name": "db.logs.query.max_parameter_length",
    "value": "2147483647"
  },
  {
    "name": "db.logs.query.obfuscate_literals",
    "value": "false"
  },
  {
    "name": "db.logs.query.parameter_logging_enabled",
    "value": "true"
  },
  {
    "name": "db.logs.query.plan_description_enabled",
    "value": "false"
  },
  {
    "name": "db.logs.query.threshold",
    "value": "0s"
  },
  {
    "name": "db.logs.query.transaction.enabled",
    "value": "OFF"
  },
  {
    "name": "db.logs.query.transaction.threshold",
    "value": "0s"
  },
  {
    "name": "db.memory.pagecache.warmup.enable",
    "value": "true"
  },
  {
    "name": "db.memory.pagecache.warmup.preload",
    "value": "false"
  },
  {
    "name": "db.memory.pagecache.warmup.preload.allowlist",
    "value": ".*"
  },
  {
    "name": "db.memory.pagecache.warmup.profile.interval",
    "value": "1m"
  },
  {
    "name": "db.memory.transaction.max",
    "value": "0B"
  },
  {
    "name": "db.memory.transaction.total.max",
    "value": "0B"
  },
  {
    "name": "db.recovery.fail_on_missing_files",
    "value": "true"
  },
  {
    "name": "db.relationship_grouping_threshold",
    "value": "50"
  },
  {
    "name": "db.shutdown_transaction_end_timeout",
    "value": "10s"
  },
  {
    "name": "db.store.files.preallocate",
    "value": "true"
  },
  {
    "name": "db.temporal.timezone",
    "value": "Z"
  },
  {
    "name": "db.track_query_cpu_time",
    "value": "false"
  },
  {
    "name": "db.transaction.bookmark_ready_timeout",
    "value": "30s"
  },
  {
    "name": "db.transaction.concurrent.maximum",
    "value": "1000"
  },
  {
    "name": "db.transaction.monitor.check.interval",
    "value": "2s"
  },
  {
    "name": "db.transaction.sampling.percentage",
    "value": "5"
  },
  {
    "name": "db.transaction.timeout",
    "value": "0s"
  },
  {
    "name": "db.transaction.tracing.level",
    "value": "DISABLED"
  },
  {
    "name": "db.tx_log.buffer.size",
    "value": "1048576"
  },
  {
    "name": "db.tx_log.preallocate",
    "value": "true"
  },
  {
    "name": "db.tx_log.rotation.retention_policy",
    "value": "2 days"
  },
  {
    "name": "db.tx_log.rotation.size",
    "value": "256.00MiB"
  },
  {
    "name": "db.tx_state.memory_allocation",
    "value": "ON_HEAP"
  },
  {
    "name": "dbms.cluster.catchup.client_inactivity_timeout",
    "value": "10m"
  },
  {
    "name": "dbms.cluster.discovery.endpoints",
    "value": null
  },
  {
    "name": "dbms.cluster.discovery.log_level",
    "value": "WARN"
  },
  {
    "name": "dbms.cluster.discovery.resolver_type",
    "value": "LIST"
  },
  {
    "name": "dbms.cluster.discovery.type",
    "value": "LIST"
  },
  {
    "name": "dbms.cluster.discovery.v2.endpoints",
    "value": null
  },
  {
    "name": "dbms.cluster.discovery.version",
    "value": "V1_ONLY"
  },
  {
    "name": "dbms.cluster.minimum_initial_system_primaries_count",
    "value": "3"
  },
  {
    "name": "dbms.cluster.network.connect_timeout",
    "value": "30s"
  },
  {
    "name": "dbms.cluster.network.handshake_timeout",
    "value": "20s"
  },
  {
    "name": "dbms.cluster.network.max_chunk_size",
    "value": "32768"
  },
  {
    "name": "dbms.cluster.network.supported_compression_algos",
    "value": ""
  },
  {
    "name": "dbms.cluster.raft.binding_timeout",
    "value": "1d"
  },
  {
    "name": "dbms.cluster.raft.client.max_channels",
    "value": "8"
  },
  {
    "name": "dbms.cluster.raft.election_failure_detection_window",
    "value": "3s-6s"
  },
  {
    "name": "dbms.cluster.raft.leader_failure_detection_window",
    "value": "20s-23s"
  },
  {
    "name": "dbms.cluster.raft.leader_transfer.balancing_strategy",
    "value": "EQUAL_BALANCING"
  },
  {
    "name": "dbms.cluster.raft.log.pruning_frequency",
    "value": "10m"
  },
  {
    "name": "dbms.cluster.raft.log.reader_pool_size",
    "value": "8"
  },
  {
    "name": "dbms.cluster.raft.log.rotation_size",
    "value": "250.00MiB"
  },
  {
    "name": "dbms.cluster.raft.membership.join_max_lag",
    "value": "10s"
  },
  {
    "name": "dbms.cluster.raft.membership.join_timeout",
    "value": "10m"
  },
  {
    "name": "dbms.cluster.store_copy.max_retry_time_per_request",
    "value": "20m"
  },
  {
    "name": "dbms.cypher.forbid_exhaustive_shortestpath",
    "value": "false"
  },
  {
    "name": "dbms.cypher.forbid_shortestpath_common_nodes",
    "value": "true"
  },
  {
    "name": "dbms.cypher.hints_error",
    "value": "false"
  },
  {
    "name": "dbms.cypher.infer_schema_parts",
    "value": "OFF"
  },
  {
    "name": "dbms.cypher.lenient_create_relationship",
    "value": "false"
  },
  {
    "name": "dbms.cypher.min_replan_interval",
    "value": "10s"
  },
  {
    "name": "dbms.cypher.planner",
    "value": "DEFAULT"
  },
  {
    "name": "dbms.cypher.render_plan_description",
    "value": "true"
  },
  {
    "name": "dbms.cypher.statistics_divergence_threshold",
    "value": "0.75"
  },
  {
    "name": "dbms.databases.seed_from_uri_providers",
    "value": "S3SeedProvider"
  },
  {
    "name": "dbms.db.timezone",
    "value": "UTC"
  },
  {
    "name": "dbms.kubernetes.address",
    "value": "kubernetes.default.svc:443"
  },
  {
    "name": "dbms.kubernetes.ca_crt",
    "value": "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
  },
  {
    "name": "dbms.kubernetes.cluster_domain",
    "value": "cluster.local"
  },
  {
    "name": "dbms.kubernetes.discovery.v2.service_port_name",
    "value": "transaction"
  },
  {
    "name": "dbms.kubernetes.label_selector",
    "value": null
  },
  {
    "name": "dbms.kubernetes.namespace",
    "value": "/var/run/secrets/kubernetes.io/serviceaccount/namespace"
  },
  {
    "name": "dbms.kubernetes.service_port_name",
    "value": null
  },
  {
    "name": "dbms.kubernetes.token",
    "value": "/var/run/secrets/kubernetes.io/serviceaccount/token"
  },
  {
    "name": "dbms.logs.http.enabled",
    "value": "false"
  },
  {
    "name": "dbms.max_databases",
    "value": "100"
  },
  {
    "name": "dbms.memory.tracking.enable",
    "value": "true"
  },
  {
    "name": "dbms.memory.transaction.total.max",
    "value": "0B"
  },
  {
    "name": "dbms.netty.ssl.provider",
    "value": "JDK"
  },
  {
    "name": "dbms.routing.client_side.enforce_for_domains",
    "value": ""
  },
  {
    "name": "dbms.routing.default_router",
    "value": "CLIENT"
  },
  {
    "name": "dbms.routing.driver.connection.connect_timeout",
    "value": "5s"
  },
  {
    "name": "dbms.routing.driver.connection.max_lifetime",
    "value": "1h"
  },
  {
    "name": "dbms.routing.driver.connection.pool.acquisition_timeout",
    "value": "1m"
  },
  {
    "name": "dbms.routing.driver.connection.pool.idle_test",
    "value": null
  },
  {
    "name": "dbms.routing.driver.connection.pool.max_size",
    "value": "-1"
  },
  {
    "name": "dbms.routing.driver.logging.level",
    "value": "INFO"
  },
  {
    "name": "dbms.routing.enabled",
    "value": "true"
  },
  {
    "name": "dbms.routing.load_balancing.plugin",
    "value": "server_policies"
  },
  {
    "name": "dbms.routing.load_balancing.shuffle_enabled",
    "value": "true"
  },
  {
    "name": "dbms.routing.reads_on_primaries_enabled",
    "value": "true"
  },
  {
    "name": "dbms.routing.reads_on_writers_enabled",
    "value": "false"
  },
  {
    "name": "dbms.routing_ttl",
    "value": "5m"
  },
  {
    "name": "dbms.security.allow_csv_import_from_file_urls",
    "value": "true"
  },
  {
    "name": "dbms.security.auth_cache_max_capacity",
    "value": "10000"
  },
  {
    "name": "dbms.security.auth_cache_ttl",
    "value": "10m"
  },
  {
    "name": "dbms.security.auth_cache_use_ttl",
    "value": "true"
  },
  {
    "name": "dbms.security.auth_enabled",
    "value": "true"
  },
  {
    "name": "dbms.security.auth_lock_time",
    "value": "5s"
  },
  {
    "name": "dbms.security.auth_max_failed_attempts",
    "value": "3"
  },
  {
    "name": "dbms.security.auth_minimum_password_length",
    "value": "8"
  },
  {
    "name": "dbms.security.authentication_providers",
    "value": "native"
  },
  {
    "name": "dbms.security.authorization_providers",
    "value": "native"
  },
  {
    "name": "dbms.security.cluster_status_auth_enabled",
    "value": "true"
  },
  {
    "name": "dbms.security.http_access_control_allow_origin",
    "value": "*"
  },
  {
    "name": "dbms.security.http_auth_allowlist",
    "value": "/,/browser.*"
  },
  {
    "name": "dbms.security.http_static_content_security_policy_header",
    "value": "default-src 'self'; script-src 'self' cdn.segment.com canny.io; img-src 'self' guides.neo4j.com data:; style-src 'self' fonts.googleapis.com 'unsafe-inline'; font-src 'self' fonts.gstatic.com; base-uri 'none'; object-src 'none'; frame-ancestors 'none'; connect-src 'self' api.canny.io api.segment.io ws: wss: http: https:"
  },
  {
    "name": "dbms.security.http_strict_transport_security",
    "value": null
  },
  {
    "name": "dbms.security.key.name",
    "value": "aesKey"
  },
  {
    "name": "dbms.security.keystore.password",
    "value": null
  },
  {
    "name": "dbms.security.keystore.path",
    "value": null
  },
  {
    "name": "dbms.security.ldap.authentication.attribute",
    "value": "samaccountname"
  },
  {
    "name": "dbms.security.ldap.authentication.cache_enabled",
    "value": "true"
  },
  {
    "name": "dbms.security.ldap.authentication.mechanism",
    "value": "simple"
  },
  {
    "name": "dbms.security.ldap.authentication.search_for_attribute",
    "value": "false"
  },
  {
    "name": "dbms.security.ldap.authentication.user_dn_template",
    "value": "uid={0},ou=users,dc=example,dc=com"
  },
  {
    "name": "dbms.security.ldap.authorization.access_permitted_group",
    "value": ""
  },
  {
    "name": "dbms.security.ldap.authorization.group_membership_attributes",
    "value": "memberOf"
  },
  {
    "name": "dbms.security.ldap.authorization.group_to_role_mapping",
    "value": ""
  },
  {
    "name": "dbms.security.ldap.authorization.nested_groups_enabled",
    "value": "false"
  },
  {
    "name": "dbms.security.ldap.authorization.nested_groups_search_filter",
    "value": "(&(objectclass=group)(member:1.2.840.113556.1.4.1941:={0}))"
  },
  {
    "name": "dbms.security.ldap.authorization.system_password",
    "value": null
  },
  {
    "name": "dbms.security.ldap.authorization.system_username",
    "value": null
  },
  {
    "name": "dbms.security.ldap.authorization.use_system_account",
    "value": "false"
  },
  {
    "name": "dbms.security.ldap.authorization.user_search_base",
    "value": "ou=users,dc=example,dc=com"
  },
  {
    "name": "dbms.security.ldap.authorization.user_search_filter",
    "value": "(&(objectClass=*)(uid={0}))"
  },
  {
    "name": "dbms.security.ldap.connection_timeout",
    "value": "30s"
  },
  {
    "name": "dbms.security.ldap.host",
    "value": "localhost"
  },
  {
    "name": "dbms.security.ldap.read_timeout",
    "value": "30s"
  },
  {
    "name": "dbms.security.ldap.referral",
    "value": "follow"
  },
  {
    "name": "dbms.security.ldap.use_starttls",
    "value": "false"
  },
  {
    "name": "dbms.security.log_successful_authentication",
    "value": "true"
  },
  {
    "name": "dbms.security.logs.ldap.groups_at_debug_level_enabled",
    "value": "false"
  },
  {
    "name": "dbms.security.logs.oidc.jwt_claims_at_debug_level_enabled",
    "value": "false"
  },
  {
    "name": "dbms.security.procedures.allowlist",
    "value": "*"
  },
  {
    "name": "dbms.security.procedures.unrestricted",
    "value": ""
  },
  {
    "name": "dbms.ssl.policy.bolt.base_directory",
    "value": "/var/lib/neo4j/certificates/bolt"
  },
  {
    "name": "dbms.ssl.policy.bolt.ciphers",
    "value": null
  },
  {
    "name": "dbms.ssl.policy.bolt.client_auth",
    "value": "NONE"
  },
  {
    "name": "dbms.ssl.policy.bolt.enabled",
    "value": "false"
  },
  {
    "name": "dbms.ssl.policy.bolt.private_key",
    "value": "/var/lib/neo4j/certificates/bolt/private.key"
  },
  {
    "name": "dbms.ssl.policy.bolt.private_key_password",
    "value": null
  },
  {
    "name": "dbms.ssl.policy.bolt.public_certificate",
    "value": "/var/lib/neo4j/certificates/bolt/public.crt"
  },
  {
    "name": "dbms.ssl.policy.bolt.revoked_dir",
    "value": "/var/lib/neo4j/certificates/bolt/revoked"
  },
  {
    "name": "dbms.ssl.policy.bolt.tls_versions",
    "value": "TLSv1.2"
  },
  {
    "name": "dbms.ssl.policy.bolt.trust_all",
    "value": "false"
  },
  {
    "name": "dbms.ssl.policy.bolt.trust_expired",
    "value": "true"
  },
  {
    "name": "dbms.ssl.policy.bolt.trusted_dir",
    "value": "/var/lib/neo4j/certificates/bolt/trusted"
  },
  {
    "name": "dbms.ssl.policy.bolt.verify_hostname",
    "value": "false"
  },
  {
    "name": "dbms.ssl.policy.https.base_directory",
    "value": "/var/lib/neo4j/certificates/https"
  },
  {
    "name": "dbms.ssl.policy.https.ciphers",
    "value": null
  },
  {
    "name": "dbms.ssl.policy.https.client_auth",
    "value": "NONE"
  },
  {
    "name": "dbms.ssl.policy.https.enabled",
    "value": "false"
  },
  {
    "name": "dbms.ssl.policy.https.private_key",
    "value": "/var/lib/neo4j/certificates/https/private.key"
  },
  {
    "name": "dbms.ssl.policy.https.private_key_password",
    "value": null
  },
  {
    "name": "dbms.ssl.policy.https.public_certificate",
    "value": "/var/lib/neo4j/certificates/https/public.crt"
  },
  {
    "name": "dbms.ssl.policy.https.revoked_dir",
    "value": "/var/lib/neo4j/certificates/https/revoked"
  },
  {
    "name": "dbms.ssl.policy.https.tls_versions",
    "value": "TLSv1.2"
  },
  {
    "name": "dbms.ssl.policy.https.trust_all",
    "value": "false"
  },
  {
    "name": "dbms.ssl.policy.https.trust_expired",
    "value": "true"
  },
  {
    "name": "dbms.ssl.policy.https.trusted_dir",
    "value": "/var/lib/neo4j/certificates/https/trusted"
  },
  {
    "name": "dbms.ssl.policy.https.verify_hostname",
    "value": "false"
  },
  {
    "name": "dbms.usage_report.enabled",
    "value": "true"
  },
  {
    "name": "initial.dbms.automatically_enable_free_servers",
    "value": "false"
  },
  {
    "name": "initial.dbms.database_allocator",
    "value": "EQUAL_NUMBERS"
  },
  {
    "name": "initial.dbms.default_database",
    "value": "neo4j"
  },
  {
    "name": "initial.dbms.default_primaries_count",
    "value": "1"
  },
  {
    "name": "initial.dbms.default_secondaries_count",
    "value": "0"
  },
  {
    "name": "initial.server.allowed_databases",
    "value": ""
  },
  {
    "name": "initial.server.denied_databases",
    "value": ""
  },
  {
    "name": "initial.server.mode_constraint",
    "value": "NONE"
  },
  {
    "name": "initial.server.tags",
    "value": ""
  },
  {
    "name": "server.backup.enabled",
    "value": "true"
  },
  {
    "name": "server.backup.exec_connector.command",
    "value": ""
  },
  {
    "name": "server.backup.exec_connector.scheme",
    "value": ""
  },
  {
    "name": "server.backup.listen_address",
    "value": "0.0.0.0:6362"
  },
  {
    "name": "server.backup.store_copy_max_retry_time_per_request",
    "value": "20m"
  },
  {
    "name": "server.bolt.additional_listen_addresses",
    "value": ""
  },
  {
    "name": "server.bolt.advertised_address",
    "value": "localhost:7687"
  },
  {
    "name": "server.bolt.connection_keep_alive",
    "value": "30s"
  },
  {
    "name": "server.bolt.connection_keep_alive_for_requests",
    "value": "ALL"
  },
  {
    "name": "server.bolt.connection_keep_alive_probes",
    "value": "2"
  },
  {
    "name": "server.bolt.connection_keep_alive_streaming_scheduling_interval",
    "value": "30s"
  },
  {
    "name": "server.bolt.enable_network_error_accounting",
    "value": "true"
  },
  {
    "name": "server.bolt.enabled",
    "value": "true"
  },
  {
    "name": "server.bolt.listen_address",
    "value": "0.0.0.0:7687"
  },
  {
    "name": "server.bolt.network_abort_clear_window_duration",
    "value": "10m"
  },
  {
    "name": "server.bolt.network_abort_warn_threshold",
    "value": "2"
  },
  {
    "name": "server.bolt.network_abort_warn_window_duration",
    "value": "10m"
  },
  {
    "name": "server.bolt.ocsp_stapling_enabled",
    "value": "false"
  },
  {
    "name": "server.bolt.telemetry.enabled",
    "value": "false"
  },
  {
    "name": "server.bolt.thread_pool_keep_alive",
    "value": "5m"
  },
  {
    "name": "server.bolt.thread_pool_max_size",
    "value": "400"
  },
  {
    "name": "server.bolt.thread_pool_min_size",
    "value": "5"
  },
  {
    "name": "server.bolt.thread_starvation_clear_window_duration",
    "value": "10m"
  },
  {
    "name": "server.bolt.thread_starvation_warn_threshold",
    "value": "2"
  },
  {
    "name": "server.bolt.thread_starvation_warn_window_duration",
    "value": "10m"
  },
  {
    "name": "server.bolt.tls_level",
    "value": "DISABLED"
  },
  {
    "name": "server.bolt.traffic_accounting_check_period",
    "value": "5m"
  },
  {
    "name": "server.bolt.traffic_accounting_clear_duration",
    "value": "10m"
  },
  {
    "name": "server.bolt.traffic_accounting_incoming_threshold_mbps",
    "value": "950"
  },
  {
    "name": "server.bolt.traffic_accounting_outgoing_threshold_mbps",
    "value": "950"
  },
  {
    "name": "server.cluster.advertised_address",
    "value": "localhost:6000"
  },
  {
    "name": "server.cluster.catchup.connect_randomly_to_server_group",
    "value": ""
  },
  {
    "name": "server.cluster.catchup.connect_randomly_to_server_tags",
    "value": ""
  },
  {
    "name": "server.cluster.catchup.upstream_strategy",
    "value": ""
  },
  {
    "name": "server.cluster.catchup.user_defined_upstream_strategy",
    "value": ""
  },
  {
    "name": "server.cluster.listen_address",
    "value": "0.0.0.0:6000"
  },
  {
    "name": "server.cluster.network.native_transport_enabled",
    "value": "true"
  },
  {
    "name": "server.cluster.raft.advertised_address",
    "value": "localhost:7000"
  },
  {
    "name": "server.cluster.raft.listen_address",
    "value": "0.0.0.0:7000"
  },
  {
    "name": "server.cluster.system_database_mode",
    "value": "PRIMARY"
  },
  {
    "name": "server.config.strict_validation.enabled",
    "value": "false"
  },
  {
    "name": "server.cypher.parallel.worker_limit",
    "value": "0"
  },
  {
    "name": "server.databases.default_to_read_only",
    "value": "false"
  },
  {
    "name": "server.databases.read_only",
    "value": ""
  },
  {
    "name": "server.databases.writable",
    "value": ""
  },
  {
    "name": "server.db.query_cache_size",
    "value": "1000"
  },
  {
    "name": "server.default_advertised_address",
    "value": "localhost"
  },
  {
    "name": "server.default_listen_address",
    "value": "0.0.0.0"
  },
  {
    "name": "server.directories.cluster_state",
    "value": "/var/lib/neo4j/data/cluster-state"
  },
  {
    "name": "server.directories.data",
    "value": "/var/lib/neo4j/data"
  },
  {
    "name": "server.directories.dumps.root",
    "value": "/var/lib/neo4j/data/dumps"
  },
  {
    "name": "server.directories.import",
    "value": "/import"
  },
  {
    "name": "server.directories.lib",
    "value": "/var/lib/neo4j/lib"
  },
  {
    "name": "server.directories.licenses",
    "value": "/var/lib/neo4j/licenses"
  },
  {
    "name": "server.directories.logs",
    "value": "/logs"
  },
  {
    "name": "server.directories.metrics",
    "value": "/metrics"
  },
  {
    "name": "server.directories.neo4j_home",
    "value": "/var/lib/neo4j"
  },
  {
    "name": "server.directories.plugins",
    "value": "/var/lib/neo4j/plugins"
  },
  {
    "name": "server.directories.run",
    "value": "/var/lib/neo4j/run"
  },
  {
    "name": "server.directories.script.root",
    "value": "/var/lib/neo4j/data/scripts"
  },
  {
    "name": "server.directories.transaction.logs.root",
    "value": "/var/lib/neo4j/data/transactions"
  },
  {
    "name": "server.discovery.advertised_address",
    "value": "localhost:5000"
  },
  {
    "name": "server.discovery.listen_address",
    "value": "0.0.0.0:5000"
  },
  {
    "name": "server.dynamic.setting.allowlist",
    "value": "*"
  },
  {
    "name": "server.groups",
    "value": ""
  },
  {
    "name": "server.http.advertised_address",
    "value": "localhost:7474"
  },
  {
    "name": "server.http.enabled",
    "value": "true"
  },
  {
    "name": "server.http.listen_address",
    "value": "0.0.0.0:7474"
  },
  {
    "name": "server.http.transaction_idle_timeout",
    "value": "30s"
  },
  {
    "name": "server.http_enabled_modules",
    "value": "TRANSACTIONAL_ENDPOINTS,UNMANAGED_EXTENSIONS,BROWSER,ENTERPRISE_MANAGEMENT_ENDPOINTS"
  },
  {
    "name": "server.http_enabled_transports",
    "value": "HTTP1_1,HTTP2"
  },
  {
    "name": "server.https.advertised_address",
    "value": "localhost:7473"
  },
  {
    "name": "server.https.enabled",
    "value": "false"
  },
  {
    "name": "server.https.listen_address",
    "value": "0.0.0.0:7473"
  },
  {
    "name": "server.jvm.additional",
    "value": "-XX:+UseG1GC
-XX:-OmitStackTraceInFastThrow
-XX:+AlwaysPreTouch
-XX:+UnlockExperimentalVMOptions
-XX:+TrustFinalNonStaticFields
-XX:+DisableExplicitGC
-Djdk.nio.maxCachedBufferSize=1024
-Dio.netty.tryReflectionSetAccessible=true
-Djdk.tls.ephemeralDHKeySize=2048
-Djdk.tls.rejectClientInitiatedRenegotiation=true
-XX:FlightRecorderOptions=stackdepth=256
-XX:+UnlockDiagnosticVMOptions
-XX:+DebugNonSafepoints
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
-Dlog4j2.disable.jmx=true"
  },
  {
    "name": "server.logs.config",
    "value": "/config/server-logs.xml/server-logs.xml"
  },
  {
    "name": "server.logs.debug.enabled",
    "value": "true"
  },
  {
    "name": "server.logs.gc.enabled",
    "value": "false"
  },
  {
    "name": "server.logs.gc.options",
    "value": "-Xlog:gc*,safepoint,age*=trace"
  },
  {
    "name": "server.logs.gc.rotation.keep_number",
    "value": "5"
  },
  {
    "name": "server.logs.gc.rotation.size",
    "value": "20.00MiB"
  },
  {
    "name": "server.logs.user.config",
    "value": "/config/user-logs.xml/user-logs.xml"
  },
  {
    "name": "server.max_databases",
    "value": "100"
  },
  {
    "name": "server.memory.heap.initial_size",
    "value": null
  },
  {
    "name": "server.memory.heap.max_size",
    "value": null
  },
  {
    "name": "server.memory.off_heap.block_cache_size",
    "value": "128"
  },
  {
    "name": "server.memory.off_heap.max_cacheable_block_size",
    "value": "512.00KiB"
  },
  {
    "name": "server.memory.off_heap.transaction_max_size",
    "value": "2.00GiB"
  },
  {
    "name": "server.memory.pagecache.directio",
    "value": "false"
  },
  {
    "name": "server.memory.pagecache.flush.buffer.enabled",
    "value": "false"
  },
  {
    "name": "server.memory.pagecache.flush.buffer.size_in_pages",
    "value": "128"
  },
  {
    "name": "server.memory.pagecache.scan.prefetchers",
    "value": "4"
  },
  {
    "name": "server.memory.pagecache.size",
    "value": null
  },
  {
    "name": "server.memory.query_cache.per_db_cache_num_entries",
    "value": "1000"
  },
  {
    "name": "server.memory.query_cache.shared_cache_num_entries",
    "value": "1000"
  },
  {
    "name": "server.memory.query_cache.sharing_enabled",
    "value": "false"
  },
  {
    "name": "server.metrics.csv.enabled",
    "value": "true"
  },
  {
    "name": "server.metrics.csv.interval",
    "value": "30s"
  },
  {
    "name": "server.metrics.csv.rotation.compression",
    "value": "ZIP"
  },
  {
    "name": "server.metrics.csv.rotation.keep_number",
    "value": "7"
  },
  {
    "name": "server.metrics.csv.rotation.size",
    "value": "10.00MiB"
  },
  {
    "name": "server.metrics.enabled",
    "value": "true"
  },
  {
    "name": "server.metrics.filter",
    "value": "*bolt.connections*,*bolt.messages_received*,*bolt.messages_started*,*dbms.pool.bolt.free,*dbms.pool.bolt.total_size,*dbms.pool.bolt.total_used,*dbms.pool.bolt.used_heap,*cluster.raft.is_leader,*cluster.raft.last_leader_message,*cluster.raft.replication_attempt,*cluster.raft.replication_fail,*cluster.raft.last_applied,*cluster.raft.last_appended,*cluster.raft.append_index,*cluster.raft.commit_index,*cluster.raft.applied_index,*cluster.internal.discovery.memberset.left,*cluster.internal.discovery.crdt.gossip_id_data.size,*cluster.internal.discovery.crdt.server_data.size,*cluster.internal.discovery.crdt.database_data.size,*cluster.internal.discovery.crdt.leader_data.size,*cluster.internal.discovery.crdt.total_merge_operations,*cluster.internal.discovery.crdt.total_update_operations,*cluster.internal.discovery.gossip.incoming_queue_size,*cluster.internal.discovery.gossip.total_received_data,*cluster.internal.discovery.gossip.total_sent_data,*cluster.internal.discovery.gossip.uncontactable_members_exist,*check_point.*,*cypher.replan_events,*ids_in_use*,*pool.transaction.*.total_used,*pool.transaction.*.used_heap,*pool.transaction.*.used_native,*store.size*,*transaction.active_read,*transaction.active_write,*transaction.committed*,*transaction.last_committed_tx_id,*transaction.peak_concurrent,*transaction.rollbacks*,*page_cache.hit*,*page_cache.page_faults,*page_cache.usage_ratio,*vm.file.descriptors.count,*vm.gc.time.*,*vm.heap.used,*vm.memory.buffer.direct.used,*vm.memory.pool.g1_eden_space,*vm.memory.pool.g1_old_gen,*vm.pause_time,*vm.thread*,*db.query.execution*"
  },
  {
    "name": "server.metrics.graphite.enabled",
    "value": "false"
  },
  {
    "name": "server.metrics.graphite.interval",
    "value": "30s"
  },
  {
    "name": "server.metrics.graphite.server",
    "value": "0.0.0.0:2003"
  },
  {
    "name": "server.metrics.jmx.enabled",
    "value": "true"
  },
  {
    "name": "server.metrics.prefix",
    "value": "neo4j"
  },
  {
    "name": "server.metrics.prometheus.enabled",
    "value": "false"
  },
  {
    "name": "server.metrics.prometheus.endpoint",
    "value": "localhost:2004"
  },
  {
    "name": "server.panic.shutdown_on_panic",
    "value": "true"
  },
  {
    "name": "server.routing.advertised_address",
    "value": "localhost:7688"
  },
  {
    "name": "server.routing.listen_address",
    "value": "0.0.0.0:7688"
  },
  {
    "name": "server.threads.worker_count",
    "value": "4"
  },
  {
    "name": "server.unmanaged_extension_classes",
    "value": ""
  },
  {
    "name": "server.windows_service_name",
    "value": "neo4j"
  }
]

@dhiren

What version of Neo4j?

You report

But when I am writing to this cluster some writes are not getting persisted.

but so some writes do persist?
Is this a timing issue? How are you confirming the writes are not persisted? Are you using bookmarks?

5.23 Startup Plan Enterprise Edition

Yes some writes persist, and it's very random which ones do.
I'm not sure if it's a timing issue because this happens with relatively small operations too ( 5-6 node 8-9 edges ). I'm confirming writes are persisted by querying the db from neo4j browser after my app has run.

@dhiren

but the Neo4j browser is not using bookmarks.

If you connect via Neo4j Browser and in the Connect URL: u defined as bolt://server-1.... :7687, rather than neo4j://server-a ..... :7687 then the usage of bolt:// will make the connection go to server-1 and stay there, and in doing so u can determine if the write was written to server-1 etc

I use bolt://loadbalancer to check (that is also the neo4j URI I use on my app) and I've verified using logs on my server as well where I explicitly fetch the number of nodes and relationships from the database after inserting.