Cypher-shell certificates

Neo4j Graph Platform
1

With Bolt configured like this cypher-shell with Neo4j 4.0 seems to not like the self-signed certificates anymore:

./bin/cypher-shell --database system -u neo4j -p $secret --encryption true --debug
org.neo4j.driver.exceptions.SecurityException: Failed to establish secured connection with the server
	at org.neo4j.driver.internal.util.Futures.blockingGet(Futures.java:143)
	at org.neo4j.driver.internal.InternalSession.run(InternalSession.java:69)
	at org.neo4j.driver.internal.InternalSession.run(InternalSession.java:51)
	at org.neo4j.driver.internal.AbstractQueryRunner.run(AbstractQueryRunner.java:37)
	at org.neo4j.driver.internal.AbstractQueryRunner.run(AbstractQueryRunner.java:55)
	at org.neo4j.shell.state.BoltStateHandler.reconnect(BoltStateHandler.java:196)
	at org.neo4j.shell.state.BoltStateHandler.reconnect(BoltStateHandler.java:173)
	at org.neo4j.shell.state.BoltStateHandler.connect(BoltStateHandler.java:161)
	at org.neo4j.shell.CypherShell.connect(CypherShell.java:143)
	at org.neo4j.shell.Main.connectMaybeInteractively(Main.java:129)
	at org.neo4j.shell.Main.startShell(Main.java:92)
	at org.neo4j.shell.Main.main(Main.java:41)
	Suppressed: org.neo4j.driver.internal.util.ErrorUtil$InternalExceptionCause
		at org.neo4j.driver.internal.async.connection.HandshakeHandler.transformError(HandshakeHandler.java:195)
		at org.neo4j.driver.internal.async.connection.HandshakeHandler.exceptionCaught(HandshakeHandler.java:96)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:276)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:268)
		at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.exceptionCaught(SslHandler.java:1095)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.notifyHandlerException(AbstractChannelHandlerContext.java:831)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:376)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
		at org.neo4j.driver.internal.shaded.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
		at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
		at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
		at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
		at org.neo4j.driver.internal.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
		at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
		at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
		at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
		at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
		at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
		at org.neo4j.driver.internal.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
		at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
		at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:645)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1048)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:995)
	at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1499)
	at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1513)
	at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1397)
	at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
	at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
	at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at org.neo4j.driver.internal.shaded.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
	at org.neo4j.driver.internal.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
	at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
	at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
	at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
	at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
	at org.neo4j.driver.internal.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	at java.base/sun.security.validator.Validator.validate(Validator.java:264)
	at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:623)
	... 36 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
	... 42 more

Here are the relevant sections from the neo4j.conf:

# Bolt connector
dbms.connector.bolt.enabled=true
dbms.connector.bolt.tls_level=REQUIRED
dbms.connector.bolt.listen_address=0.0.0.0:7687
dbms.connector.bolt.advertised_address=10.132.13.45:7687

# SSL settings (dbms.ssl.policy.<scope>.*)
#  .base_directory       Base directory for SSL policies paths. All relative paths within the
#                        SSL configuration will be resolved from the base dir.
#
#  .private_key          A path to the key file relative to the '.base_directory'.
#
#  .private_key_password The password for the private key.
#
#  .public_certificate   A path to the public certificate file relative to the '.base_directory'.
#
#  .trusted_dir          A path to a directory containing trusted certificates.
#
#  .revoked_dir          Path to the directory with Certificate Revocation Lists (CRLs).
#
#  .verify_hostname      If true, the server will verify the hostname that the client uses to connect with. In order
#                        for this to work, the server public certificate must have a valid CN and/or matching
#                        Subject Alternative Names.
#
#  .client_auth          How the client should be authorized. Possible values are: 'none', 'optional', 'require'.
#
#  .tls_versions         A comma-separated list of allowed TLS versions. By default only TLSv1.2 is allowed.
#
#  .trust_all            Setting this to 'true' will ignore the trust truststore, trusting all clients and servers.
#                        Use of this mode is discouraged. It would offer encryption but no security.
#
#  .ciphers              A comma-separated list of allowed ciphers. The default ciphers are the defaults of
#                        the JVM platform.

# Bolt SSL configuration
dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.bolt.base_directory=/etc/certs
dbms.ssl.policy.bolt.private_key=private.key
dbms.ssl.policy.bolt.public_certificate=public.crt

# Https SSL configuration
dbms.ssl.policy.https.enabled=true
dbms.ssl.policy.https.base_directory=/etc/certs
dbms.ssl.policy.https.private_key=private.key
dbms.ssl.policy.https.public_certificate=public.crt

# Cluster SSL configuration
dbms.ssl.policy.cluster.enabled=true
dbms.ssl.policy.cluster.base_directory=/etc/certs
dbms.ssl.policy.cluster.private_key=private.key
dbms.ssl.policy.cluster.public_certificate=public.crt

# Backup SSL configuration
dbms.ssl.policy.backup.enabled=true
dbms.ssl.policy.backup.base_directory=/etc/certs
dbms.ssl.policy.backup.private_key=private.key
dbms.ssl.policy.backup.public_certificate=public.crt

Looking at the cypher-shell docs (this is on the same box as the database) there doens't appear to be an easy way to tell cypher-shell about the certificate. Is there a way to enable encrypted Bolt connections without needing to have all clients (cypher-shell, java apps, etc) needing to know the certificate via JKS or other PKI implementations?

2

To elaborate, here is how I was able to do it and why it works.

Requirements:

You have set up TLS configuration in neo4j, you own certificates.

dbms.connector.bolt.tls_level=REQUIRED

Explanation

cypher-shell uses Java cacerts. If the certificate of the neo4j server you want to connect to is not known to or not trusted by cacerts, cypher-shell will not allow to connect.
Use "keytool" to import the certificate into a keystore. I am running containers, therefore I used default store with -cacerts parameter. Replace path to certificate file with your path as configured in neo4j.conf. Alias can be chosen to your liking.

Default password is "changeit" (without quotation).

keytool -importcert -alias neo4jcert -cacerts -file /var/lib/neo4j/certificates/default/cert.pem

Keep in mind, that when connecting using encryption you can either specify the protocol or specify --encryption true/false not both. must match name of certificate, it should not be IP.

cypher-shell -a bolt+s://<HOST>:<PORT>

cypher-shell -a <HOST>:<PORT> --encryption true

Expected Results:

cypher-shell -a <HOST>:<PORT> --encryption false

Connection to the database terminated. Please ensure that your database is listening on the correct host and port and that you have compatible encryption settings both on Neo4j server and driver. Note that the default encryption setting has changed in Neo4j 4.0.

cypher-shell -a <HOST>:<PORT> --encryption true​

prompt for username, password, connection succesful.

username: neo4j
password: *********
Connected to Neo4j using Bolt protocol version 4.4 at #REDACTED# as user neo4j

:slightly_smiling_face: that's it for my setup.

Further reading:

  1. https://docs.oracle.com/en/java/javase/13/docs/specs/man/keytool.html
  2. https://stackoverflow.com/questions/21076179/pkix-path-building-failed-and-unable-to-find-valid-certification-path-to-requ
  3. https://stackoverflow.com/questions/8980364/how-do-i-find-out-what-keystore-my-jvm-is-using
  4. https://stackoverflow.com/questions/63482370/unable-to-find-a-valid-certification-path-to-requested-target
  5. https://stackoverflow.com/questions/6908948/java-sun-security-provider-certpath-suncertpathbuilderexception-unable-to-find
  6. https://stackoverflow.com/questions/61541942/how-to-enable-tls-on-bolt-protocol-on-noe4j