Troubleshooting a Self-Hosted Server

Problems with the server. For a device that will not appear, see Device Not Appearing; for a connection that is refused, see Connection Denied.

Start here

$ ./bin/docker-compose ps
$ ./bin/docker-compose logs -f

Note the wrapper. ./bin/docker-compose, not bare docker compose — the wrapper assembles the compose files and the environment chain, and running the plain command gives you a different and misleading picture.

The server starts with no database

Symptom: the server cannot reach PostgreSQL, and ps shows no postgres container at all.

Cause: you ran bare docker compose. PostgreSQL comes from an overlay the wrapper appends; without it, the service is simply not in the composition.

Fix: use ./bin/docker-compose, or make start.

Cannot connect to the database

Check the variable names, because two of them are commonly written wrong:

Right Wrong
SHELLHUB_POSTGRES_USERNAME SHELLHUB_POSTGRES_USER
SHELLHUB_POSTGRES_DATABASE SHELLHUB_POSTGRES_DB

Defaults are admin / admin / main. A variable with the wrong name is not an error — it is simply ignored, and the default is used, which is why this fails in a confusing way.

The gateway will not start

With SHELLHUB_TLS_CERT_FILE or SHELLHUB_TLS_KEY_FILE set: you must set both. One alone stops the gateway from starting.

With SHELLHUB_AUTO_SSL=true: the certificate challenge requires the server to be reachable at SHELLHUB_DOMAIN. Check the domain resolves to this host and that nothing is blocking the port.

HTTPS is not listening

The HTTPS port is only bound when SHELLHUB_AUTO_SSL=true — the port mapping comes from an overlay applied only in that case.

If you terminate TLS elsewhere, this is correct: publish HTTP and let your proxy hold the certificate.

SSH connections do not arrive, but the console works

The SSH port is published by the server service, like HTTP and HTTPS. A load balancer or firewall configured only for the gateway’s HTTP ports will pass web traffic and drop SSH.

SSH is not HTTP and will not survive an HTTP proxy — it needs its own path through.

Every client appears to come from the same address

You are behind a proxy or load balancer and ShellHub is not reading the client address from its PROXY protocol header. For web traffic, SHELLHUB_PROXY is not set on the gateway, or SHELLHUB_PROXY_TRUSTED_IPS does not include the balancer. For SSH, the header does not reach the server: the balancer sends none, or decodes it itself. Enable PROXY protocol on the SSH path, pass the header through unread, and list the balancer in SHELLHUB_SSH_PROXY_TRUSTED_IPS.

This is worth fixing even if it looks cosmetic: access policy source IPs, firewall rules and the session audit trail read the client address of SSH and web terminal sessions, and password lockout reads it at web sign-in. Left wrong, a lockout on an account applies to everyone signing in to it.

SSH closes before the server banner behind a load balancer

The client prints kex_exchange_identification: Connection closed by remote host, and the server logs closed an ssh connection whose PROXY protocol header came from an untrusted peer. The balancer sends a PROXY protocol header from an address the SSH port does not trust. Add the IP from the peer field of that log line, without its port, or a range that contains it, to SHELLHUB_SSH_PROXY_TRUSTED_IPS.

Configuration changes have no effect

Check which file. .env is overwritten by upgrades and read before .env.override. Put changes in .env.override, which is read last and wins.

Check the name. A misspelled variable is ignored silently.

Restart:

$ make restart

Turning up the logs

# .env.override
SHELLHUB_LOG_LEVEL=debug
SHELLHUB_LOG_FORMAT=json

Then make restart. Put it back when you are done — debug is noisy.

Where to go next

esc