Troubleshooting a Self-Hosted Server

Problems with the server. For a device that will not appear, see Device Not Appearing; for a connection that is refused, see Connection Denied.

Start here

$ ./bin/docker-compose ps
$ ./bin/docker-compose logs -f

Note the wrapper. ./bin/docker-compose, not bare docker compose — the wrapper assembles the compose files and the environment chain, and running the plain command gives you a different and misleading picture.

The server starts with no database

Symptom: the server cannot reach PostgreSQL, and ps shows no postgres container at all.

Cause: you ran bare docker compose. PostgreSQL comes from an overlay the wrapper appends; without it, the service is simply not in the composition.

Fix: use ./bin/docker-compose, or make start.

Cannot connect to the database

Check the variable names, because two of them are commonly written wrong:

Right Wrong
SHELLHUB_POSTGRES_USERNAME SHELLHUB_POSTGRES_USER
SHELLHUB_POSTGRES_DATABASE SHELLHUB_POSTGRES_DB

Defaults are admin / admin / main. A variable with the wrong name is not an error — it is simply ignored, and the default is used, which is why this fails in a confusing way.

The gateway will not start

With SHELLHUB_TLS_CERT_FILE or SHELLHUB_TLS_KEY_FILE set: you must set both. One alone stops the gateway from starting.

With SHELLHUB_AUTO_SSL=true: the certificate challenge requires the server to be reachable at SHELLHUB_DOMAIN. Check the domain resolves to this host and that nothing is blocking the port.

HTTPS is not listening

The HTTPS port is only bound when SHELLHUB_AUTO_SSL=true — the port mapping comes from an overlay applied only in that case.

If you terminate TLS elsewhere, this is correct: publish HTTP and let your proxy hold the certificate.

SSH connections do not arrive, but the console works

The SSH port is published by the server service, not the gateway. A load balancer or firewall configured only for the gateway’s HTTP ports will pass web traffic and drop SSH.

SSH is not HTTP and will not survive an HTTP proxy — it needs its own path through.

Every client appears to come from the same address

You are behind a proxy or load balancer and ShellHub is not reading the client address from its PROXY protocol header. For web traffic, SHELLHUB_PROXY is not set on the gateway, or SHELLHUB_PROXY_TRUSTED_IPS does not include the balancer. For SSH, the header does not reach the server: the balancer sends none, or decodes it itself. Enable PROXY protocol on the SSH path, pass the header through unread, and list the balancer in SHELLHUB_SSH_PROXY_TRUSTED_IPS.

This is worth fixing even if it looks cosmetic: access policy source IPs, firewall rules and the session audit trail read the client address of SSH and web terminal sessions, and password lockout reads it at web sign-in. Left wrong, a lockout on an account applies to everyone signing in to it.

SSH closes before the server banner behind a load balancer

The client prints kex_exchange_identification: Connection closed by remote host, and the server logs closed an ssh connection whose PROXY protocol header came from an untrusted peer. The balancer sends a PROXY protocol header from an address the SSH port does not trust. Add the IP from the peer field of that log line, without its port, or a range that contains it, to SHELLHUB_SSH_PROXY_TRUSTED_IPS.

Configuration changes have no effect

Check which file. .env is overwritten by upgrades and read before .env.override. Put changes in .env.override, which is read last and wins.

Check the name. A misspelled variable is ignored silently.

Restart:

$ make restart

Turning up the logs

# .env.override
SHELLHUB_LOG_LEVEL=debug
SHELLHUB_LOG_FORMAT=json

Then make restart. Put it back when you are done — debug is noisy.

Where to go next

esc