Field notes

The Node.js Website That Cannot Reach the Internet

A reverse proxy is often treated as the security boundary. I kept going: the Node.js process can serve the site, but the operating system will not let it reach the internet.

By Mikinho, Chief Mad Hatter 11 minute read

Start with a capability budget

Most production hardening begins with a long menu of controls. I started with a shorter question: what does this process actually need to do?

The application renders pages, calculates a star chart and answers health checks. It has no database, no upstream API, no analytics collector and no email provider. At runtime it needs to read its release, write a PID file, listen on one Unix socket and log to the journal. That is the entire capability budget.

Once I had written the list down, unrestricted network and filesystem access stopped looking like harmless defaults. They were capabilities I had never asked for and would never use, and unused capabilities are attack surface.

A website that does not need the network should not have the network.

Put the internet at the edge

The request path is deliberately uneventful:

internet -> nginx -> Unix socket -> Node.js

Nginx owns the public TCP sockets, TLS, canonical-host redirects, request identifiers and static files. The Node.js process listens only on a group-protected socket under /run; Why I Use Unix-Domain Sockets Behind nginx is the case for that socket. There is no application port to discover, accidentally expose or leave listening on every interface.

A reverse proxy is useful here, but it is not the whole boundary. If the application were compromised, an ordinary service could still call a command-and-control server, scan the local network or bind a second listener. The operating system should reject those actions even when the application asks correctly.

Deny the network explicitly

The examples below are excerpts from the [Service] section of themadhatters_web.service, the unit that runs this site. If you borrow it, rename the unit, user, group and paths together.

The core of the policy is small enough to read in one glance:

themadhatters_web.service
[Service]
User=%p
Group=%p
RuntimeDirectory=%p
RuntimeDirectoryMode=0710

IPAddressDeny=any
SocketBindDeny=any
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6

RuntimeDirectory=%p gives the socket a home that systemd creates and removes with the service. Its mode, the socket's own permissions and the handshake that lets nginx connect are the subject of Why I Use Unix-Domain Sockets Behind nginx. The rule that matters here is narrower: the process may not own an IP listener.

IPAddressDeny=any applies an IP firewall to the service's cgroup. Both inbound and outbound IP traffic are denied. SocketBindDeny=any separately rejects binding IPv4 and IPv6 socket addresses. The Unix socket remains available, which is the one transport the application was designed to use.

RestrictAddressFamilies closes off unrelated families such as packet, netlink and Bluetooth sockets. I still list AF_INET and AF_INET6 because the runtime may create those socket types internally; the cgroup rules decide that no IP traffic may pass and no IP port may be bound. This is deliberate overlap, not three spellings of the same control.

Make the release read-only

Network denial limits where an attacker can go. A read-only filesystem limits what can be changed while they are looking around. In this layout, the deployed application lives under /srv; /opt is reserved for the separately versioned Node.js runtime.

themadhatters_web.service
ProtectSystem=strict
ProtectHome=yes
ReadOnlyPaths=/srv/%p/current
ReadOnlyPaths=/srv/%p/releases
ReadWritePaths=/run/%p
PrivateTmp=yes
PrivateDevices=yes

ProtectSystem=strict makes the service's filesystem view read-only. The release paths are named again because they are the boundary I care about most, and ReadWritePaths opens only the runtime directory that holds the PID file and Unix socket. Temporary files go into a private namespace; device access is reduced to the minimal private set.

This changes a common deployment assumption. The application cannot patch its own files, upload into its checkout or turn a runtime compromise into a modified release that survives restart. Deployment is the only writer. That is simpler to reason about than asking every code path to remember where writes are acceptable.

Remove privilege and kernel reach

The rest of the unit narrows what a compromised process can ask the kernel to do:

themadhatters_web.service
CapabilityBoundingSet=
NoNewPrivileges=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources @mount

ProtectKernelLogs=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
LockPersonality=yes

The empty capability bounding set gives the service no Linux capabilities. NoNewPrivileges makes that decision sticky for the process and its descendants: executing a setuid program or a binary with file capabilities cannot turn into an escalation path.

The syscall policy starts with systemd's @system-service group, then removes privileged, resource-control and mount operations. The remaining directives hide or protect kernel state and prevent the process from creating new namespaces, changing its execution personality or manufacturing SUID files. Every one of these is testable. None should be copied into an unrelated service without testing its real workload.

Keep the honest exception

One line in the unit is intentionally less impressive:

themadhatters_web.service
# V8's JIT generates machine code and then executes it.
MemoryDenyWriteExecute=no

MemoryDenyWriteExecute=yes is a strong control for daemons that never need dynamically generated executable code. I turned it on, and the service stopped working. Nothing in the failure pointed at the directive, so I spent hours in the man pages before I found the reason: V8 writes machine code into memory and then jumps to it, which is exactly what the directive forbids. I turned it off, wrote the comment above the line, and left the score where it was.

Hardening isn't a scavenger hunt for the highest score. A visible, justified exception is better than a control that quietly disables production or a claim that the unit is stricter than it really is. The other boundaries still matter, and the exception is obvious when the next maintainer revisits the runtime.

Add a second axis with SELinux

systemd describes the sandbox around the service. SELinux describes which labelled subject may act on which labelled object. I use both because they answer different questions.

The launcher transitions the process into an application-specific domain. Release content, runtime files and the Unix socket have distinct types. The web server's domain may connect to that socket; unrelated services may not. The application domain may read its release and use its runtime directory; it does not inherit the broad permissions of a generic unconfined daemon.

That policy caught me before it caught anyone else. My deploy replaced the current symlink and left it with the wrong SELinux type, and systemd failed the service at its WorkingDirectory step. Unix permissions were fine; the label wasn't. The tempting fix was a permissive rule. Instead I restored the expected context and made relabelling part of both activation and rollback, so the next deploy could not repeat my mistake.

If rollback does not restore labels as well as bytes, it is not a complete rollback.

Verify behaviour, not decoration

A hardened-looking unit file is not evidence that the deployed service is hardened. My checks work outward:

  1. Parse the unit and inspect its effective settings.
  2. Confirm SELinux is enforcing and the process, release and socket carry the expected types.
  3. Confirm Node.js owns a Unix socket and no TCP listener.
  4. Exercise liveness, readiness and the public HTTPS page.
  5. Reload the service and prove in-flight serving recovers cleanly.
  6. Inspect the journal and audit log for unexpected denials.

These shell commands use the resolved name, themadhatters_web; %p is expanded by systemd inside the unit file.

systemd-analyze verify /etc/systemd/system/themadhatters_web.service
systemd-analyze security themadhatters_web.service
systemctl show themadhatters_web.service \
    -p IPAddressDeny -p SocketBindDeny -p ProtectSystem

ss -lxnp
ss -ltnp
curl --unix-socket /run/themadhatters_web/themadhatters_web.sock http://localhost/~/ready
ps -eZ | grep themadhatters_web
ls -lZ /srv/themadhatters_web/current /run/themadhatters_web/themadhatters_web.sock

systemd-analyze security is useful as a checklist and regression signal, not a proof. Its score covers systemd's own service controls; it cannot understand the SELinux policy, the reverse proxy, the application's data flows or whether the site actually works. The test that matters is the composed system.

Make hardening maintainable

The unit, SELinux policy, proxy configuration and verification scripts live with the application. Focused tests assert the consequential directives: network denial, the empty capability set, read-only paths and the documented V8 exception. A deployment promotes an immutable release, switches one symlink, restores its context and runs the same checks before declaring success.

That arrangement matters more than the length of the unit file. Security settings that exist only in a production administrator's shell history will drift. Settings reviewed beside the code evolve when the runtime evolves, and a breaking change becomes a failed test instead of an archaeological discovery.

What this buys

None of this makes the Node.js application bug-free. It changes the value of a bug. A compromised worker cannot call home over IP, open a surprise port, rewrite its release, browse home directories, load a kernel module or acquire new privilege through a helper. It can still misuse the capabilities I deliberately left: reading public application code, consuming bounded CPU and memory, logging, and answering through its Unix socket.

That's least privilege in a form I can operate: not “the service probably won't,” but “the service is not allowed to.” For a small site with no runtime dependencies, the strict policy was not expensive. The hard part was noticing how much authority the default service never needed.

Sources and further reading