Developing on a local-remote DevContainer
Two computers, one container
I was helping a co-worker tidying up its project the past week. Everything seemed fine and we were on-schedule, but life tends to complicate things when you least expect it. She had to travel 1,000 Km away the next week at the same time the project had to hit some critical milestones.
While she was making the arrangements for traveling, I was tasked with resolving a simple problem: let her connect to her project in a DevContainer from her computer office, remotely.
I set up a VPN Server through WireGuard on their offices for this occasion—you can do the same with TailScale or HeadScale — but you can do anything you need to make the computer reachable from the outside. Once done, the next step was to make the DevContainer available through SSH. The problem? DevContainers are meant to be used either locally, or through a Cloud provider.
They’re not meant to be usable from computer-to-computer, even in the same local network, but that wasn’t going to stop me.
DevContainers Standard I/O
DevContainers are essentially “Docker” containers with some SSH magic. The provisioner, which can be VSCode, JetBrains IDE, DevPod, or even the official devcontainer-cli, creates the container but also an on-demand SSH server with appropriate credentials inside, opaquely to the developer.
For example, DevPod uses their own CLI which embeds its own SSH client and server, which gets copied inside the container. When you make an SSH connection to the Workspace (the container) created by DevPod, it creates a tunnel to the appropriate workspace on demand, as shown by the ProxyCommand parameter in the SSH configuration of the user, without the need of exposing a port or a socket.
How this is possible, if there is no port or socket exposed to host? Well, DevPod (and presumably every other provisioner) uses a neat trick that makes local DevContainers secure and private by default: sending and receiving data using stdin and stdout, respectively.
In other words, you can only connect to the DevContainer in your local machine.
One does not simply walk into Mordor™
Since the provisioner is the one responsable of setting the SSH Server, without it you have to do it manually. Plus, you will need the public IP or hostname of the container you want to connect, and the appropriate credentials (user and password). Even if the SSH Server is up, you cannot use it outside your computer until you expose a port from the container — typically 22.
We need our own SSH Server, and we have many alternatives. I decided to reuse DevPod itself because when it provisions the workspace it injects its CLI into the container that already offers an SSH Server.
Just be sure to expose the port of your container. You can do it in the Dockerfile, Docker-Compose, or the DevContainer JSON file — the latter usually doesn’t work because some DevContainers provisioners do weird stuff.
DevPod no dice
We need to change the SSH Server of DevPod CLI to actually listen a port instead of using stdin/stdout. Luckly, the command has an option to do just that, which uses address 0.0.0.0:22 by default.
❯ /usr/bin/devpod-cli helper ssh-server --help
Starts a new ssh server
Usage:
devpod helper ssh-server [flags]
Flags:
--address string Address to listen to (default "0.0.0.0:8022")
-h, --help help for ssh-server
--stdio Will listen on stdout and stdin instead of an address
--token string Base64 encoded token to use
--track-activity If enabled will write the last activity time to a file
--workdir string Directory where commands will run on the hostOur luck runs short immediately because this command is hardcoded. In other words, the command that creates the SSH Server in the container cannot be configured and it will always use stdin. Heresy!
If we’re going to make an SSH Server within a DevContainer, we can reuse the DevPod CLI in the container and run it when the container starts using the PostStartCommand property of the DevContainer JSON.
Assuming we’re forwarding the port 22 of the container to the 2222 port of the host, we can easily call the server and detach the command from the shell.
{
"postStartCommand": "/usr/local/bin/devpod helper ssh-server --address 0.0.0.0:22 &>/dev/null & disown"
}There are three tricks here:
-
&>/dev/nullsets the command’sstdoutandstderrto/dev/nullinstead of inheriting them from the parent process.
-&makes the shell run the command in the background.
-disownremoves the “current” job, last one stopped or put in the background, from under the shell’s job control.
If we don’t detach the command from the provisioning process, the provisioner will think the setup hasn’t been finished.
In case the above doesn’t work, you will have to make a script to load the DevPod SSH Server when the container starts. If you’re using Laravel Sail, my suggestion would be to publish the configuration files and add the SSH server as part of the supervisord.conf.
DevContainer features to the rescue
DevContainer Features are basically scripts that do something in the container after it’s created. Most of these install utilities and tools.
There is one feature to officially add an SSH Server, but didn’t work for me. Anyway, I’m sharing how it works, which is rather simple.
You only need to add the feature to your devcontainer.json file (or whatever you are using) and some environment variables to allow creating the proper user and password. I’m pretty sure the remoteEnv key should work, but if not, use containerEnv instead.
"remoteEnv": {
"SSHD_PORT": "2222",
"USERNAME": "sail",
"NEW_PASSWORD": "skip",
"START_SSHD": "true",
},
"features": {
"ghcr.io/devcontainers/features/sshd:1": {}
},That should be it for adding an SSH Server to a DevContainer without DevPod.
Another alternative is to edit your Dockerfile to install the OpenSSH package that includes sshd, the SSH Server. Then, you may use supervisord or S6 Overlay to start PHP (or whatever you need) along the SSH Server, or the server alone.
Turn it on at startup
The computer on the office uses the BIOS/UEFI RTC Alarm to turn on and turn off, instead of hoping a human doesn’t forget to press the power-on button of the computer. Mac users can also configure this via Terminal, so search accordingly.
What we want now is to turn on the DevPod Workspace so it’s ready to accept connections. If you’re not using DevPod, you should use Laravel Sail command to bring up the DevContainer, the devcontainer-cli (that you would install separately) or just Docker.
# DevPod Workspace
devpod-cli up big-project
# Laravel Sail
~/Projects/big-project/vendor/bin/sail up
# Docker
docker container start -d big-project
# Docker Compose
docker compose --project-directory ~/Projects/big-project up -d DevPod SSH Fingerprint changing every time
The DevPod SSH Server will change its fingerprint every time the container starts.
The source code shows using a given “token” that is just a JSON object with some data, encoded in BASE64.
It’s not documented how to get or generate a key, and not even if we use the key would solve the problem, so you will have to manually remove the fingerprint from your ~/.ssh/known_hosts and add it again.
ssh-keygen -R {ip-or-hostname-of-remote-computer}Alternatively, you can change your SSH Known Host configuration at ~/.ssh/config to disable strict check of keys by setting StrictHostKeyChecking key to no and just voiding the known hosts file with UserKnownHostFile /dev/null.
Host big-project
ForwardAgent yes
LogLevel error
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
HostKeyAlgorithms rsa-sha2-256,rsa-sha2-512,ssh-rsa
Hostname my-office-computer.companydomain
Port 2222
User sailIf you do that, you can just run ssh big-project as it will take care of the configuration itself.
Overall, it kinda sucks, but at least it works. This shouldn’t be a problem if you install an OpenSSH Server, since the server keys would be created when the container is provisioned. Of course, rebuilding the container would change the fingerprint.
