How do you set up a server? Do you do any automation or do you just open up an SSH session and YOLO? Any containers? Is docker-compose enough for you or are you one of those unicorns who had no issues whatsoever with rootless Podman? Do you use any premade scripts or do you hand craft it all? What distro are you building on top of?
I’m currently in process of “building” my own server and I’m kinda wondering how “far” most people are going, where do y’all take any shortcuts, and what do you spend effort getting just right.
I have a git repository with all my compose files sorted neatly into directories, i.e. my “stack”. Portainer allows adding stacks using a repository, so it’s essentially one click deployment once the compose file is on a remote git server.
I just have a pi 4 running OpenMediaVault with docker and portainer. 😅
Proxmox, then create LXC for everything (moslty debian and a bit of alpine), no automation, full yolo, if it break I have backup (problems are for future me eh)
I used to do the same, but nowadays I just run everything in docker, within a single lxc container on proxmox. Having to setup mono or similar every time I wanted to setup a game server or even jellyfin was annoying.
This.
Proxmox and then LXCs for anything I need.and yes - I cheat a bit, I use the excellent Proxmox scripts - https://tteck.github.io/Proxmox/ because I’m lazy like that haha
I’m a lazy piece of shit and containers give me cancer, so I just keep iptables aggressive and spin up whatever on an Ubuntu box that gets upgrades when I feel like wasting a weekend in my underwear.
Fedora-server with Podman and Quadlet on btrfs drives. Although I must admit I often use rootful mode in Podman as it works better with Containers made for Docker. Ah and you might want to turn off SElinux in the beginning as it can get frustrating fast.
For a while I tried to run k8s (k3s mostly), then I did run nomad for a while. Now I am just running docker compose on Ubuntu (still have one box running Proxmox, but that will be decommissioned eventually, and mostly just runs one VM running Ubuntu).
I am building a few things to solve specific problems I have with this:
- Some basic ansible scripts to set up ssh, users, basic packages, etc
- Docker label-based service discovery/announcement that traefik can consume. (currently working!)
- Deployment: getting the compose files, config files, and docker images to the right machine and getting them running. (in progress)
- At some point I will probably get around to automating deployment of the rest of the above via Ansible when it is more stable.
Most of my server hardware is oriented toward having a bunch of disks plugged into them (I am 100% guilty of being a data hoarder), and I am running gluster to glue that all together, so that is something I install onto the servers to share their physical disks and/or mount the logical disks.
If doing a fresh server external, I’d go for debian as base(don’t need to update it too often + stable)
For apps it’s mostly docker-compose to set up portainer/nginx-proxy then from ther just manage the rest from portainer/nginx-proxy web-ui. ony log on the server for the occasional docker updates / pruning for space.
I see a lot of guys going the full kubernetes route and it’s something I’m hoping to get into at some point but it seems like a lot to unpack for now.
I use a heterogeneous environment with some things hosted in various cloud providers and others locally. Often times, I can usually find the package I need - but if I can’t, I usually go for Docker and docker-compose. This is often the case in Oracle Linux on OCI - where docker just makes things so much easier.
For my static stuff I just use Cloudflare Pages and forget about it.
On my homelab it is Arch Linux with my own set of scripts. I used to do VFIO gaming a lot (less now), so I had the host only be a hypervisor and used a separate Arch VM to host everything in a docker-compose stack. The VM makes my server operations a lot more tidy.
My RPI is using dietpi and is natively running the pihole software and a couple other things.
I know some folks swear by UnRaid and Proxmox, but I’ve always found those platforms limited me vs building things my way. Also borking my own system unintentionally on occasion is a thrilling opportunity to learn!
I have a base Debian template with a few tweaks I like for all my machines. Debating setting up something like terraform but I just don’t spin up VMs frequently enough to wan tto do that. I do have a few Ansible playbooks I run on a fresh server to really get it to where I want though.
K
Ansible and docker compose.
Xen on Gentoo with Gentoo VMs. I’ve scripted the provisioning in bash, it’s fairly straightforward - create lvm volume, extract latest root, tell xen whick kernel to boot.
Ideally would like to netboot a readonly root off nfs and apply config from some source. Probably bash :D
Some things like opnsense are much more handcrafted because they’re a kind of unicorn compared to the rest of the stuff.
That’s impressive effort for a home lab.
I run Debian + Docker, and use Portainer to manage the docker stacks
Proxmox and shell scripts. I have everything automated from base install to updates.
All the VMs are Debian which install with a custom seed file. Each VM has a config script that will completely setup all users, ip tables, software, mounts, etc. SSL certs are updated on one machine with acme.sh and then pushed out as necessary.
One of these days I’ll get into docker but half the fun is making it all work. I need some time to properly set it up and learn how to configure it securely.
About two years ago my set up had gotten out of control, as it will. Closet full of crap all running vms all poorly managed by chef. Different linux flavors everywhere.
Now its one big physical ubuntu box. Everything gets its own ubuntu VM. These days if I can’t do it in shell scripts and xml I’m annoyed. Anything fancier than that i’d better be getting paid. I document in markdown as i go and rsync the important stuff from each VM to an external every night. Something goes wrong i just burn the vm, copy paste it back together in a new one from the mkdocs site. Then get on with my day.
