Back up
Save the database and settings before replacing the server or changing the setup.
I’m currently an IT Assistant, and I’m working towards a Systems Administrator role.
WackFM started as something for me and my friends to use while gaming. Moving it to AWS gave me a reason to get more comfortable with Linux, Docker, networking, and keeping data safe when making changes.
I wanted this page to show how I work through problems: checking the logs, narrowing down what’s failing, making a change, and testing the result. That’s the experience I’m building on as I work towards my next role.
We use WackFM to play music while gaming.
I used it to get hands-on with AWS.
I’m Warren. I adapted an existing Discord music bot for our group, moved it from a friend’s PC to AWS, and worked through the problems along the way. Here’s what I changed and how I checked it.
DISCORD MUSIC BOT
We wanted music while gaming.
I wanted a useful project to learn AWS.
My friends and I wanted background music in Discord, with everyone able to add songs without constantly managing the bot.
The starting point was Vocard, an open-source Discord music bot ↗. I changed how requests and autoplay work, added a manual update tool, and moved the bot and its supporting services to AWS.
Along the way, I dealt with a Docker disk taking up about 200 GB, a bot that was online but couldn’t play music, and an updater that rolled back a working deployment. Those problems taught me more than simply getting the containers to start.
Each service runs in its own Docker container.
Caddy provides the public web address.
A simplified diagram of the setup. It does not show live status.
A new request should change what’s coming next, without pushing someone else’s request to the back.
Request the first song. Recommendations keep the session going when manual requests run out.
If two people request songs, the first request plays first. Both go ahead of autoplay, and the current song keeps playing.
Requesting a song removes the old autoplay suggestions, but keeps everyone’s requests. New suggestions follow the latest song played.
Try adding a request to see how the queue responds.
These are the problems we actually ran into.
Here’s how I narrowed them down and checked the fixes.
At first, every new request went straight to the front. If someone requested a second song, it pushed the first person’s song back. Autoplay could also leave a queue full of the previous genre.
I traced how manual requests and autoplay tracks entered the same queue. They needed different treatment: a user’s request should get priority over autoplay, without overtaking another user’s request.
Manual requests now stay in arrival order. Autoplay tracks carry their own marker, so a new request can clear only the old recommendations. New recommendations use the latest played song.
We tested it in Discord with multiple requests. They stayed in order, the current song kept playing, and the recommendations changed after the new song.
The Windows host had only 1.7 GB free, and Docker would no longer start. The bot itself was much smaller, so the size of its image didn’t explain the disk usage.
I compared the virtual disk’s maximum capacity with its actual file size, then inspected the filesystem read-only. It reported about 201 GiB used, but the directory scan could account for only about 9 GiB and hit damaged file metadata. A filesystem check also found allocation inconsistencies.
We preserved the bot’s settings and database outside Docker, rebuilt the Docker environment, and restored the services. I also set rotating container logs and turned off the bot’s duplicate file log.
After the rebuild, Docker’s disk was about 4.5 GiB and Windows had about 199 GiB free. We checked the log limits on all six containers. The original cause of the growth was never fully proved; log rotation alone isn’t a guarantee against it happening again.
After moving to AWS, the bot connected to Discord before Lavalink was ready. It looked online, but couldn’t play a requested song. A manual restart got playback working.
I checked the startup logs for the bot and Lavalink separately. A running container wasn’t enough: the audio service needed to finish starting, and the bot needed a usable connection to it.
I added a Lavalink health check and made the bot wait for healthy dependencies. I also changed the connection handling so it retries when the node isn’t available and recovers after a closed connection.
Startup and reconnection checks passed, and we tested playback in Discord. That gave us a better check than simply looking for an “Up” status in Docker.
The update script built the image and passed its checks, but then timed out waiting for startup connections. It restored the previous image even though Docker showed the bot running.
I read the saved update log instead of repeatedly running the update. Compose had left the existing container running, while the verification step was looking for fresh startup messages that hadn’t been produced.
The updater now forces a fresh startup for deployment and shows each connection-check attempt. It saves a rollback image first, and restores it if startup cannot be verified.
The next saved log showed the connection checks passing and “Update succeeded.” We then tested the changed queue behaviour in Discord. This also showed why the last warning on screen wasn’t necessarily the cause of the failure.
Around the first public dashboard rollout, playback stopped working. Reverting the dashboard and restarting the server didn’t bring it back, so I couldn’t treat the timing as proof of the cause.
I tested search, direct track lookup, authenticated stream retrieval, and the cipher helper separately. I also compared the same YouTube request from AWS and a local PC. The results differed. Logs revealed a separate recommendation loop repeatedly requesting the same mix.
I guarded against retrying recommendations when no usable tracks were added. On the clean replacement server, I used a verified YouTube plugin and client configuration, then checked lookup and streaming before starting the bot. The dashboard was added afterwards and tested again.
Track lookup and media retrieval passed on the replacement. We then tested a song through the HTTPS dashboard, checked its queue entry, and confirmed audible playback in Discord. The exact cause of the earlier failure is still unproven.
I could no longer connect to the EC2 server from PowerShell. The connection timed out even though the instance was still running.
I checked the AWS security group and compared its allowed SSH source with my current public IP. My IP had changed, so the existing rule no longer allowed my connection.
I updated the SSH rule to allow the new address only. There was no need to rebuild or restart the server, or open SSH to everyone.
I connected again over SSH and confirmed access worked. The rule stayed restricted to one administrator IP.
Starting the containers is one check.
Hearing the music is the final one.
Save the database and settings before replacing the server or changing the setup.
Build the changed code into Docker images. Keep tokens and passwords out of Git.
Check the database, dashboard, and Lavalink are ready, then check the bot’s connections.
Request a song, listen in Discord, and let it finish to check autoplay.
What I’d work on next Automatic failover between AWS and the PC is still an idea, not something I’ve built. I also need to keep an eye on AWS costs and storage as the project grows.