how long since the boss has been asleep so you can finally restart without them calling two seconds later cause they didn’t bother reading the scheduled downtime email
Hardware errors often cause system instability hence this is false.
How false are we talking? A couple seconds? Minutes?
A lot of windows errors are actually hardware acting up. Such as an aggressive overclock or random issues.
An operating system cannot prevent that
Are we not doing kernel upgrades?
Yeah that’s about the only time I have to do reboots at work which are 99% linux. Well the production ones anyway.
Or the other reason is my lab having power issues due to malfunctioning UPSes, faulty NEMA L6-30 plugs, janky 240v circuit breakers or… I’m beginning to think my lab is electrically cursed.
Was about to say, “or if you’re running Arch, the last time you updated the kernel or systemd version, so probably last week or summit.”
Pretty sure everybody is missing the joke. The joke is that Debian packages are so stable and stale that you likely will need a reboot before an update.
Also, it’s a joke…please patch your boxes, k?
I’ve got a patch in my boxers right now.
Oh boxes.
I got obsessed with uptime in the early 2000s, but for my desktop Slackware box. It ran a bunch of servers and services and crap but only for me, not heavy loads of public users. Anyway, I reached 6 years of uptime without a UPS and was aiming for 7 when a power outage got me.
Skill issue. Next time you can open up the computers power supply while it’s running, splice in a second power cable, and attach a UPS without powering down or getting electrocuted.
For legal reasons, /s
Not sure what your signature is supposed to do here but now I have 3rd degree burns and a fireball has engulfed my office wall
But more importantly, did your uptime get reset?
How do I check when the last power outage was if it’s connected to a UPS?
“Uptime” — aka the anxiety meter for every sysadmin.
That’s ridiculous. It’s much more complicated than that.
You need to check NUT.
Deez? 🌰🐿️
Debian admin here. Even Debian gets regular kernel upgrades that like a reboot afterwards. Security updates are more important than uptime. Also regular testing for clean recovery after a reboot is a must so a power outrage doesn’t bring any new surprises with it. Also test your backup restores regularly.
power outrage
New fear unlocked.
The sun was angry that day, my friend…
What do you use for backup restores ?
The same tools.
Even Debian gets regular kernel upgrades that like a reboot afterwards
As someone running a UPS on my ubuntu server, “uptime” represents the time since the last kernel release, and not much else.
Novice homelaber here, is this just a case of apt update & upgrade or is there different commands for security and kernel updates? Also what’s your preferred backup/restore software? Thanks!
I think you can do
apt upgrade --updatenow.‘apt upgrade -U’
Incredible that it’s not written everywhere, I always wanted to use something like this without the " update && upgrade" which looks like is not working oftentimes
Is it really not written? I saw apt upgrade --update and knew the standard shortcut would be -u, but that didn’t work so I tried -U, bingo bongo off I went.
It am quite sure in the manual, but if you just look on the interwebz tutorials every command line just writes the full shebang. So you don’t look up the manual and get flabbergasted when you see this post. btw: if you are able to guess “what the standard shortcut would be”, you are a wizard Harry 🥳
WHAT. Does this do both sudo apt update and sudo apt upgrade?
see also –autoremove
Your note is very interesting about the difference between the commands and how autoremove will automatically remove stuff before or after the upgrade is performed. Should it always be done after, or are there instances when running it before is more beneficial? Is there any need to do both like this:
# sudo apt --update --autoremove upgrade -y && sudo apt autoremove -yI can’t really imagine a benefit to
--autoremoveexcept for keeping old packages a bit longer before removing them.Eg, if you run
apt --update --autoremove upgrade -yonce a day you’ll keep your prior-to-currently-running-version kernel packages a day longer than if you ranautoremoveimmediately after each upgrade.To make things more confusing: the new-ish
apt full-upgradecommand seems to remove most of whatapt autoremovewants to… but not quite everything. 🤷
Yup
I think so. I read it a few months back, but I don’t use any apt based systems to check on.
🤯
Nope it’s just apt update & upgrade. Iirc apt tells you when the kernel was updated and needs a reboot as well.
Only if you installed the package
needrestartfull-upgrade probably a better pick
Not if you use Proxmox! One has to be careful.
Kernel updates are usually held back and need to be selected manually. E.g. apt-get install linux-image-amd64.
I prefer rsync for private backups and employ bareos in my company for all servers.
Also worth checking out restic. It’s more command line oriented and is generally stateless
I configured restic once, forget about it and saved my files because it was making backups since forever.
Oh, never heard about it. A quick research showed me that restic is a very viable solution. Thanks for mentioning it, I added it to my comment.
While researching, I also came across a fancy WebUI, which is mostly what non-CLI users want: backrest
I am using restic and backrest on my yunohost server and I really like it! It is really set up and forget for me. Only the uploads to backblaze b2 are still triggered manually. Also did a full recovery from the backblaze repo (downloaded locally) without problems.
Thanks just installed immich and I need a quality backup system.
I appreciate the link!
I’m not the person you asked the question of. I’m a fellow novice homelaber.
I use Kopia to backup my data folders and Docker container data. Works really well. The project for this weekend is to set offsite backups to be uploaded to iDrive.
When I update I use this:
sudo apt update && \ sudo apt upgrade -y && \ sudo apt full-upgrade -y && \ flatpak update -y 2>/dev/null; \ sudo apt autoremove -y && \ sudo apt autoclean && \ sudo journalctl --vacuum-time=7dYou can get rid of upgrade if you also use full-upgrade
Yeah, people that brag about uptimes are just bragging about the fragility of their infrastructure. If designed correctly you should be able to patch and reboot infrastructure while application availability stays up.
With an uptime of greater than 5 years I’m going to be concerned about the system potentially not coming back up after a reboot/power outage, especially for physical hardware
At a bank I worked at, we had an old IBM Power server which was at that point purely used for historical data. It had multiple years of uptime and was of course a good 10+ years old. When we went to take it offline, we actually just disabled the nic on the switch so we could reduce the number of powercycles it would see in fear that it would not power on anymore. Theoretically the data on it is purely historical, backed up and not needed, but there was enough question marks on each of those fronts we just played it safe
I haven’t had a kernel update on Debian that triggered the “you should restart” message in quite some time. I was under the understanding that most newer systems now use splicing at the kernel level to not require periodic reboots.
I haven’t seen it in a while either, but also, if there is a kernel update,
uname -salways returns the old kernel until a reboot.Check for the existence of the for containing packages that recommend a reboot. Debian does not do live patching like Ubuntu does. Not least because updates to firmware are usually not applied until reboot. Also even if that were the case, regular checks for healthy reboots make sense.
Or if you have a UPS and backup generator or a house battery (do these need a UPS as well still?) it will tell you how long since you setup the system.
I would suspect you would still want a UPS. I don’t think house “power” setups have the switch over speed even if they’re automatic. Most home generator setups are manual not sure about battery setups.
My home generator is automatic but you still need an ups because the transfer switch and power on process for the generator isn’t instant. Takes like 10-30 seconds depending on how cold it is and how recently I serviced the generator.
You also ideally need a higher quality ups that can handle the shitty power coming from a generator, although the overall ups doesn’t need to be as “hefty” as a result. My ups is the kind that has extra filtering and stabilization of incoming power. My old ups was a cheaper cyberpower and it died after a few months of generator usage (we lose power here roughly every 4-6 weeks, thus the auto generator). The cheaper cyberpower would be fine in the majority of home circumstances tbh otherwise.
Pretty sure batteries can be set up for ups. One of the companies I worked for had part of the power for the building as ups. They used orange outlets to mark them. They constantly had to keep telling people not to overload it.
Battery should be automatic, but yeah not certain how quick the switchover is. At least you could go for the smallest capacity UPS there is as long as it can manage the wattage you are going with.
On my Gentoo server, uptime:
- 21:47:56 up 2455 days, 15:09, 1 user, load average: 0.00, 0.01, 0.00
Solid.
Would have been double that by now if not for the fire.
You forgot to say “this is fine”, I take it?
Joking aside, I hope you didn’t lose anything. Was it a big fire?
Those who manage the dedicated server racks service kept my stuff intact. Thankfully. Just disrupted my uptime.
[User “error” since, has cost me a TB of data. “Error”, fearquoted, because it was intentional… probably unnecessary clearing of space, partly regretted since.]
I don’t know how big the fire was, happened over 1000 miles away from here.
So, it really was fine. :3
😁👍 Good to hear
step 1:
sudo apt install slstep 2: fuck up
step 3: ???
step 4: profit!!!
Alt+L in fish shell, works great. It runs the ls command without having to modify your current command that you’re typing. Or even change your cursor position.
Won’t work for me, I am more of a “;l” guy, I have the good direction, but wrong starting point 90%of the time 🥲
A must have IMO
Also ‘fuck’
https://github.com/iffse/pay-respects there is a new alternative.
Does NixOS apply kernel updates live? I can’t recall from when I used it.
Mine doesn’t. I reboot when I get a new kernel.
This is why we have UPS ;-)
Seriously, one black out and suddenly you see the need for a UPS. Now my desktop is on a USB, my work laptop and monitors are on a UPS, my homelab is on a UPS, even my modem and router are on a UPS. I just wish I could get a backup generator, but that’s not happening anytime soon.
My experience with using an UPS is that they have caised an outage every few years, which is more often than we get power outages where I live, so I didn’t replace the batteries last time the UPS took down my server, and are just running straight from the wall. It might be better with a more expensive UPS, but it’s not worth it for me.
Yeah I read up on them before I bought, APC seemed like the best. I test them at least once a year and so far I’ve only had to replace one battery. Depending on the application I paid between $80 and $180 for each, but the higher upfront cost seems to have paid off for me at least. I am a sample size of 1, your results may vary.
A hale storm earlier this year and the power outage it caused created some bizarre issue with my home server I have yet to diagnose. All of my containers and VMs corrupted in some way, so I had to restore from backup, but my file server container has some sort of permissions issue on top of that.
Honestly the brownout before the outage is almost definitely what did it, but the cost of a UPS that also protects against brownouts is well outside of my usual hobby budget so it’s hard to justify on ewaste hardware that I got a pallet of for less than what the UPS would cost used
I sometimes have power outage in winter (snow storm, ice, etc) and working from home I need a UPS ; modem cable, router, PC, monitors, are on it, it can stand ~5h
I’ve had good luck with APC. Just be ready to pay a bit more upfront. But so far in the last 6 or 7 years, I’ve only had to replace one battery.
I bought a used APC Back-UPS Pro BR1500G for $100 on market place, it was a good deal, I replaced the 2 battery inside and added 4 outside (it is supported), I’m ready!
$100 is a good deal, though with the batteries you probably paid as much as a new one. But you have extra battery life so win.
I got tired of my network puking every time the power went out for 5 seconds.
Edit- My NAS really dislikes having the power cut off.
Yep, the black outs have stopped now but for a while it was a daily occurrence. My NAS took a beating and so did my desktop. I spent a ton on ups’s to make sure that stuff was protected and bonus, I wouldn’t loose connection while on phone calls with government officials while at work… they get pissy when you suddenly drop off.
At some point when I am less busy again I think I am gonna swap back to a debian based system because my experience on arch and red hat systems just hasnt been as good (this may be because I started on Debian based systems and keep trying to use commands that dont work on the other ones out of muscle memory)
I get bored every so often and move all the important stuff to an external drive or a separate internal one and completely change my os
I am on manjaro but I have also run arch, red hat, void, mint, Debian, Ubuntu and a bunch of others that I either put on laptops or something similar as messing around with devices
Tails and slitaz have to be my favorite to run from a USB but peppermint isn’t the worst
I just did the contrary. Moved from debian to arch. After the update to trixie my network stack completely died somehow, so I’m going back to arch.
I have had minimal issues from my manjaro desktop but I just dont like it as much as my mint based systems because everything feels wrong and I can barely remember how to update my graphics drivers on manjaro vs mint where I am confident I could run my entire system mostly command line from installs to updates and random other shit that I just can’t remember how to do through arch systems because I dont run them as hard for some reason
