I don't think the cheapest APC Back-UPS units can be monitored except in Windows

https://strugglers.net/~andy/blog/2024/02/23/i-dont-think-the-cheapest-apc-back-ups-units-can-be-monitored-except-in-windows/

by trelane • 2 years ago

191 249 2 years ago

strugglers.net

ethbr12• 2 years ago

> TL;DR: Despite otherwise seeming to work correctly, I can’t monitor a Back-UPS BX1600MI in Linux without seeing a constant stream of spurious battery detach/reattach and power fail/restore events that last less than 2 seconds each.

It sounds like the Windows software probably debounces the incoming events, only bubbling up those that are sufficiently time-stable, to account for bargain sensor false positives?

Edit: Looks like the apcupsd collection also has some EEPROM programming utilities... (for supported devices) http://www.apcupsd.org/manual/#configuration-directives-used...

bell-cot3• 2 years ago

I'd bet you're right. Experience at $Job is that certain cheapo UPS's can be miserably twitchy about detecting power problems and switching to their inverters. After a week of the clickity-clicks of the cut-over relays - $Employee and his office mates may ask for the UPS to be removed.

bulk88• 2 years ago

Right, the window tool debounces the events, IMO the battery attach/disconnnect messages are a poor mans heartbeat signal on the protocol. That ACPI/Windows didn't "sleep" the USB port, or that the controller in the UPS (chinese junk) didn't crash.

bell-cot3• 2 years ago

Our power grid is very reliable. Looking at availability information from “uptimed“, my home file server has been powered on for 99.97% of the time in the last 14 years. That includes time spent moving house and a day when the house power was off for several hours while the kitchen was refitted!
However, in December 2023 a fault with our electric oven popped the breaker for the sockets causing everything to be harshly powered off. My fileserver took it badly and one drive died. That wasn’t a huge issue as it has a redundant filesystem, but I didn’t like it.
I decided I could afford to treat myself to a relatively cheap UPS.

Just from that, I'd say he for-sure should have avoided UPS's, and gotten a "real" surge protector. The "shockwave" from that high-amperage short is far more likely to have done the damage than the mere loss of power. And the output of cheap UPS's is often far worse than the input.

[Edit: And good surge protection is never a feature of cheap UPS's.]

jeffbee• 2 years ago

The quoted material sounds like a poor analysis. It doesn't take into account all the times the UPS would have failed while the grid was fine. UPSs are not pure goods with no downsides that merely add to availability. Commercial UPS are very unreliable, bordering on a scam.

Symbiote• 2 years ago

It also doesn't account for the fire risk — a cheap UPS is much more likely to catch fire than a PC / NAS.

gnuser• 2 years ago

Sine-Wave UPSs are the only thing I allow in my DCs for a reason! (don’t forget the generators either!!!)

Night_Thastus3• 2 years ago

Pure Sine-wave UPSs are massive overkill and offer no advantage for any normal consumer.

If you have incredibly power-sensitive $100k equipment that needs protecting, maybe. But not for normal consumer electronics. None of them are that picky about the power.

criley2• 2 years ago

We recently upgraded to a whole house surge protector installed at the panel. They're not that expensive ($100-$200 around me, depending) and with the reduced quality of power we're seeing lately here and the damage it has done to our circuit breakers, it's become a necessity.

da7683• 2 years ago

And after the UPS will inevitably fail to power on, he'll probably want to add an automatic transfer switch to the mix.

ssl-36• 2 years ago

Can you be more precise about how a high-amperage short circuit can produce a "shockwave" of a form that would be ameliorated by a "real" surge protector?

bell-cot3• 2 years ago

Can you be more precise...

Apologies - I know enough to do long-winded hand-waving on the subject...but if you want precise and concise (and really accurate), you'll need to ask a real electrical engineer.

javawizard• 2 years ago

This Wikipedia article may point you in the right direction: https://en.m.wikipedia.org/wiki/Voltage_spike

I'm not an electrician by any means so take everything I saw with a grain of salt, but: in general, electricity flowing through a system has some amount of "inertia" (the technical term for this is inductance), and shutting off a circuit breaker will cause a brief voltage spike upstream of the breaker due to the flow of electrical current suddenly being forced to stop.

Have you ever unplugged a running appliance from the wall and noticed a brief spark at the plug? That's the same effect: motors in general have high inductance and try quite hard to fight changes in current.

It's bad enough that when designing circuits that switch power to inductive loads like motors, you have to include a flyback diode (read: one way valve) to allow a path for electricity to continue to flow internal to the load when the power is switched off, otherwise it will generate a voltage spike high enough to damage the switching circuit.

Put another way: it's not the "high amperage short circuit" that causes the voltage spike; it's the rapid stop to said high current that happens when the circuit breaker realizes what's going on and trips.

Saris• 2 years ago

I'm curious how a voltage spike would have gotten through the computer PSU conversion to DC, and then only damaged 1 drive somehow?

dijit• 2 years ago

it hit all the drives, but it’ll affect things unevenly

tgsovlerkhgsel• 2 years ago

I assume the drive failed due to the power cycle, not due to electrical damage.

Especially if it was 14 years old.

justinclift• 2 years ago

As a side note with UPS's, it can somewhat informative to look at the pricing for their replacement batteries.

At least here in Australia, replacement batteries for some of the common UPS's (for desktop usage) cost nearly as much as the UPS itself.

alright25653• 2 years ago

From my own adventure in this area, the battery is very standard and IMO reasonably priced. What gets expensive is shipping it, since lead is so heavy. I remember having trouble finding local sellers for them that would have avoided the shipping cost.

twisteriffic• 2 years ago

I've found home security companies to be a good source, either of the batteries themselves or at least a local supplier who can get what you need.

TMWNN• 2 years ago

Also consider how the size of the UPS affects the battery replacement cost. I purchased my most recent UPS, refurbished by CyberPower, from its eBay store <https://www.ebay.com/sch/i.html?_ssn=cyberpower>. For a few dollars more I could have bought a larger size, but

* I didn't need more capacity (the one I bought will power the server it serves for an hour, far more time than I need or want) and, more to the point,

* The larger-size model has two batteries to replace, significantly increasing the cost.

Libcat99• 2 years ago

I wish the battery layout/ amp hours was more obvious in UPS sales listings. The watts/VA are always front and center, but generally I want to know runtime which is harder to find and compare.

Johnny5552• 2 years ago

I never buy APC replacement batteries - third party batteries are much cheaper, and seem to work just as well based on testing by the UPS itself.

Springtime2• 2 years ago

Also these brands are just using off the shelf batteries anyway (eg: a Cyberpower model using two 12v/8Ah SLA batteries connected together[1], with various OEM brands seen on the labels). Searching online discussions one finds which OEMs make better quality batteries.

[1] One such example, replacing with a generic kind: https://www.youtube.com/watch?v=R6qcon76i4s

solraph• 2 years ago

For what it's worth, I was gifted one of these. A replacement battery was $45 from Jaycar.

Com port is a standard USB-B port and is compatible with but

https://www.telephonewreckers.com.au/power-shield-safeguard-...

My only complaint is the formfactor is about weird.

jonathantf2• 2 years ago

I own two Back-UPS 1400VA units, one monitored with apcupsd and the other with a MikroTik router - so I guess it's just the newer units that have this issue?

1970-01-01• 2 years ago

I then asked about this on the apcupsd mailing list. The first response:

“Something’s wrong with your..."

"Returning to my thread on the apcupsd mailing list, I asked again if there was actually anyone out there who had one of these working with non-Windows."

Linux's reputation in a nutshell. It just works after you spend days checking if you have a bad device and another on the mailing list. And then you are forced to find a working alternative.

ethbr12• 2 years ago

Hardware validation is time consuming, boring, and expensive.

Not surprising that given a mostly volunteer effort... it just doesn't happen. Or vendors self-certify their own gear.

thayne4• 2 years ago

It's not like Microsoft or Apple do that either. But Vendors are more thorough with testing on those OSes.

gpderetta2• 2 years ago

typically you check for linux compatibility before buying the actual hardware/software.

ziddoap• 2 years ago

There’s an entry in nut‘s hardware compatibility list for “Back-UPS (USB)” of “supported, based on publicly available protocol”. I made the order.

angry_moose• 2 years ago

I probably spent 20 hours this week trying to get a not-that-obscure piece of software that explicitly states "Debian 12" support on their website working on a freshly-installed (as in - install Debian 12, update system, install software); and it still just doesn't work. After hammering my way through 4 different errors, and hopelessly out of date (and wrong) documentation, I finally got them to issue a refund this morning.

"Linux Compatibility" often means "someone got it working once 6 years ago, on an extremely out of date version of the OS, but its probably still fine".

bravetraveler• 2 years ago

Once you learn a few soft guidelines (not rules, nothing is perfect), you don't really even need to research.

Intel/AMD: probably fine. Realtek: works, one driver codebase for everything means weirdness is likely. Broadcom: don't even bother. Nvidia/mobile: what kernel/stack are you intending to use?

I haven't had to care about compatibility with desktop or server components in over a decade.

It's fingerprint readers and dual-GPU laptops that really bum a lot of people out.

justsomehnguy2• 2 years ago

Ah, yeah, the typical "you should had checked for the compatibility, it's you who are the problem, not Linux".

And yes, I never check for Windows compatibility before buying the actual hardware/software. Not even /s.

lbotos• 2 years ago

I mean -- I built a PC a few months ago and formatting the windows disk from a non windows env was not trivial.

So yes, Linux is still not on par with Windows in many ways, but boy oh boy was it weirder than it should have been to format a USB drive to install windows.

I lost a solid 6 hours to that try, yay it work, wait I'm getting a nonsensical error at a strange screen. Retry with different flags, with different hardware, etc etc etc.

skirmish• 2 years ago

Ditto, I was building a Windows desktop and spent hours faffing around attempting to follow official Microsoft instructions even with another Windows computer available; in desperation tried Rufus Installer, worked on the first attempt. Then months later was building another desktop and went straight to Ventoy installed from Linux and just dropping a downloaded ISO on the USB stick -- worked immediately.

HumblyTossed3• 2 years ago

Or...

Don't buy a consumer grade UPS if you want to monitor it properly.

nickt• 2 years ago

Any suggestions?

Is there a power delivery equivalent of Ubiquti (I know we’ve discussed their shortcomings here) that offer better than consumer grade for less than enterprise prices?

NotYourLawyer• 2 years ago

Not just that, but also “this is all your fault for buying cheap trash!”

kayodelycaon• 2 years ago

I have what was the cheapest APC UPS years ago and the macOS software works fine.

Doesn’t help anyone running Linux though. :/

e402• 2 years ago

I have an APC on macOS, too, and as of a couple of OS iterations ago it recognizes it natively. I used to run apcupsd on macOS, but it was quite tricky to keep it working, due to the USB interface. This was on an older Mac than I'm using now (the trashcan Mac Pro), but it mostly worked and I would always have an indication when it stopped working as the icon in the menubar would change. I think the usual issue was so lock file was owned by root and couldn't be deleted.

dooglius• 2 years ago

If the vendor gives a proprietary Windows application for monitoring then... of course? What did you expect?

criddell• 2 years ago

There's a USB standard for HID power devices. I would expect basic functionality from any UPS with a USB port.

mrlonglong2• 2 years ago

I have one of these APC Back-UPS 900 (BR900GI) here, and they work perfectly. Fully supported under apcupsd, that device has saved my bacon quite a few times. The only thing is that you MUST use the USB cable they provide.

throwawaaarrgh• 2 years ago

Doesn't that kind of prove they're messed up, if no other USB cable works?

rincebrain• 2 years ago

APC doing horrible things with "standard" cables isn't new, though - they used a nonstandard null modem serial cable back in the day, such that if you tried to connect to the UPS using a standard null modem cable, it'd trigger shutting off the UPS.

This isn't to say it's good, just that it's not new.

rcxdude• 2 years ago

The connector on the back is essentially a non-standard serial connector, and the USB cable is a USB-serial adaptor. You can probably make another work, but only by making an adaptor cable yourself. They have some models where the USB-serial adaptor is integrated into the unit, which is generally a bit easier to use and a more typical approach.

(Generally, such a protocol is a better option from a compatibility point of view: it's usually fairly easy to reverse engineer. It would be interesting to sniff what the windows software is doing to work out what protocol or behaviour difference is responsible)

tempest_2• 2 years ago

I just bought batteries for a 20 year old UPS, still chugs along, network interface is base 10!

StillBored2• 2 years ago

I had a 20 year old APC start misbehaving on me a few months back, and new batteries didn't fix it. After messing up the repair on the first one, a second one started doing the exact same thing, and I immediately took it apart and, now an expert on how not to fix them and what the root cause was, I swapped every cap on the board that my ESR meter claimed was bad. It's now back in operation, and while I was testing it, I realized it's so inefficient that about 10% of its max rating is just going up in heat. The replacement for the first one, I assumed, would be much more efficient, and it is, but it's still dumping about 7% of its energy into heating the transformer.

OTOH, the older UPS behaves much better with apcupsd, going so far as giving me the firmware versions, battery replacement times, better power monitoring/etc. While the new one works its missing most of what the older UPS could do (although its not a APC).

So, i'm sorta on the fence about just using "backup generators" from Amazon/etc because one of the ones I have has about 5x the capacity of the new UPS and seems to be able to switch without the computer plugging into it having an issue. Plus, its power waste is less than 1%.

iforgotpassword• 2 years ago

I've had an AMD based home server a decade ago. Connected an HP printer, and it would frequently cause disconnect and reconnect events in dmesg, sometimes ending up in some error state when Linux would just give up trying to identify the device. After trying different ports and cables I eventually, out of desperation, threw in an unpowered old USB 2.0 hub and lo and behold, the connection to the printer was rock solid.

spintin2• 2 years ago

It's better to just build your own with a 12v charger, a few 12v lead acid deep discharge batteries in parallel and a 9-18v -> 12v aligner.

If you live in EU go with 110-220v charger so your setup can outlive a brownout.

alright25653• 2 years ago

How do you power a PC off this setup? Mine has a C14 inlet.

spintin2• 2 years ago

Inverters have losses (~10%), you need 12v devices for this to work.

PicoPSU is the way I solve this.

shrubble• 2 years ago

Just a warning: there are differences in the output from UPSes with the cheaper ones' output not working well with higher efficiency power supplies. Better UPSes will output less 'jagged' AC.

krunck• 2 years ago

Yeah, I ran into this once. You want to find ones that state they have true sine wave output.

folmar2• 2 years ago

If you do you shopping around getting clear sine output does not increase the price usually, or maybe a few percent.

BlueTemplar• 2 years ago

Is it me or going cheap on anything power-related seems like a terrible idea ?

Borg3• 2 years ago

Yeah, it is. I own two good old APC SmartUPS 1000 (one for desktop and second for server). They are absolutly reliable, quiet and output Sine when on battery. Good old reliable tech, easly monitored using apcupsd for example.

aaronmdjones• 2 years ago

I have my APC Back-UPS CS650 (BK650EI) plugged into my router (running OpenWrt) which works fine with Network-UPS Tools. The router shares the UPS status with the whole LAN, and the machines on the LAN that are powered by the UPS will shut down automatically when the UPS battery goes critical.

zzzeek• 2 years ago

I had some battery issues with my previous BackUPS 750 but once I got a new battery, things were fine with apcupsd. Have been on a BackUPS-Pro 1500 S for a couple of years now also using apcupsd and zero problems. it legitimately records a low power event when my Brother printer first starts up, switches to battery then comes right back. Similar happens when it's stormy here and power lines are going down and being re-routed.

simpaticoder2• 2 years ago

Can anyone recommend a good modern UPS? It seems like Lithium Ion would have the best power density, but perhaps lead-acid is actually better for this application? Good surge protection would also be great. It would be nice if there existed a UPS brand that is like Anker for USB cables and power adapters or Brother for printers.

mplewis• 2 years ago

I have had good luck with my refurbished APC BR line from ExcessUPS.

slashink• 2 years ago

The Cyberpower series sold at Costco is unironically good. No frills or cool stuff, UI from 1995 but supports USB monitoring out of the box and just… works. I was about to make the brother comparison but after reading your entire comment realize you already did it.

Springtime2• 2 years ago

Just FYI, Cyberpower, at least for their consumer line, use problematic yellow glue[1] which becomes brittle over time and can become conductive, causing arcing/fire hazards[2][3]. There was also a prior HN discussion about it[4], with some similar experiences.

It can be removed[5], with some effort, though. Given its use in various electronics it wouldn't surprise me if some other UPSes use it, though have only seen discussions about Cyberpower.

[1] https://sound-au.com/articles/yellow-glue.htm

[2] This thread had various videos from users showing the arcing on Cyberpower units: https://web.archive.org/web/20220715134034/https://old.reddi...

[3] https://www.youtube.com/watch?v=9gqBzLNMFe4

[4] https://news.ycombinator.com/item?id=31954938

[5] https://www.eevblog.com/forum/beginners/how-to-remove-the-dr...

bongodongobob7• 2 years ago

In general, a UPS is just supposed to buy you time to assess the situation and shut things down gracefully. Power density doesn't come into the equation at all. It's just supposed to give you an hour to figure out what you need to do. It's all about the lifetime of the battery. I go with Cyberpower.

Johnny5552• 2 years ago

I put an oversized UPS with extended battery pack on my network gear (Cable modem, Router, ethernet switch and Wifi nodes) to give me almost 8 hours of runtime in a power outage. I've found, however, that the cable company's UPS only lasts around 6 hours since that's how long it takes for the internet to go out and the cable modem to lose carrier.

In the winter months (when we have most of our power outages), I pay for a pre-paid cellular SIM for backup. The cell tower(s) that serves my area must be on generator, it doesn't seem to go down even in long power outages.

Night_Thastus3• 2 years ago

APC's are good once you get to the 'PRO' line. I would not get anything cheaper than that - those ones are poorly made.

folmar2• 2 years ago

Power density is not a problem usually as you don't care for the size and weight, and lead-acid give best bang for buck.

radicality• 2 years ago

Are Anker cables considered good? I have a few, but recently one of my anker usb-c cables started to only go up to 15W, making me think something broke with the usb pd chip.

yjftsjthsd-h2• 2 years ago

Yes AFAIK Anker generally has a good reputation, but what does that have to do with uninterruptible power supplies?

ianburrell2• 2 years ago

Anker is good. The limit for USB-C cables is normal ones go to 60W, marked ones go to 100W. The cables are passive except for the marker chip. It is the charger that does USB-PD. Do you have it plugged into regular charger that does 15W, or USB-PD charger that does more?

I guess you could have a bad cable where the CC wire has broken and can't do USB-PD.

ssl-36• 2 years ago

Anker products are generally good (and they were the first to make consistently-good cables and chargers -- in the early smartphone days, everything non-OEM was junk), but things still break eventually.

Their customer service is supposed to be legendary. If you drop them a (polite) descriptive note, signed with your name and address, I'll bet you a beer that you'll have a new cable in your hands by the middle of next week.

dhd4152• 2 years ago

I've found my moderately-priced Eaton 5S 1500 unit to perform considerably better than any of the non-pro APC models I used before it.

CamperBob2• 2 years ago

Look for used enterprise-grade or at least higher-end office grade equipment on eBay and other surplus marketplaces, and install new batteries. Cyberpower is a good brand in my experience.

The only advantage to buying new is weight. Newer UPSes tend to be a lot lighter, but that's often because they cut corners on the magnetics and/or use expensive, hard-to-replace lithium batteries. Weight is a high priority for the manufacturer to optimize, but not for the consumer. When it comes to UPSes, old school big-iron transformers and heavy-metal batteries are the way to go IMO.

ianburrell2• 2 years ago

Lead acid is used for most UPSes. Lithium ion is bad battery type for UPS since has lower cycles and safety, and power density doesn't matter. LiFePO4 would be perfect, with good cycles, current, and safety.

But manufacturers don't use it. Maybe they haven't redesigned after the price dropped. Or maybe cause prices is still too high. I think LiFePO4 would be most useful for rack mounts, or small UPSes that aren't possible with lead acid.

soneil2• 2 years ago

Lithium in smaller UPS is mostly stuck in a marketing gap at the moment.

Generally lithium UPS win in a 10-year TCO, but SLA wins in up-front costing. Which leaves us with two issues - most consumers look at the sticker price not the TCO, and no-one wants to market their products as "Our lead acid offer only /looks/ cheap, until you see what we charge you for consumables!"

mardifoufs• 2 years ago

Regular Li ion has more recharge cycles than most lead acid batteries that come with UPS. Even high power density liion still usually has 4-500 recharge cycles before reaching a maximum of 80% SOC vs the original capacity. With smarter power management you can get away with much more (for example, if you don't fully cycle the battery)

ssl-36• 2 years ago

LiFePO4 can be had in the "portable solar generator" space, and some of them have enough input/output capacity to function as a usable UPS. All that they lack is comms for status and automatic shutdown.

It seems like a fairly recent shift -- heck, the whole solar generator concept is a fairly recent shift -- but maybe it's an indication that LiFePO4 devices are becoming more common in the consumer space.

annoyingnoob• 2 years ago

After having tried several brands, I like CyberPower for the combination of price/performance.

Things may have changed, the APC UPS systems I've used required you to power off to replace the batteries. the CyberPower units I've used allow for replacing the batteries hot.

I've had terrible results with Tripp Lite UPS systems - where they fail for no reason at all, and will not auto power up after a power failure.

At a very large size of 30KVA, I've used Eaton UPS systems and they have been rock solid.

tadfisher• 2 years ago

Eaton/Tripp-Lite have a few Li-ion models: https://www.eaton.com/us/en-us/products/backup-power-ups-sur...

EvanAnderson• 2 years ago

The Liebert / Vertiv lithium GTX series is nice but very pricey. I was shocked at how light a 3000VA unit was.

SadWebDeveloper• 2 years ago

I have a BR700G it works flawlessly with apcupsd, minimal to no configuration... the cheap units of APC is for really really desperate consumers, the battery didn't last more than a year from my experience like the BR* ones that usually last 3 years, these lines is the better of the both worlds cheap and sophisticated enough to work "smart" but as expensive as the SMT/SMC units.

Also like everyone say... remember to look for pure sine wave output if you are going to "protect" servers, for network equipment you could use the pwm based ones.

rpcope1• 2 years ago

Eaton makes much better UPSes at a close price point. I don't know why anyone buys APC or things like Cyberpower anymore.

dhd4152• 2 years ago

This. I finally switched from using various APC models to an Eaton 5S 1500 unit for my desktop and a couple peripherals and it's much better while still being comparably priced.

topspin• 2 years ago

Eaton makes much better UPSes

What makes Eaton products better? Sincerely want to know anything you can tell me.

yjftsjthsd-h2• 2 years ago

I bought one because that's what I could get at Best Buy. This was particularly important because being able because being able to purchase it at a known physical store near me meant I could get it the same day from a relatively trusted vendor, be acceptably confident that I wasn't getting a counterfeit, and that there was hopefully at least some minimum level of quality (read: it probably isn't going to burn down my house).

macintux• 2 years ago

I just looked at Eaton's website. They recommended two companies near me that carry their products: one doesn't have a website (it's not a retail address, maybe just some random person's home), the other's website has lorem ipsum text on their Eaton partnership page, and their shopping site has no Eaton products at all.

So, yeah, it's understandable why people might not be buying Eaton.

BenjiWiebe• 2 years ago

Sorry, don't have the model numbers handy, but:

We have an Eaton and several APCs, all fairly cheap/lowend. One recent power outage, the APCs flipped on but the Eaton just turned off...

Toutouxc• 2 years ago

I’ve had an ultra cheap Eaton 5E (900 VA) for a few weeks. Because I still don’t entirely trust it, I’ve tried cutting the power to it several times (like 10x maybe) and it’s been fine every single time (it talks to a Debian box with NUT). The whole thing was around $80, if it doesn’t burn the house down, there are no downsides.

rcarmo• 2 years ago

I have a 550ES, works fine. Some models will require apcd instead of nut, but I have configs for using Proxmox with both here: https://github.com/rcarmo/ground-init/tree/main/samples

throw0101c• 2 years ago

Note that APC has different lines (at various price points):

* Back-UPS: stand-by, Stepped approximation to a sinewave

* Back-UPS Pro: line interactive, Stepped approximation to a sinewave

* Smart-UPS: line interactive, Sine wave

* Smart-UPS Online: double-conversion online, Sine wave

teddyh• 2 years ago

I wonder what happened to Eric Raymond’s UPSide project¹.

1. <https://gitlab.com/esr/upside>

g1sm• 2 years ago

Some discussions here [0] some months ago. Seems they are stuck in the design phase. I had high hopes regarding this project. Shame nothing came out of it.

[0] https://gitlab.com/esr/upside/-/issues/67

liberix• 2 years ago

I've been monitoring several dozen APC Back-UPS units for more than 10 years using apcupsd on Debian without any issues.

There were only a few times (maybe 2 or 3 in all that time) when I saw some messages indicating that something was wrong with the battery (though they never were as frequent as in the author's case). In each of those cases, the messages went away after replacing the battery.

mmphosis3• 2 years ago

We get a lot of power failures. I use laptops, but for other non-battery powered equipment, I have a BN1500M2-CA with a raspberry pi running debian plugged into one of the USB charging ports and the USB Data Port cable connected from the UPS to the raspberry pi. It keeps my ISP's 185 watt router going for longer than the ISP's switches will stay up when the power is out. I get more than 3 hours with the router and a 4K monitor plugged into the UPS. I remember seeing NUL characters in the text of an old log, but otherwise apcupsd works well:

  2024-01-30 08:15:13 -0800  Power failure.
  2024-01-30 08:15:19 -0800  Running on UPS batteries.
  2024-01-30 08:20:06 -0800  Mains returned. No longer on UPS batteries.
  2024-01-30 08:20:06 -0800  Power is back. UPS running on mains.
  2024-02-04 12:52:58 -0800  Power failure.
  2024-02-04 12:53:04 -0800  Running on UPS batteries.
  2024-02-04 12:54:04 -0800  Mains returned. No longer on UPS batteries.
  2024-02-04 12:54:04 -0800  Power is back. UPS running on mains.
  2024-02-18 12:10:13 -0800  UPS Self Test switch to battery.
  2024-02-18 12:10:21 -0800  UPS Self Test completed: Battery OK

xp84• 2 years ago

ISP’s 185 watt router

Wait, what?

Does it also heat your room? How can it be so power-hungry?

mmphosis3• 2 years ago

I call it "the chimney."

p1mrx• 2 years ago

A Starlink terminal uses less than 185W, and that talks to space while melting snow.

bongodongobob7• 2 years ago

I'd bet the power supply is just rated at 185W. No fucking way is a router actually drawing that much power.

tenebrisalietum2• 2 years ago

It sounds like OP works at or is an ISP.

tristor5• 2 years ago

I would never buy a cheap UPS. The one time I did so many years ago, it ended up catching fire. UPS fires are the most common cause of datacenter fires, and that's the "good" stuff. Adding a UPS is not a zero-risk action that has only upsides. You need to actually think about it.

At home, I have had a single rack w/ network and server gear for more than a decade and I always run two good quality UPS, that I run a test on annually, tied to two different circuits. My setup now is a bit annoying because I'm renting currently, but when I owned a home prior to my recent move I had an electrician run two 20A Dual Function (AFCI+GFCI) circuits with hospital grade outlets to power the two UPSes in the same room but on separate circuits and breakers at the panel. I don't think that's overkill, but I think anything more would be overkill at home. When I buy a house again, I intend to do the same.

Since buying better quality UPS and testing them regularly so I can replace batteries when needed I haven't had any fires, but using batteries past their useful lifespan can greatly increase risk of fire. Expect that you should replace the batteries every 3-5 years, and that they will cost nearly as much as the UPS itself. If you're not comfortable with that, invest in a quality surge protector instead and figure out how to deal with failures. UPS batteries are a big fire risk, so you need to understand it and work to contain it. Also, keep a good dry fire extinguisher near your rackmount equipment and get it tested annually.

turtlebits• 2 years ago

Can't you buy a UPS with a LiFePO battery, or find an old lead acid one and do a battery swap?

cptskippy• 2 years ago

UPSes with dead lead acid batteries are a dime a dozen. People don't realize or don't both servicing the batteries and toss them out all the time.

bongodongobob7• 2 years ago

Nonsense. UPS's don't pose any sort of serious fire risk. UPSs starting on fire are exceedingly rare. Sure, it's probably happened. People get hit by lightning too.

silon42• 2 years ago

I'd guess they are certainly safer than a laptop, but it depends on a manufacturer... (I've seen lots of "balooned" laptops, including many Macs)

tristor5• 2 years ago

Not to be snarky, but UPS fires are one of the leading causes of datacenter fires: https://duckduckgo.com/?hps=1&q=UPS+Fire+datacenter&ia=web

Grazester• 2 years ago

After using hundreds, hundreds of very cheap to huge expensive APC UPS' I can say I have never experienced a fire. They have all used lead acid batteries, some as heavy as 50lbs. I have even had to cut UPS' open to remove swollen batteries.

dheera• 2 years ago

Is this specific to Lithium battery UPS units?

For home use I bought a lead acid UPS specifically to avoid a fire. For a 5 hour power outage it makes no difference to me if the UPS powers my machine for 5 minutes or 20 minutes; as long as I can save my stuff and shut down it works for me.

wannacboatmovie4• 2 years ago

I had an electrician run two 20A Dual Function (AFCI+GFCI) circuits

Don't do this. This is a terrible idea if you intend to run a UPS.

with hospital grade outlets

Complete overkill and waste of money

UPS batteries are a big fire risk

No they aren't.

Judging by all these incorrect assumptions about electrical, I'd suggest stopping while you're ahead before you burn your own house down.

bloopernova• 2 years ago

> I had an electrician run two 20A Dual Function (AFCI+GFCI) circuits
Don't do this. This is a terrible idea if you intend to run a UPS.

Can you please elaborate why someone shouldn't do that? (not who you responded to, I'm just curious about the reasoning behind that statement)

tristor5• 2 years ago

Judging by all these incorrect assumptions about electrical, I'd suggest stopping while you're ahead before you burn your own house down.

For anyone reading this, this person has no idea what they're talking about. Besides the fact that I once considered becoming an electrician and completed my apprenticeship prior to entering the tech industry, it is a published standard how things should be installed in residential properties in the US, called the National Electric Code or NEC.

The NEC 2020 states that GFCI is required (in additional to a general AFCI requirement) for any 125V or 250V receptacles in kitchens, bathrooms, laundry areas, finished or unfinished basements, garages and anywhere within six feet of a sink. Since my server rack was in a closet in a finished basement, it is /REQUIRED/ by code to have a dual-function breaker.

Also, I once again refer you regarding fire risk to simply search "UPS fire datacenter". UPS fires are the primary cause of datacenter fires. But sure, I don't know what I'm talking about, I've only been responsible for multiple datacenters spanning the globe for one of the world's largest hosting providers as part of my career after giving being an electrician a go and deciding against it as a career in my youth.

If anyone is ever in doubt about how to install something that's electrical: 1. Hire a certified electrician and 2. Read the applicable standard, which is the most recent published version of the NEC. Don't listen to people who make rude smart-ass comments on the Internet.

daydream• 2 years ago

What’s the failure mode you’re guarding against by having the UPSs on separate circuits and breakers?

Is the idea to protect against a breaker failing, a wiring issue, or something similar?

hedora• 2 years ago

I have one of the nicer APC units (for use in offices, not racks). Recently, when the power blipped, it went into backup mode and got stuck there, draining the battery. I had to physically unplug it and plug it back in to get it to turn back on.

I'm not sure if there's a decent UPS brand any more. I get the impression APC has been going downhill since the acquisition.

newsclues• 2 years ago

Eaton

da7683• 2 years ago

Vertiv Liebert

js2• 2 years ago

No one on this site has a large enough sample size to speak authoritatively about the quality of consumer-oriented UPSes.

In my home right now, I have an APC (Schneider) Back-UPS Pro 1000, Tripp Lite (Eaton) AVR550U, and CyberPower CP1500PFCLCDa.

I purchased the APC in April 2017, the CyberPower in Nov 2020, and the Tripp Lite in Sep 2021.

The APC was originally for all my IT gear but for some reason that load caused it to cycle to battery at random times. I replaced it with the CyberPower which works fine with my IT gear. Meanwhile the APC works fine with my AV gear where I re-purposed it.

Both of those are still on their original batteries which I test every 6 months.

The Tripp Lite I use at my desk and I also test it every 6 months, but last month the power went out and the Tripp Lite immediately shut down. The battery had failed w/o warning and the Tripp Lite battery-monitoring is apparently useless since it still thought the battery was good when power was restored, but an actual load test proved otherwise.

When I replaced the battery, the Tripp Lite had a cheap Chinese battery in it. I replaced it with a Duracell (manufactured in Vietnam). I've had less trouble with non-Chinese UPS batteries.

Another fun thing: Tripp Lite has reused the model "AVR550U" for three different UPSes all which use a different battery. Turns out my version uses the quite common APC RBC2 sized 7Ah battery.

Which is all to say: I've had a lot of UPSes over the years and you can't make any claims about any single manufacturer. They all manufacture a range of models and none of us have any actual data about failure rates. The best you can do if you have the expertise is to take one apart and evaluate its design and manufacturing quality.

pastorhudson• 2 years ago

Eaton is what I use and it is one of the only ones that provide pure sine wave. It’s good stuff.

tomek_ycomb• 2 years ago

100% agree, but have never had $1k to drop like that on a backup dedicated to my PC ;p.

csnover2• 2 years ago

I'm not sure if there's a decent UPS brand any more. I get the impression APC has been going downhill since the acquisition.

I’ve noticed a similar thing.

I had a 2005 Smart-UPS tower that never failed, but developed some transformer buzz that was annoying in an office environment—and hey, improved standby efficiency and an LCD panel would be nice—so I replaced it in 2013 with its latest equivalent. This one ran for about nine years and then started rebooting itself randomly, dropping the load each time. (It had no issues transferring and holding a load on battery, and self-test passed.) Its 2022 replacement now uses a non-standard USB-A male-to-male cable, is missing information that used to exist in the LCD menu, seems to have a problem charging from 98% to 100%, and has a broken event log (event 1 is always “Site wiring” and every other event is always “None”, even though there have been multiple power events, and there is no site wiring issue). It works, but QA issues are evident.

I’d previously tried a prosumer-grade CyberPower UPS (CP1500PFCLCD) on some other less critical equipment. When its battery failed after two years, it cut power to the load. When the charger failed a few years later, it cut power to the load. It died completely in about six years.

Tripp-Lite’s consumer grade stuff seems to work well enough for what it is, but their higher-end equipment seemed to all be designed for environments where noise doesn’t matter, which makes it a non-starter for an office. Eaton (their parent) seems the same.

So it would be great to know what is an actually good choice these days. At the moment I’m not in the market for anything and hopefully what I’ve got will run for another decade, but if it doesn’t, I have no good idea about what else to buy today.

aidenn0• 2 years ago

I’d previously tried a prosumer-grade CyberPower UPS (CP1500PFCLCD) on some other less critical equipment. When its battery failed after two years, it cut power to the load. When the charger failed a few years later, it cut power to the load. It died completely in about six years.

I've never (n=4) had a stock CyberPower UPS battery last me 2 years. 3rd party replacement batteries have all lasted significantly longer than what shipped in the box.

th3typh00n• 2 years ago

event 1 is always “Site wiring” and every other event is always “None”, even though there have been multiple power events, and there is no site wiring issue

Some UPS:es are adamant about wanting live and neutral on specific pins on the power plug and will throw that error message if they're swapped.

If it's a reversible power plug, flip it around. If the power plug can only be connected in one orientation the outlet is likely incorrectly installed (which is not all that uncommon, as 99.99% of stuff will work perfectly fine with live and neutral swapped).

jaredhallen• 2 years ago

I still have pretty good luck with Eaton. Agree on APC, although I was never particularly impressed with them in the first place.

johnklos• 2 years ago

First, it's extra work, but it's possible to configure apcupsd to run a script on events, then use that script to ignore spurious events.

Second, for people who can't necessarily afford either new UPSes or something other than "consumer grade", a good option is to find local sellers, perhaps on Craigslist or Facebook Marketplace, who are selling or giving away old UPSes. Many people and businesses don't bother replacing batteries, so they just buy new devices when the batteries near or reach end of life, and since they're heavy, they often just want to give the hardware away.

New batteries are not expensive. A common 9AH battery for many / most common smaller UPSes costs around $25 USD from Amazon sellers that have 4.5 or more stars.

I've personally never bought a new UPS for myself, yet all of my equipment and everywhere I've set up home Internet equipment for others has UPSes. I've only ever bought batteries. Just a thought.

underlogic5• 2 years ago

I have had maybe 10 UPS units over the years but I would never buy another one. I was once severely poisoned by an APC unit with a leaking battery. It released an odorless vapor, had me in the hospital twice before I figured it out. Not a fan of APC quality anymore either. I don't trust the product, don't want to plug it into my equipment. Most everything I use now has a battery or will just reset on power failure anyway.

The model he is referring to is the last UPS I owned and had some very serious QA issues;

https://www.eevblog.com/forum/dodgy-technology/apc-ups-now-w...

e402• 2 years ago

Wow. What would you use?

We had a bunch of Cyberpowers and they were absolutely crap. The failed at a high rate and had terrible software.

underlogic5• 2 years ago

A UPS is really just a car battery and a voltage regulator. Actually the cheaper APC models have no voltage regulator which is why you should avoid them. It's obsolete technology. Modern regulated power adapters are enough just use a strip with a surge protector. If you really need something to protect your gear get a voltage regulator. But if the power drops frequently then maybe you do need a UPS idk. If you really do then the APC enterprise units. But I'd avoid if at all possible

https://www.apc.com/us/en/product-range/61877-liner/?parent-...

skottenborg• 2 years ago

How did you figure out the UPS was the cause?

underlogic5• 2 years ago

When I came back from the ER I got sick again. The UPS only had arrived a couple of days before

atesti• 2 years ago

What was this odorless substance and where did it come from? (just curious what to watch out for)

underlogic5• 2 years ago

It came from a leak in the battery during charging. Exactly what the gas was is still unknown but it knocked me for six a couple of days after I received the unit. I thought about getting it tested and even suing but it took out my whole apartment and I just had to get rid of it and cut my losses. I was in the ER unable to stand and in quite a state, lost balance and coordination and was incoherent for a while. CAT scan w/contrast, it was a whole production on Thanksgiving at 3am. They monitored me for a couple days and sent me back to the apartment where all the symptoms immediately returned, and then I put it together and looked closer at the UPS. Underneath it was a couple of drops of black liquid that had come from the battery compartment and after returning to the ER again to go throw up some more I went back and took it straight to the garbage. I think it was one of those types of gas that can effect you in the PPM range. Totally odorless. This was the enterprise APC UPS, the model was SUA1500

NelsonMinar2• 2 years ago

I once installed apcupsd only to be so appalled by its lack of security design I ripped it out PDQ. It demands to run as root so it can run shutdown (and maybe access the USB?!) and goes downhill from there.

nottorp3• 2 years ago

Well, you want it to run shutdown don't you?

Afraid your ups will hack you?

NelsonMinar2• 2 years ago

There's many safer ways to let a mostly untrusted process run shutdown. Like a sudo setup letting the UPS user only run shutdown. Or /etc/shutdown.allow. Or something using CAP_SYS_BOOT. systemd might have a solution too. I get the impression I just spent more time thinking about this writing my response to you than the APC folks ever did.

Yes, I am afraid my UPS will hack me. More specifically I'm afraid this badly written closed source software will have some security hole that can be used to escalate to root.

rubatuga• 2 years ago

Seems like your best bet is to use an industrial battery inverter/charger system (Victron Multiplus) for slow-switching (>20ms) and then layer a secondary industrial grade UPS to provide fast switching (<15ms) and shutdown process through USB/serial/network etc. Is it cheap? No, but you shouldn't be using cheap power supplies to protect your more expensive server equipment.

shmerl• 2 years ago

I have no issues with Back-UPS RS 1500MS2 on Linux.

I had to configure an udev rule though for it to set up a static USB device name since that wasn't happening out of the box, but other than that - no problems.

See http://apcupsd.org/manual/manual.html#usb-configuration

ek750• 2 years ago

I'm kinda sick of buying these cheap lead-acid based UPS units (only tried APC and Cyberpower so far) to have the batteries die in just a couple of years.

Are the lithium-based batteries more durable (probably more dangerous)? Anyone have experience with these? https://www.amazon.com/Xtreme-Power-Conversion-J60-600-Lithi...

alright25653• 2 years ago

What's interesting is that APS's Windows-only software, Powerchute Serial Shutdown, is actually Java-based, accessed through the browser, and trivially de-compilable and readable.

secabeen2• 2 years ago

In my experience, all modern operating systems and widely-used databases handle power interruption well. I can't recall an time where a filesystem or database was corrupted by power loss in decades. Adding a monitoring/shutdown system for basic UPSes is not worth the risk/hassle. I'd just monitor the power directly and let the system fall over when the UPS runs out of battery.

nayuki2• 2 years ago

Some SSDs brick themselves in an abrupt shutdown.

secabeen2• 2 years ago

Do you know which ones, so I can avoid them?

neilv3• 2 years ago

After many UPSes at home over the years, I currently have only 1 (a business-grade rackmount Cyberpower), and once it needs a new battery pack, I'll probably just switch to a surge protector.

My local electric is pretty stable, I no longer run servers at home that need to be up 24/7, most of my work at home is on a laptop with integrated battery backup, I can use my phone WiFI AP mode for backup Internet, UPSes are a pain to move between apartments, and UPSes have a small risk of dangerous battery leakage.

1letterunixname• 2 years ago

Don't even bother buying used enterprise APC units because firmware updates now require $$$.

NMC2 and NMC3 can't even install proper SSL certs without some shitty Windows-only tool.

mastax• 2 years ago

I'm having exactly the same issue with apcupsd and my APC Smart-UPS X 1500, so it's not just an issue with the cheapo units.

binkHN• 2 years ago

I refused to go down this route; the hours lost in frustration are not worth it.

I purchased a UPS that supports a network management card, basically an overpriced SNMP daemon with an Ethernet interface, and have been happy ever since, since every system can query the status of the UPS.

Animats2• 2 years ago

Have low-end UPS units gone lithium iron phosphate yet? That's the battery technology of choice now for non-portable applications. Cheaper and longer-lived than lithium-ion, no thermal runaway risk, about the same energy density per cubic meter, but twice the weight.

Such UPSs are available in small rackmount units.[1] The usual serious players, such as Tripp-Lite, have LiFeP04 battery powered units. Searching for consumer-grade LiFePo4 UPSs turns up many articles on how to replace crappy batteries in UPS units with newer battery technology. The batteries themselves are cheap now. APC doesn't seem to have caught up yet.

This looks like an APC corporate problem.

[1] https://www.falconups.com/ups/lifepo4-ups-sl.htm

paulmd• 2 years ago

Have low-end UPS units gone lithium iron phosphate yet?

no, and afaik it's a cost thing. the unit you link is "request for quote" and being a rackmount unit it's probably not what consumers would think of as inexpensive.

I have three of the classic fire-hazard cyberpower 1500PFCLCD and each of them has a pair of small sealed lead-acid (SLA) batteries that cost maybe $30-40 each to replace. If you assume that the OEM gets them for $20 apiece, how much LiFePO4 battery does that buy? I know the big industrial cells are cheaper than what you can get as small individual cells, but I'm positive it's still going to be 4-10x more expensive.

now remember that's a unit that retails for $135-150, and there are like 3-4 tiers of units below that... how much LiFePO4 can you buy for $10 to put into that $50 UPS?

(fake edit I looked one of them up from the datasheet and it's $700 for the lowest-tier 1000VA unit lol)

oceanplexian• 2 years ago

It’s not only a cost thing, they don’t have the power output for the same form factor. You can technically swap in those 12v LiFePo4 “replacements” into a cheap UPS but they don’t put out nearly as many peak amps as good old SLAs. Lithium Ion and Polymer can but they also run the risk of blowing up. I’ve played with basically all these chemistries doing model airplanes and DIY drones.

Dylan168073• 2 years ago

(fake edit I looked one of them up from the datasheet and it's $700 for the lowest-tier 1000VA unit lol)

That's the bigger issue, that there's nothing medium price.

I can get a LiFePo4 battery that can do well over a kilowatt for an hour for <$150 with free shipping, and I can get an entire UPS for <$100, but the combination costs so much more.

Marsymars• 2 years ago

Lithium not common for 120V, but I have several APC CP12142LI units for 12V devices. (No USB monitoring, and they're not terribly price-competitive for the capacity, but they've worked well for me.)

pgcudahy• 2 years ago

Has anyone tried the new anker or ecoflow lithium portable power stations as a UPS?

russelg• 2 years ago

It's very likely their switching time between wall power and battery isn't quick enough to keep a computer on. With ATX you need to switch power in no longer than 16ms else the computer will shut off.

rasz• 2 years ago

EcoFlow has non continuous output. Shouldnt be a problem for PC, but some loads ae sensitive to it:

EcoFlow Delta Pro - RTFM Dummy! EEVblog2 https://www.youtube.com/watch?v=5JT8M8v_zNg

Bluetti dos much better:

Bluetti AC500 / B300S Power Bank - Review & Teardown mikeselectricstuff https://www.youtube.com/watch?v=RuEO17uvI8M

joecool1029• 2 years ago

The batteries themselves are cheap now.
about the same energy density per cubic meter

Energy density isn't the issue. Need to either buy the best cells or oversize the pack to meet normal loads. Both options cost additional money.

Go look at the spec sheets on the cheap ones, they aren't power dense enough to make a small (1000-1500VA) UPS with comparable or even double runtime of the lead acid.

neilv3• 2 years ago

For most home use, you don't need UPS monitoring.

If you're home when it switches to battery power, the equipment will probably have been protected from any surge, and you'll hear the beeps from the UPS, and can manually shut down equipment.

If you're not home, and the power doesn't come on soon enough, power will be cut to protected equipment, without a surge, and at most the equipment probably only needs the auto-`fsck`.

And because no monitor means that you can have the UPS airgapped for data, there's less theoretical risk of a remote firmware hack causing a fire/explosion or toxic leak.

zettabomb4• 2 years ago

The point is for when you're not home, it can automatically shut down your PC. It's actually quite useful if you happen to have a home lab. The nicer units can monitor temperature and humidity and send you an email if they exceed specified limits, which can be VERY useful if you live in a flood prone area, for example.

neilv3• 2 years ago

I'm sure it's useful for some, but not most.

I have a homelab of a few servers, with a nice rackmount UPS, and I decided data connection to the UPS is more risk than good, in my case.

Environmental monitoring doesn't have to be part of the UPS. And APC has long made separate devices for that, for data centers/racks. In addition to the modern IoT home devices.

persnickety2• 2 years ago

Are there any UPS that output ATX directly?

I'm not seeing much of a point powering a small server from battery with AC if it all gets converted to DC anyway. Laptops got it right.

jauntywundrkind• 2 years ago

Only another 4 years till Google's patent on using a laptop configuration in a server expires! https://www.datacenterknowledge.com/archives/2008/02/15/goog...

I doubt this one in any of the patent pools (OIN) but maybe they wouldn't sue anyhow?

persnickety2• 2 years ago

I'm feeling a mix of outrage, disbelief, and "what was I expecting" right now.

bl_valance• 2 years ago

I use one for my Synology UPS with no issues, has saved it multiple times and is able to gracefully shutdown when switched to battery for x-minutes.

TowerTall• 2 years ago

My biggest grief with the APC UPS are that they initiate the shutdown process immediately in case of power loss. I live in a location where we frequent has 10 - 60 seconds power cuts. The optimal solution would be if the UPS waited like 5 minutes before starting to shutdown the connected PC. For this reason alone I stopped connecting my desktop pc to UPS and stopped installing the UPS powershute software. I wrote their support and asked if i could daisy chain a battery with the UPS so the battery would be used first and when that battery was depleted the UPS would take over and start the shutdown process. They advices me against this and pointed me to this FAQ. It doesn't directly adressing daisy chaining a UPS with a battery. Anyone here that has experience with doing that?

https://www.apc.com/us/en/faqs/FA157424/

dantastic• 2 years ago

I had this problem with a Cyberpower VP1500EILCD. I created this udev rule to stop the USB auto suspend feature and it's been rock solid since:

SUBSYSTEM=="usb", ATTR{idVendor}=="0764", ATTR{idProduct}=="0501", GROUP="nut", ATTR{power/autosuspend}="-1"

Udo• 2 years ago

The thing I keep re-reading is

popped the breaker for the sockets causing everything to be harshly powered off. My fileserver took it badly and one drive died.

Maybe someone with more hardware knowledge could chime in, but my default assumption would be that the hard drive died either because it couldn't take another power-off/on cycle, or because the event that caused the circuit breaker to pop was harsh enough to damage it (not sure how much the computer's power supply insulates components from things like that, but intuitively, there are much more voltage-sensitive components in PCs than hard drives). While a UPS might have been effective against a surge, it would not have saved the drive from dying by power cycling.

My hypothesis is that most equipment doesn't need to be notified when on UPS, it could in fact be harmful when it causes a power cycle that turns out to be unnecessary. An orderly shutdown doesn't matter that much unless you're dealing with an app that can become internally inconsistent, such as a database. In all other scenarios, the journaling FS takes care of data integrity.

So, especially for home use, why not just let the UPS run until it's out?

sebazzz• 2 years ago

I have a BX750MI and this one still works. Still, it is good to have the warning.

I don't think the cheapest APC Back-UPS units can be monitored except in Windows

Related Stories