Sol-Ark manufacturer reportedly disables all Deye inverters in the US

https://solarboi.com/2024/11/17/sol-ark-oem-disables-all-deye-inverters-in-the-us/

by walterbell • 2 years ago

290 300 2 years ago

solarboi.com

boredatoms• 2 years ago

We need laws to prevent this

jfengel• 2 years ago

There probably are. But it appears to be coming out of China, so good luck enforcing it.

dymk2• 2 years ago

There is a US based company that is importing and selling these devices. Go after them.

_trampeltier3• 2 years ago

It seems they shut the inverters down because a legal dispute. So the reason is the law.

Dylan168072• 2 years ago

That's like saying if I punch someone because of a legal dispute, the law is to blame.

perihelions• 2 years ago

Civil contract disputes don't empower or obligate you to commit crimes in the process of trying to make things right.

The power inverters were *not their property*. Remotely accessing them, without authorization and with the intent of disabling them, is a textbook CFAA felony.

rvba• 2 years ago

Could USA wrap it under the terrorism laws?

gdjskshh2• 2 years ago

We have those laws. You return the faulty device to the entity you purchased it from.

I bet some small-time installers that were sourcing on the grey market will go bankrupt because of this.

malfist3• 2 years ago

You return a solar inverter you already have installed? Maybe purchased years ago? And in the meantime you might be without power. That's not recourse.

sam_goody• 2 years ago

I don't know which "this" was referred to, but I think we need laws to prevent a foreign company or hacker from shutting down our power.

There was an article on HN about a month ago, that two companies each have the ability to overload or shut down the entire grid in many parts of the states, just by their remote control of the solar panels and batterires.

They should be regulated like any other utility.

mleonhard2• 2 years ago

There is no such thing as "grey market". USA is a free market. Everyone is authorized to sell all safe items.

totallykvothe• 2 years ago

People responsible for this kind of evil need to pay with personal property seizure.

anon8487362815• 2 years ago

So like, the people who bought devices from unauthorized distributors should have their devices stop working? Seems you got your exact wish!

tdeck• 2 years ago

Can someone who has a solar inverter explain why these are connected to the internet?

hrkfmud50k• 2 years ago

because they have remote configuration and reporting on solar production, consumption, battery state of charge, grid export, import vs time.

HarryHirsch2• 2 years ago

Practical Engineering had a video on the subject not too long ago: https://www.youtube.com/watch?v=7G4ipM2qjfw

The short answer is: it's for load balancing, it can't be avoided.

viraptor3• 2 years ago

That part is independent of internet connection. Especially since you can't rely on the internet connection in case of power delivery issues. It's a completely different network.

viraptor3• 2 years ago

Usually you want some way of monitoring how much energy your panels are producing. This helps to realise you need to clean the panels or do some maintenance if panels start failing. Or it may be useful for scheduling home appliance usage.

But in practice this almost always means connecting to the internet, because the simplest interface is wifi and data collection/display at the producer's servers. So any extra features == internet connection.

lcnPylGDnU4H9OF• 2 years ago

What would be a good method for keeping the IoT Thing from talking to a machine beyond my locally administered network?

ericd3• 2 years ago

Highly recommend using solarassistant for this, instead - local server software that install on a raspi, and you hook a usb on the raspi to the WiFi dongle port on your inverter with a serial cable. Don’t provide the inverter itself with any wifi credentials.

Solar assistant has the bonus of interfacing your inverter with homeassistant, and letting it control the inverter/get signals from it (so you can do things like, if grid voltage drops to zero, do xyz)

layoric• 2 years ago

Mainly data collection (previous lead dev at solar forecasting startup). All the web UIs to view usage are also collecting useful information that can be used in forecasting models. One of the researches I worked with wrote some papers on using distributed home solar output measurements to assist with generating higher resolution irradiance forecasts and estimated actuals/observations. You have to do a lot of data cleaning to get this reliable though. Anyway, this data from memory was bought/sold for various research/commercial weather modeling.

lxgr3• 2 years ago

Besides the reasons others have already mentioned, load management comes to mind:

Getting rid of excess energy in the grid can be just as hard a problem to solve as to deal with excess load, and being able to simply and very quickly remove some supply from the grid is very useful for that.

bartvk2• 2 years ago

It's just a bad idea. I got caught up in a situation where one company sold me a solar installation, then a subcontractor installed and configured it. Apparently they got into a spat about money, because the subcontractor told me to pay the bill straight to them.

Otherwise they'd shut down the newly installed solar installation. I said, can you do that? Of course while talking, I changed the WiFi password.

bennettnate5• 2 years ago

Solar installations are expensive enough that some manufacturers can probably afford to integrate a cellular modem into the product (similar to how all new cars do it today). Good luck changing the Wi-Fi password on that!

LorenPechtel• 2 years ago

That's why you get lien releases from subcontractors before you pay the main contractor.

plagiarist2• 2 years ago

Too many idiots have bought internet-connected devices so now the inertia is in favor of the corporations to continue selling that.

nunez5• 2 years ago

Hi, idiot here. I badly wanted a US-made robot vacuum that uses LiDAR for mapping and a camera for object classification. This does not exist. Your only options are Chinese-owned-and-operated.

I could flash them with Valetudo and wire them up to Home Assistant, but doing so requires me to solder shit to the JTAG circuit and buy some niche hardware, which requires me to open up the vac and potentially brick it. I'm not risking that on a $1200 device.

greenthrow7• 2 years ago

It's really nice to be able to check whether the state of your power is at home before you go there if there's a question.

whitehexagon• 2 years ago

I have a Axpert MAX E. It has a WiFi AP constantly advertised. The only way to configure/disable that is via a .cn app! The app also allows remote control and monitoring of the inverter, via some unknown cloud server. I run everything local-only, so that is never going to happen.

gruez2• 2 years ago

The only way to configure/disable that is via a .cn app!

What does it even mean for an app to be ".cn"? Apps typically aren't identified by DNS names. Did you have to download it from a .cn domain? Is it just a roundabout way of saying the app was Chinese?

danans• 2 years ago

It's not the solar inverters themselves that are usually internet connected, but rather the controller box (some kind of embedded system) that is internet connected to allow monitoring and control. Perhaps this manufacturer decided to economize and make both of them part of the same "box", with the result that an error condition in the controller would result in the non-operation of the inverter part.

Some systems like mine (Enphase) do a good job of letting the inverters operate independently of the monitoring/control software. But to do this, I believe they need to add data storage to the inverters themselves in order to log data during a controller "outage".

tguvot2• 2 years ago

nice dashboards for information about generation. but most importantly remote troubleshooting/diagnostics. as example i have system made from multiple inverters, batteries, car charger and backup interface. after installation some stuff slightly misbehaved. manufacturer support were able to look at system logs and configuration and identify that system is slightly incorrectly wired/configured, after what installer was able to fix it. same thing goes for malfunctioning parts of system. support can take a glance at it and issue rma on spot

Aeolun• 2 years ago

Where is the government when you need it…

shrubble• 2 years ago

It’s not clear how the device was bricked. Could it be reset to not be bricked by disconnecting it from the Internet and rebooting or reflashing?

bagels2• 2 years ago

This time, it's a malicious manufacturer, next time it's a malicious hacker. Doesn't seem like connecting these to the internet is worth it.

4ntiq3• 2 years ago

I love the narrative of a Chinese manufacturer selling electronics to the West only to one day shut everything off for no reason at all than to fuck with people and disappear and for people to find out the supposedly registered company never existed. It's like a trashy, second-rate William Gibson knock off novel but there's something awfully amusing about it.

lazide7• 2 years ago

Frankly it doesn’t even require (special) maliciousness (per-se) - spinning up random ‘brands’ to sell to rubes on Amazon while obfuscating beneficial owners is essentially standard operating procedure.

The only surprising thing here is they took an action to brick something instead of just abandoning it.

profsummergig• 2 years ago

and for people to find out the supposedly registered company never existed

This already happened to me. Sort of.

Saw an advt for Air Jordans for $7. With a pic of actual Air Jordans. Thought to myself, "it's only $7, let's see what happens".

A very sorry looking pair of shoes arrived a couple weeks later. With "Air Jordan" printed on them. They weren't actual Air Jordans.

There was no way, absolutely no way, to get in touch with the Chinese company that did this.

t-32• 2 years ago

Probably wrong to classify the manufacturer as malicious rather than the importer. Sounds like these units were brought to the US in violation of contractual agreements and thus were disabled when the manufacturer decided to enforce it.

yuliyp2• 2 years ago

But regardless, they're clearly not owned by Deye any longer. Causing damage to an unrelated party in retaliation for a contract dispute between two manufacturers is not OK.

A1kmm• 2 years ago

It's likely they had no contractual agreement with the current owners of the inverters, and yet they have elected to wilfully damage the property of the current owners because they can.

Wilfully damaging someone else's property without permission of the current owner seems pretty malicious, regardless of whether the importers (or maybe someone who supplied to the importer) were in breach of a contract.

lxgr3• 2 years ago

Deciding to enforce something like this after your product has already been sold/installed seems extremely dubious.

Even just building in the capability (assuming this wasn't installed via a generic software update, in which case I'd have some follow-up questions on the security against malware of these things) shows significant malicious intent.

bagels2• 2 years ago

Manufacturer did something with intent to damage someone else's property. Seems to fit the definition to me.

keyle• 2 years ago

Any idea what the impact is for the state grids? I wonder if they got a sudden drop in feed-ins and whether it affects pricing.

Any idea how common this manufacturer is across the place?

I'm not from the states, but I do know that if my solar would be bricked, it would take me weeks to find out. I don't exactly check up on it and it's out of sight.

nullc• 2 years ago

Most of the users of these products were off grid.

A number of the products used in off grid installs have invasive IOT remote access/administration.

It's only a matter of time until it leads to loss of life-- e.g. from people who freeze to death because they can't reconfigure or turn up a system without internet access which is out or doesn't work without power--, if it hasn't already.

lazide7• 2 years ago

Yeah, off grid (as in actually off grid) is a great example of ‘simple is better’ and ‘physical redundancy is essential’.

It’s also the place where money ($$) is often the most constraining factor, so cheap amazon shit tends to be the norm.

crooked-v• 2 years ago

Going by the article, it looks like the title is incorrect and it was Deye (the manufacturer) that did it and not Sol-Ark (the US distributor).

greenthrow7• 2 years ago

To be clear, Sol-Ark isn't only the distributor but per their own claims also designed and engineered the units too.

tibbydudeza2• 2 years ago

Deye manufactured vs the units for OEM use different components - they build to spec.

buckle8017• 2 years ago

Sol-Ark likely complained about Deye selling within the US.

Deye said something to the effect of "we have contractual obligations".

I think they're both at fault.

echelon2• 2 years ago

The biggest takeaway here should be that we need a domestic solar industry.

We can't hold Deye or Chinese companies culpable.

Moreover, this should serve as a warning shot for what could become a national security issue if we keep juggling international suppliers for critical infrastructure. They'll all have the capability of shutting down US electricity, which is unacceptable.

There's no reason we should be importing this stuff.

10u1523• 2 years ago

There are US manufacturers. I have a Tesla PW3 made in the U.S. and it includes solar charge controllers, batteries and inverter.

Pretty competitive too.

AyyEye• 2 years ago

The biggest takeaway here should be that we need a domestic solar industry. We can't hold Deye or Chinese companies culpable.

No, the takeaway is to not allow corps to have remote access to end-user owned devices in the first place.

This story of perfectly capable devices being bricked or having servers shut off has been told so many times with domestic (or friendly countries) companies it's laughable that the conclusion is 'do the same thing but onshore'.

AtlasBarfed• 2 years ago

That would require foresight, investment, subsidies, and good policy.

(looks at election results)

Ok, tariffs. I guess tariffs are the new invisible hand.

And really, what we're talking here isn't domestic manufacturing. It's probably Mexican manufacturing.

andix• 2 years ago

Im sure there is some US law, that considers this an act of terror against the national power grid ;)

Firerouge2• 2 years ago

Sol-Ark certainly seems to embody 'never let a crisis go to waste '.

Sol-Ark may not have pulled the trigger on bricking the inverters, but it certainly sounds like their legal actions pressed Deye's hand.

And then to shake down all the individuals who's inverters broke with a limited time opportunity to buy a brand new one from them....

kstenerud• 2 years ago

Wait, what? So defending your rights under an exclusivity agreement through the courts is somehow now "forcing" their hand? The evil Sol-Ark by suing for compliance to their contract pushed the hapless Deye into bricking consumers hardware?

Firerouge2• 2 years ago

I like how you quoted forcing, but I very specifically did not use that term.

Had there been no exclusivity agreement, I think we can agree that the inverters would not of been bricked for being located in the wrong regions.

I think the malice from Sol-Ark here is that they are only offering a limited time deal, which may pressure people to pay up before the courts clear this up.

Regardless of who shares the majority of the blame, Sol-Ark, Deye or 3rd party vendors, this could of been handled better by all parties involved, and should not have harmed end consumers in this way.

greenthrow7• 2 years ago

Why are you blaming Sol-Ark when Deye is the one in breach of contract taking illegal actions the entire time? Seems very disingenuous. They also did not force Deye's hand in this action and seem surprised by it.

jeroenhd• 2 years ago

in breach of contract

I can't really figure out what they did that was in breach of contract. As far as I understand it, they don't do business inside the areas affected, so there is no contract to speak of. Instead, their authorized resellers seem to be the ones installing for their hardware; I don't even think it's legal to sell their hardware if it doesn't comply with FCC/etc guidelines.

Is geo-blocking illegal? Am I entitled to a refund if I import American hardware that refuses to operate in my country?

I think people were risking a broken setup for a big discount, and now it's come back to bite them in the ass. If the units affected were official installations done by their American reseller, their reseller wouldn't be so ready to offer up free replacements.

SoftTalker2• 2 years ago

Reason #42 that I don't want to own my electric supply equipment. I'm happy to pay a utility to provide AC power to my service panel.

knappe2• 2 years ago

The same utilities that are already turning off power at even the chance there are red flag warnings? Surely you're joking.

SoftTalker2• 2 years ago

I have never had my utility power cut for any cause other than storm/ice damage. And it's generally back on within a day, without any involvement on my part. If a hailstorm destroys my rooftop panels or a misbehaving vendor remotely shuts off my inverter, these are problems I now have to solve for myself. No thanks.

triceratops• 2 years ago

Not a very productive comment...

t-32• 2 years ago

Can the firmware still be flashed? I found cloud-free custom firmware exists for these inverters with a quick search, so if the units can still be flashed many may be salvageable.

lxgr3• 2 years ago

I'm almost grateful to the manufacturer for demonstrating the terrifying kind of cyberattack enabled by such remote update/lockout functionality.

Just imagine this kind of thing happening in a (probably not so distant) future in which a significant fraction of all electricity is being generated in a decentralized way, using devices such as this...

_trampeltier3• 2 years ago

There was already a case (many years ago), where something was wrong with an update. All inverters from a country did not start anymore. (You have to set the country or grid code in each inverter, so they know the grid limits).

joe_the_user2• 2 years ago

I'm not sure about this.

I know various hackers, back in the day, were congratulated for their "public service" of showing vulnerabilities. The problem is that we've to a network infrastructure that is only secure by piecemeal bug fixes and ad-hoc filtering and moved to situation where hacking is a (maximally shady) business.

Will things be different with power grid and other infrastructure because lives depend on it? I don't see any indications.

"The society at the stage of the integrated spectacle is characterized by five principal features: incessant technological renewal; fusion of State and economy; generalized secrecy, unanswerable lies; a perpetual present." Guy Debord, Commentaries on Society Of The Spectacle

rootusrootus2• 2 years ago

That is sort of a silver lining. We can use PR disasters as levers to make regulation happen which will hopefully add some protection in the future.

tw042• 2 years ago

Not sure why sol-ark is getting blamed.

People were buying Chinese inverters meant for the Chinese market off aliexpress on the gray market and shipping them to other countries. Deye decided to crack down on the behavior.

There’s nothing indicating this has anything to do with sol-Ark at this point other than them being the approved distributor of rebranded deye inverters in the US.

stavros6• 2 years ago

What harm was it to Deye that these were being sold elsewhere, that they couldn't fix by saying "sorry, we only support China"?

quintushoratius• 2 years ago

Two possibilities come to mind:

1. They're not properly licensed for other markets. Something equivalent to selling a radio transmitter in the US that's not registered with the FCC.

2. They price units outside of Asian markets much higher and don't want to allow/encourage arbitrage that they don't control.

This is definitely a case of "porqué no los dos" (or more).

stefan_• 2 years ago

Different countries have different laws and requirements around grid-connected inverters, mostly so people working on the grid don't get electrocuted when a stray inverter keeps feeding in power.

CyanLite2• 2 years ago

Sol-Ark’s markup is like 5x the list price just for the official rebadged version. Sol-Arks (“US veteran owned company”) still have the firmware made in China, and are susceptible to Chinese hackers, and had to be bought through a distributor. So naturally people went with off-listed Deye inverters because of the scheningans from Sol-Ark.

Now, people are without power and they have to go to Sol-Ark to get power restored, likely by paying through the nose.

greenthrow7• 2 years ago

That's one way to frame it. Another is Sol-ark incurs costs of developing, marketing and supporting their official devices and the contract manufacturer is able to sell their own version in the Chinese market. Greedy people who don't want to pay Sol-ark for all the costs they incurred bought grey market devices that Sol-ark has repeatedly warned are in contract violation in this market. The manufacturer, not Sol-ark, has now bricked those devices, and people are blaming Sol-ark anyway because they want to continue to justify their actions.

technofiend3• 2 years ago

I would expect a vigorous effort to reverse engineer Solark's firmware to spin up, assuming it hasn't already.

josephcsible• 2 years ago

Deye decided to crack down on the behavior.

Contempt of business model is legal, and vigilantism is not.

mleonhard2• 2 years ago

USA is a free market. Everyone is authorized all the time to sell every safe product. The terms "gray market" and "authorized reseller" are linguistic manipulations which benefit manufacturers at the expense of everyone else in society.

I think Daye broke US law when they destroyed law-fully purchased products inside USA. I hope the inverter owners bring a class-action lawsuit against Daye in the US. The court could block the sale of the company's products in USA until they restore the inverters and pay restitution.

oefrha3• 2 years ago

That’s laughably wrong. Exclusive distribution rights are probably enforced more strictly in the U.S. than anywhere else in the world. They are governed by contract law. In addition, many product categories need to be demonstrated as safe to the right licensing agencies before being sold, not after.

MortyWaves• 2 years ago

Actions like this should forever ban an organisation and its executives from operating in anyway in the countries affected.

metalman• 2 years ago

off grid here,off and on since the early 90's current iteration uses US made charge controller and inverter, midn9ght and magnum both capable of firmware updates, but continue to function after 10 years without coms.The midnight controller did pop up a cheeky message of "got coms?" for years, but for some reason , gave up. The thing with both of these pieces of equpiment is that they are designed by bad ass electrical engineers to survive and continue to function under the worst conditions..... and then some, which I have personaly tested. I believe that a firware update could be done with any old laptop, and that while as a new owner I did go all ocd watching all of the data(did learn a lot), now I sometimes forget that the system exists, ......its that reliable

Schiendelman5• 2 years ago

What products would you recommend today?

Filligree2• 2 years ago

Not GP, but Victron makes some serious beasts. Their whole system is modular, so easy to expand, and it’s local-only by default.

cenamus• 2 years ago

Interesting to see China do the same thing as the US did to China so many times. Only now it's wrong

Cheer2171• 2 years ago

Export controls =! Remotely disabling already-purchased hardware

_trampeltier3• 2 years ago

As far I know, software like CADs also just stopped in russia after the war started.

zrail• 2 years ago

As a consumer and homeowner I try my hardest to buy "smart" things that only have local control, especially for important systems like power and HVAC. Our standby generator has a manufacturer supplied wifi pod that I never set up. Instead I use an RS485-to-USB dongle and monitor it myself with open source software. Our HVAC is the same to the greatest extent possible. When shopping for a new robot vacuum Valetudo[1] compatibility is an overriding concern.

If/when we have solar installed it will not be connected to the manufacturer or distributor's cloud systems.

[1] https://valetudo.cloud

nunez5• 2 years ago

I love the idea of Valetudo but flashing devices with it is a hell of a lot of work (if you can at all) and projects like these aren't entirely safe from takeovers from malicious actors either.

greenthrow7• 2 years ago

Title is a bit misleading and makes it sound like Sol-Ark did this. They did not. Title should be "Deye manufacturer reportedly disables all Deye inverters in the US". They are the same entity but this wording avoids confusion about Sol-Ark being responsible.

api• 2 years ago

If it depends on the cloud to operate it's not yours.

greenthrow7• 2 years ago

These devices do not depend on the cloud. If I want to take my Sol-Ark inverter offline I can just take the wifi dongle off it. Dunno about the bootleg Deye one.

tibbydudeza2• 2 years ago

I presume they locked out the solar recharging of the battery and home supply but do not stop mains grid power to the home ???.

jchw2• 2 years ago

It is extremely frustrating to watch "connected" "smart" devices repeatedly do exactly what we knew they would do, and yet nobody ever learns a damn thing. People will keep on buying Internet-connected devices, manufacturers will keep making them, this sort of thing will keep happening, and the rest of us will struggle to even find mass-manufactured things that are not Internet-connected and "smart".

Even devices that are pretty much for "self-hosting" are increasingly trying to sneak in cloud-connected back doors, like Synology DSM trying to sneak in cloud authentication to your local NAS. Stop trying to make the devices I bought for the purposes of having locally-managed devices depend on cloud services! My local network is not just a fucking gateway to cloud services!

gdjskshh2• 2 years ago

I gave up on Ubiquiti because of the cloud nonsense. Altium is pushing cloud hard (and pushing me to KiCAD). I'm a weirdo for using a mac w/ only a local account (no apple id). I can't buy any new or electric vehicles because they're all 'smart'.

The cloud is artificial, so it must be chemtrails, which explains why modern software feels like its giving me cancer. Wake up sheeple. /s

Matheus28• 2 years ago

Can’t you use ubiquiti fully locally? I haven’t tested my setup but I can access the web ui directly through the device ip

CuriousCosmic• 2 years ago

If you want an electric truck (or potentially an SUV), consider looking at an Edison Motors pickup truck retrofit. They are technically Diesel Electric instead of pure electric but you can customize the battery load if you want to run full electric. They don't do all the stupid cloud connected software stuff and they are all about repairability/self maintenance.

Probably the only electric vehicle manufacturer that isn't egregiously tech-bro-y and dripping in dark patterns.

rootusrootus2• 2 years ago

Maybe the solution is not to abstain from the latest tech, but to regulate companies that make these devices so the shenanigans are actually illegal. It is not a problem that should only be solved by ideologically driven people who are willing to tinker and suffer, but rather a protection all citizens enjoy.

pixl97• 2 years ago

As long as companies can buy politicians in the US don't expect it to take off.

jchw2• 2 years ago

I'm not fully convinced that legislation alone can fix all of our problems, but for what it's worth, I'm all for it.

That said, regulation probably won't solve my problem, because what I want are devices that are specifically not designed to just be cloud-connected thin-client devices. I doubt regulation is going to entirely prevent this class of device from existing. And it's only going to get worse: look at what Microsoft is doing, they're literally trying to shift Windows into being a fucking cloud service.

binary132• 2 years ago

If anything regulators will prefer to abolish NON-connected devices.

isodev• 2 years ago

The very iPhone I’m reading this on is one trade war/sanction away from becoming a useless brick of electronics that probably can’t even show the time without calling to Apple every now and then.

nunez5• 2 years ago

I feel for customers impacted by this but hate that the only real choices customers have are local, but expensive, equipment or affordable, but outsourced equipment.

This is endemic in the home automation space. Nearly everything is made and operated on Chinese soil. Like security cameras, or, in my case, our LiDAR and camera augmented robot vacuums.

Some components, like lights and switches, have (very) expensive American alternatives. Some support ZigBee or Matter and can be controlled locally. Many many others require cloud infrastructure operated outside of the US and become bricks without it.

I would love to see the US mandate ITAR for all IoT devices sold in the US. If anything, that will help prop up local alternatives like Matter since that will be way cheaper than building compliant cloud-connexted devices.

joe_the_user2• 2 years ago

The question is whether a customer has to actually connect these things to the net.

nunez5• 2 years ago

They do in many cases. Example: GE CYNC Wi-Fi lights require a connection to Savant's servers, which I believe are split between US and CN. They are one of few vendors that make BR30 smart lights. Philips and LIFX aside, all of the other vendors require an Internet connection.

hippich• 2 years ago

Regarding solark statement about using their own backend. I am pretty sure they transitioned to it around May 2024. Before that it was different site, which I am pretty sure was shared by all deye customers. I wonder if this event was planned well in advance...

anon8487362815• 2 years ago

It is possible that Deye waited to pursue this blunt remedy until Sol-Ark customers would no longer be caught up in it. Doesn't mean Sol-Ark knew what they intended.

mastazi• 2 years ago

Here's what I want: by law, any device that is connected to the internet needs to have a warning on the box, similar to the one that's on cigarettes packaging, stating the risks of that device being online (bricking/loss of service, data might be compromised in a cyberattack, etc.)

ronsor• 2 years ago

That will end up being as effective as California prop 65, better known as "that cancer warning everyone ignores."

morningsam• 2 years ago

For it to be effective, all it needs is its complement: An easily recognizable green label saying "Doesn't connect to the internet", which is only allowed on the boxes of devices for which this is the case.

Maybe some more levels in the middle like "only connects to the internet for firmware updates" (yellow) and "doesn't require internet access for core functionality" (orange). Basically Nutri-Score [1] for hardware.

[1] https://en.wikipedia.org/wiki/Nutri-Score

echelon2• 2 years ago

Hard to enforce those laws across international borders. Especially if there's an escalation of geopolitical tensions.

Imagine if a country could turn off power to US homes during a conflict. This is critical infrastructure we should be making at home.

Xelbair3• 2 years ago

Here's what I want: by law, any device that i own should work perpetually until broken by me. If it requires 3rd party servers, let me configure alternatives. and if you sunset the servers completely you are mandated to release either: complete documentation how to create your own service to keep device working, or a full binary that supports ALL the features that were available throughout device's lifecycle. If you go bankrupt you are mandated to just open source your software in that case.

and that's a minimum I'll settle on.

RHSeeger5• 2 years ago

If you go bankrupt you are mandated to just open source your software in that case.

Or insurance that covers the complete refund cost of all assets sold. There are cases where you may be using 3rd party software that you license that you cannot open source. And, in that case, you're on the hook for refunding the cost of the item.

WJW• 2 years ago

What about mechanical devices that simply wear out? Even electronic devices can fail due to circumstances controlled neither by you nor by the manufacturer, like lightning strikes introducing violent transients in the grid supply.

Also, cool beans that that is the minimum you'll settle on but how on earth would anyone enforce that? Open sourced software is not enough by far to make something work perpetually: the software will need to be run somewhere and most likely (since you are talking about some sort of net-connected software if this is relevant in the first place) will need security patching to keep up with CVEs. Who is going to pay for that? I don't think it will be the bankrupt entity that stopped existing 10 years ago.

mindslight• 2 years ago

We need straight anti-trust unbundling. You should not be allowed to abuse your market position as a hardware manufacturer to push your network-connected software by tying them together as one product. At a minimum, the software should have to be developed by a separate business unit, using only documentation that's been published for everyone. (and yes, having been an embedded hardware/software designer, including for things like power electronics, I'm quite aware of the implications)

anon8487362815• 2 years ago

That law won't mean much when people are importing products from other jurisdictions that don't have that law... Which is essentially what happened here. (The broken "law" being the exclusivity agreement).

And if we assume that complying with the law somehow increases costs in the US market, people will still go buy the cheaper thing anyway. Which means you need to enforce the regulations on importing these things just as strictly as we regulate the import of cigarettes...

ryao5• 2 years ago

I have a solar edge inverter. I never connected it to the Internet out of concern that this was possible. While it is a different company, this vindicates my concern.

zbrozek2• 2 years ago

Do you have some other mechanism for getting telemetry?

pclmulqdq• 2 years ago

Why do people insist on having remote telemetry from these kinds of appliances?

speransky• 2 years ago

I have inverter of different brand and also had concern to allow it internet connection, so i ended with pi zero connected to it’s internal wifi with socat port gateway, a route on router to simulate it’s internal network and it’s app works thinking it is connected locally to device, even over vpn back to home.

zeroping• 2 years ago

FWIW, the Sol-ark's (and presumably the Deye's) support getting telemetry via local RS-485.

ryao5• 2 years ago

I could use the device’s buttons and LCD to get some stuff, but I generally don’t bother. Maybe if I plugged it into the network and disallowed internet communication, I could poke around to see if there is a way, but I have not felt motivated to try.

RA2lover• 2 years ago

I own a Guangzhou Sanjing R5-8K-S2 inverter that had issues shortly after installation where it was generating far less power than expected.

The web telemetry panel had multiple gaps throughout the day where energy generation dropped to 0, but having datapoints logged every 10 minutes didn't give out enough information to determine why that was happening.

It also had a current status endpoint which updated every 10 seconds. I wrote a python script to log those updates into a file, and eventually discovered the inverter was shutting down itself and waiting 5 minutes every time it found its grid voltage to be greater than 241V.

Installer wanted utility to lower the house's grid transformer tap, but needed authorization from Utility, who declined claiming it was already on the lowest tap possible. Cynically, i think they declined because lowering further would lower grid voltage at night below minimums they're contractually required to maintain.

Tried going into the manufacturer's website to see if a firmware update could solve this. Couldn't find firmware updates, but i did find a manual for their local monitoring app, including a password for installer-only settings, set to "123456".

The app doesn't include any functionality to change said password to something else, so i assume it's hardcoded. There was one change i could still legally do without violating anything - raising the grid shutdown threshold voltage from 241 to 242V. This change did get reflected in subsequent logs, so the settings panel is functional. I could technically increase that further (to a maximum of 275V), but that would expose me to liability.

Parents suggest contacting the inverter's distributor for support, and they asked for a password i was never given. Apparently the manufacturer is suppopsed to create accounts for installers/distributors buying directly from them, and i somehow bypassed that process when creating an account for myself, without even realizing it.

Some more clarification later, it turns out they can still remotely access the inverter with its serial number. After doing so, they "fixed" the issue without explaining how. Checking the installer settings interface, it turns out they just increased the grid overvoltage shutdown threshold to 275V right off the bat.

At least i got them on record saying they did that, so i'm technically in the clear. Still, having that kind of access was scary enough to want to make me disconnect the inverter from the internet.

Turns out its warranty (which only expires in 2036) has terms requiring it to stay connected to the internet. That's enough time to trigger WW3 and a resulting horus scenario (https://horusscenario.com/).

Until then, the best i can do is to throttle the inverter's internet connection to something like 10kbps, which isn't enough to prevent someone persistent enough from uploading new firmware.

Stories like this make me reconsider keeping it connected. I'm surprised we haven't seen inverter ransomware yet.

buzer• 2 years ago

Turns out its warranty (which only expires in 2036) has terms requiring it to stay connected to the internet.

Does it define any requirements for the internet connection? What if the connection latency was, let's say, 1 week?

thot_experiment• 2 years ago

To most of us HN denizens it's obvious that OTA updates and internet connectivity generally leads to the things we rely on being worse. It sucks to have something that works when you go bed and is broken the next morning because of some idiotic update.

What can we do to modify capitalism so that this externality is correctly captured? I think most people, especially those who rely on these systems to do their jobs would tell you "I would gladly pay a premium to prevent outside influences from being able to brick my tractor (or whatever), if it's broken I want to be the one who has broken it." Is this something that could simply be solved by aggressive anti-trust? Surely this isn't the best future we can come up with.

anon8487362815• 2 years ago

First we need an industry certification that encapsulates all the end user requirements. Then we need consumers to vote with their dollars, or regulation to enforce compliance.

rstat1• 2 years ago

I feel like stuff like this shouldn't be anywhere near the internet. Partly because of reasons like this where the manufacturer can just randomly decide to disable it, but also because its usually the software equivalent of Swiss cheese.

forgetfreeman4• 2 years ago

I feel the same way about cars, lightbulbs, and thermostats.

echoangle3• 2 years ago

I’m not saying those things are safe but isn’t the attack surface pretty limited if you are behind NAT/a pretty basic firewall? The only connection to the internet should be the device reaching out to a server and asking for an update from time to time, it shouldn’t really be reachable from the outside. Unless the update server is compromised too, I don’t really see what can happen.

shakna• 2 years ago

Only if that attack surface doesn't include employees, household members, contractors, shared spaces, etc. That is, a small business may be fairly safe if they're no cohabiting. A corporation probably isn't.

relistan• 2 years ago

In this case the manufacturer was the one that triggered it. Even if it weren’t, how secure their servers are, or which foreign legislation they are subject to is a total unknown.

tgsovlerkhgsel2• 2 years ago

These kind of things will stop when they start getting treated as malicious attacks (similar to ransomware), i.e. the perpetrators become wanted people and if caught, see significant jail time.

This goes both for the malicious bricking of normal consumer devices, and attacks on critical infrastructure like this, except of course the punishment for the latter should be correspondingly more severe.

hedora4• 2 years ago

Assuming that one of these inverters is in North Carolina, they’re facing a $250,000 fine (hopefully per inverter), and second degree murder/40 years if anyone died:

https://pemc.coop/bill-protecting-critical-infrastructure-si...

tgsovlerkhgsel2• 2 years ago

Only if this actually gets treated as an attack though, which I haven't seen happen in similar cases in the past.

Sony BMG with the hidden DRM rootkit malware on their music CDs got some civil penalties but no criminal prosecution. Sony with the Playstation OtherOS removal had to pay a ridiculously low class action, no criminal prosecution. Lenovo got a slap on the wrist for putting an adware firmware bootkit into the machines, again civil only.

A lot of companies are still getting away with exfiltrating memory dumps by default as part of their error reporting, selling your location data, etc.

The only criminal prosecution (as in "butt in jail") for similar behavior that I'm aware of is Volkswagen's Dieselgate, and that was only prosecuted because it was seen as screwing over the US government, not consumers.

aldousd666• 2 years ago

Trade wars have casualties. This is the first in a long line to come.

ggm• 2 years ago

If you sold equipment which wasn't certified for connection to another economies electricity grid, and discovered resellers were selling it into that economy, what would you do?

Calling this trade war invokes issues which may exist, but ignores more present dangers. Selling unlicensed radio equipment (--for example) into different economies has massive financial risks.

ajross• 2 years ago

Your position is that "selling unlicensed equipment" is a liability risk for Deye but... deliberately disabling equipment you don't even own is not?!

All they need to do is the same thing any manufacturer whose stuff ends up on a gray market does: "We're very sorry and we don't know how this happened. We'll work with regulators to better audit our export shipments in the future." This kind of thing happens all the time.

walterbell5• 2 years ago

10 years ago, https://www.finnegan.com/en/insights/articles/u-s-supreme-co...

U.S. Supreme Court Holds that Books Printed and Sold Abroad May Be Freely Resold in the U.S. Because the Copyrights Are Exhausted Under the First-Sale Doctrine.. The Kirtsaeng decision is significant to copyright owners, and it may also have important ramifications for patent owners who make and sell goods abroad that practice a U.S. patent.

https://www.iveticlaw.com/owning-vs-controlling-understandin...

The first sale doctrine is a legal principle that limits the copyright owner's control over a particular copy of their work after it's been lawfully sold. This doctrine, in essence, acts to cut off the copyright owner's rights in the created work after the product is first sold (ie. when the copyright owner releases their work into the marketplace). Another way to describe it is that the copyright holder's right to control the distribution of their work goes away after the “first sale” of the work,(hence the name). In more straightforward and more practical terms, once you buy a book, CD, DVD, artwork or any other authorized copy of a copyrighted work, the copyright owner generally loses the right to control what you do with that specific copy. You can resell it, lend it, give it away, or even destroy it, without their permission.

Remote bricking requires software, which is sold under copyright law.

anon8487362815• 2 years ago

And yet some software licenses do dictate what the user can/must do with regards to further modifying or distributing the code.

This also has nothing to do with exclusivity agreements arranged between companies, as seems to be the case here.

walterbell5• 2 years ago

nothing to do with exclusivity agreements arranged between companies

The 2013 U.S. Supreme Court case depended on a plaintiff that was making enough money on textbook arbitrage to fund a legal case all the way to the Supreme Court. It provided new clarity on book distribution and geographical "exclusivity".

If software enforcement of device distribution agreements affects a large enough flow of capital, then corner cases will accrue enough economic impact to be tested in courts. Manufacturers do not have carte blanche to manipulate hardware remotely, e.g. they cannot take actions that could injure humans. Where are the limits? For now, we have many opinions and few laws.

MarkusWandel2• 2 years ago

The root problem, as always, is that it has been normalized that devices in your house, accessed from your house, need cloud access to do this, or even to function all.

Metrics from an inverter, once upon a time, would have been a local web server in the device. Maybe with QR code printed on the device so the typical smartphone user could access it. Firmware updates ought to be physically "opt in" - like stick a USB stick or MicroSD card into the device and push a button.

Not some mysterious cloud that through legal issues, malice or sheer incompetence, can reach in and modify or delete functionality without warning.

My dishwasher has a little nag light to remind me I haven't connected it to my Wifi yet. I never will. It washes dishes just fine.

brianwawok• 2 years ago

Opt in security updates is generally bad. It means 99% will never ever get one.

Opt in for major functionality, that is fine.

MarkusWandel2• 2 years ago

No security updates needed if the device isn't connected to the internet in the first place!

feoren• 2 years ago

If opting out of security updates for your dishwasher is bad, then your dishwasher is shit.

hedora4• 2 years ago

Yeah, but what if a “security update” breaks functionality?

None of the IoT devices we own have had an update that fixes a user facing bug, but most have had critical updates that break existing functionality.

crote• 2 years ago

Metrics from an inverter, once upon a time, would have been a local web server in the device.

Or just a regular serial port! For example, IEC 62056 [0] provides a fairly trivial standardized way to interact with an electricity meter using an IR reader head. Even easier, the DSMR standard outputs serial data via a 5V RJ12 connector [1]. You can connect that to a PC with a $5 USB-to-serial adapter, directly to a Raspberry Pi, or to one of a dozen $20 cloud dongle thingies.

Just mandate a serial interface, and the inverter itself doesn't need any kind of web interface whatsoever.

[0] https://en.wikipedia.org/wiki/IEC_62056

[1] https://jensd.be/1183/linux/read-data-from-the-belgian-digit...

fsckboy• 2 years ago

the word "bricked" was used. are these units actually bricked, as in permanently? or do they perform the same checks every time they start and will simply keep failing till they are in a designated geolocation?

tehcopec• 2 years ago

I've been using Deye since 2019. I was using those blue grid-tie inverters with limiters to do zero-export and they had the Deye manufacturers sticker on it. So I googled them, found them on Alibaba, and purchased some directly from them. I had no idea that they were related to Sol-Ark until later. Fortunately, I never trusted their dongle/cloud solution and so only ever plugged it in when I requested firmware updates from them.

I learned at some point that they were designed with Sol-Ark, and that Sol-Ark put pressure on them to stop selling to the U.S. market, and indeed they stopped selling to anyone directly in the U.S. I get Sol-Ark's position, but there are numerous people that bought them not even knowing the relation nor that they would be considered grey-market (since buying products direct from China is really common). Also, like pretty much everybody I know that screws around with this stuff, none of use would have paid for the Sol-Ark nor were we really aware of them because they so expensive anyways.

Even if this decision came purely from Deye, it's a direct result of Sol-Ark putting pressure on them to stop the sell of inverters into the U.S. market, and enough middle-men were getting around that, that they felt the need to do this.

Again, I get Sol-Ark's position, but it's just a really bad look for them. I think it's similar to certain media piracy: The people wouldn't have paid for it how you are offering it anyways, and all you're doing is making potential future customers hate you; It would certainly be more profitable to focus those resources on something else.

Sol-Ark manufacturer reportedly disables all Deye inverters in the US

Related Stories