Not all automated testing tools support Shadow DOM in web components

https://matuzo.at/blog/2024/automated-testing-tools-and-web-components

by robin_reala • 3 years ago

45 41 3 years ago

bob10297• 3 years ago

That's one more reason not to rely on automated testing only.

I took an alternative conclusion - don't use shadow DOM or other bleeding edge tech when something much simpler will do. What ever happened to using document.getElementById? You can even skip much of this if you finish your work on the server before sending that text/html response...

I sometimes wonder if there's a huge thing I'm missing, but then I remember our customers love our product. Not a single one of them can tell that we don't use react or WASM. In fact, we have yet to receive a single complaint regarding our multipart form posts resulting in actual browser navigation. For whatever reason, the users don't seem to give a shit about this. It only ever seems to come up in the most ridiculous nerd conversations.

colordrops2• 3 years ago

Shadow DOM is not "bleeding edge". It's been in all major browsers for years.

Already__Taken• 3 years ago

shadow don't has always been on the platform the only new big but is an API exposing it. before you just never got to change certain components like select dropdowns

chrismorgan8• 3 years ago

Shadow DOM is frankly more bleeding edge than most new things, in the sense that you’ll cut yourself badly if you try using it much. There are just so many subtle and annoying problems with it because it broke the DOM model that has been around from the start, and now a bunch of stuff is just inaccessible from the DOM. The vast majority of places I see people thinking “Shadow DOM might be suitable here”, it’s a terrible idea. Good places to use it are actually really niche.

Declarative Shadow DOM, more recently added and not yet universally supported, fixes one of the two biggest problems with it (that you have to load and run scripts before it can work), but the other one (that it’s strong insulating encapsulation, which is just occasionally what you want but normally not) is by design and is not going to change.

In the forms they’ve ended up in, Shadow DOM was a mistake (yes, they should have just left browser widgets like <select> and <audio controls> special), and Web Components were almost certainly a mistake.

TheAceOfHearts• 3 years ago

It's fine to use whatever tools you consider most appropriate for your situation and use case.

The tools you mention have helped enable a ton of new forms of development and interactions on the web.

Consider: I was recently baking a cake and didn't have to use a sous vide for anything, I don't understand why people keep hyping those up so much. That's similar to how your comment reads to me.

lcnPylGDnU4H9OF2• 3 years ago

Consider: I was recently baking a cake and didn't have to use a sous vide for anything, I don't understand why people keep hyping those up so much. That's similar to how your comment reads to me.

This analogy seems to work pretty well. On the other side of the fence, I see people talking about how good their sous vide cakes are and how they’d never work at a bakery that didn’t exclusively use a sous vide.

That’s not to say that the cooking method is never viable, just that it’s touted as the only thing necessary. Meanwhile online baking recipes list “make sure you own a sous vide” as the first step.

bob10297• 3 years ago

It's fine to use whatever tools you consider most appropriate for your situation and use case.

1000% agreed.

The part where this breaks down for me is the observation that many developers don't seem to have conducted the barest of due diligence regarding what tools and options are even available.

I can accept that you went with ReactJS, but not if you are operating in a vacuum where access to MDN is restricted and you didn't even consider the merits of a "no-build" web ecosystem.

diggan• 3 years ago

In fact, we have yet to receive a single complaint regarding our multipart form posts resulting in actual browser navigation.

Just out of curiosity, how do you handle errors, specifically timeouts here? I'm guessing the HTTP server serves a custom error page, does it have a reference to the previous page with all the data they tried to submit and so on? Probably how I would have handled it.

Timeouts are a bit trickier, as the user would just see the browser error page and dassit.

Nice thing with decoupling server responses from what the browser actually renders is that you can handle things like that exactly like how you (as a developer) want to, without breaking the current page.

But, as always, it depends. Seems you folks are giving your users a lot of value from whatever you're doing regardless, so seems what you're doing is working :) Congrats.

chrismorgan8• 3 years ago

Such error handling is easier, because it’s a normal part of the flow, rather than a separate path that’s not tested. If submitting fails to load the page, you get the normal browser error page and reload, submitting it again. And that’s what you want it to be, unless you can actively do something else in the meantime (which is still a good deal more complex and not typically supported by web apps). By contrast, my experience says that scripting-based submission is very flimsy in the presence of unreliable network. It’s crazy just how many things enter broken, unrecoverable states because it didn’t occur to the authors that maybe that fetch could fail, and maybe you might want to try again. Or worse, and practically never handled properly, when network conditions lead to a stuck request, where the client is waiting for something that will never come. In the browser, you just press F5/Ctrl+R/⌘R when you recognise this situation, but when scripting-powered the only way of fixing this may be briefly toggling the network adapter in order to kill the request (because you can’t just reload the page because you’d lose in-page form state, another of these things that the browser gives you for free but scripting plays havoc with, though Firefox is best at compensating), and hoping that leads the page to let you submit again.

Seriously, scripting-based submission, navigation, &c. is reimplementing a lot of what the browser give you for free, and it can’t be done perfectly (e.g. tab loading indicator) and what is reimplemented is absolutely never as flawless as possible. Yes, you get some interesting advantages, but you lose a lot too, more than developers tend to admit.

bob10297• 3 years ago

how do you handle errors, specifically timeouts here?

We don't.

We are using stateless HTTP semantics throughout. There is no web socket or other streaming abstraction. You can key in a form, come back tomorrow and submit it and the server won't care (assuming business logic checks out).

If the system is down at submission time and someone wants to retry after it comes up? No problem. The browser will give you an opportunity to resubmit the form data since the prior POST timed out. This has been a thing since ~forever.

The end user experience when the service is completely down is that the browser displays "connection to server timed out" with various initial troubleshooting steps that the vendor manages. Either the server is up and everything works, or it is down and nothing does. Whatever partially-available thing we might be going for doesn't make much sense in my head.

selalipop2• 3 years ago

I recently launched an app in an extremely saturated space (AI characters, https://tryspellbound.com)

Most of my differentiation is in the UX: specifically things like animations that were carefully selected to make the UI feel snappy even though the AI isn't that quick.

It sounds silly, but there are literally 100 apps doing exactly what mine does. If my site felt like submitting a form, and the UI had native selects, and overall it felt like a site that leaned more on native web technology, the demographic that uses it would have shunned it for other options before giving it a chance.

Sometimes you're in a market where you get the luxury of ignoring looks and just delivering value in a way that doesn't allow people to complain, but other times you have to give the people what they want: and a lot of them want that "it feels like an app" experience.

bob10297• 3 years ago

How many distinct aspects of your product actually require asynchronous submission of form data?

A little bit of javascript/AJAX/etc is perfectly acceptable. We didn't get 100% of the way there with multipart/form because iOS/Safari exists and will crash hard if you try to post big forms.

You don't need ReactJS to submit a user input and avoid a page load. This is ~20 lines of vanilla javascript.

a13o• 3 years ago

To this day a lot of frameworks and libraries are built with convenient CSS selector string APIs, thinly wrapping querySelector. These don't work well with shadow DOM.

Passing element references is more compatible, but more verbose. And in the case of test framework POMs, a poor choice - because it leads to stale element references in a reactive or virtual DOM.

Some libs like Playwright are re-living the jQuery timeline and building enhanced selector engines that pierce shadow DOM by default.

We're forced into a future where your toolbox either contains shadow DOM or querySelector.

Shadow DOM's adoption problem comes down to ergonomics. We prefer jQuery's vision of the web, grabbing elements with handy one-liners. Shadow DOM broke that. And the practical benefits of shadow DOM have never outweighed the convenience of selectors.

bob10297• 3 years ago

Some libs like Playwright are re-living the jQuery timeline and building enhanced selector engines that pierce shadow DOM by default.

Our ability to use a testing framework like Playwright without having to resort to a bunch of weird adapter code is a big part of why we went with dumb-simple web technology. Their recording feature "just works" for us and we get perfect replay 100% of the time.

chrismorgan8• 3 years ago

enhanced selector engines that pierce shadow DOM by default

It’s worth noting, for those that don’t realise it, that from client-side scripting you can’t reliably pierce the Shadow DOM. Playwright must be doing this with privileged APIs.

Specifically, although element.attachShadow({ mode: "open" }) sets element.shadowRoot to the ShadowRoot that it returns, element.attachShadow({ mode: "closed" }) only returns the root, and you can throw that away or store it in a private property or closure variable or whatever, so that it’s impossible to access from outside. Yes, shadow roots give you an inviolable membrane (and one that I don’t think content blocker are piercing yet, because not many sites have been abusing it yet). The only way outside code will even be able to discern that there is a closed shadow root is by trying to create one and having that fail (which is a canary, you can only do it once).

gedy2• 3 years ago

I like the idea of web components, but feels like a mistake to have welded shadow dom to the implementation (or at least the default way they are implemented/described/advertised).

The vast majority of devs I've worked with who try and use them are basically using them like JS frameworks or HTML templates in normal apps.

Shadow DOM is just really a poor fit for this type of use IMHO and results in practical issues like the OP.

zaphar• 3 years ago

Shadow Dom is completely optional and you can leave it out if you don't need an encapsulated DOM at runtime. It's not at all welded to the implementation.

gedy2• 3 years ago

Shadow Dom is completely optional and you can leave it out if you don't need an encapsulated DOM at runtime. It's not at all welded to the implementation.

That is partly my point then - every time I've run into this with devs, the defaults, docs, comments, etc assume Shadow DOM is needed/important and then things like the OP bite them.

spankalee• 3 years ago

Shadow DOM is amazing - it enables interoperable composition and has much safer encapsulation again random styles.

austin-cheney• 3 years ago

People over complicate this all the time. Things work as expected or they don't. Things cause interference/regression or they don't. Your tests are not Agile user stories so they should not read like Agile user stories. Your tests should determine if things are broken as directly and clearly as possible without doing more, and they must do so as fast as possible.

At the start, actions from good unit tests in the browser execute user simulated events against DOM nodes the user has access to. That is all, because that is the actual user experience If your testing approach can do this directly, as in without framework magic, it will work the same with your favorite framework and vanilla JS and it will execute much faster.

Then test assertion is as simple as validating changes to DOM nodes, or no changes in the case of negative tests. Such assertions can include the presence and/or visibility of a DOM node, value of text, the size and/or location of a DOM node, presentation, node order, and so forth. User facing events include absolutely everything a user could do with a mouse, touch, or keyboard with only exception to mouse cursor movement.

If that is no clear do nothing else with testing until the above is ironed out. Once the above is solidified more advanced testing can determine whether events/listeners are attached as expected or whether hidden data/metadata is available to the page as expected.

Also, the major common testing tools make use of Selenium styled remote access to the browser via means like CDP or W3C's WebDriver protocol to backdoor a browser. These are really complicated, way more complicated than you need if testing on a site you own. Testing is far more simple, and secure due to same-origin policy, if you conditionally embedded a test receiver script into the page and beacon out test assertion results via WebSocket.

whalesalad• 3 years ago

This would be like testing bytecode produced by Python. Why would you look at an abstraction that is not meant to be looked at? Test the actual DOM.

mikeyinternews• 3 years ago

mabl does https://help.mabl.com/changelog/shadow-dom-support

punyearthling• 3 years ago

shadow DOM doesn't play well with assistive technology, so you shouldn't really be using it in production right now anyways

stuffbreaker• 3 years ago

I haven't had the same experience. Can you provide an example of this?

Not all automated testing tools support Shadow DOM in web components

Related Stories