Barcode Detection API

https://developer.mozilla.org/en-US/docs/Web/API/Barcode_Detection_API

by addcninblue • 4 years ago

331 137 4 years ago

developer.mozilla.org

miohtama• 4 years ago

As the page does not discuss the motivation behind the API, does anyone know why this is ”native” API and not just a generic image processing script implemented with JavaScript / WebGL / etc.?

bena• 4 years ago

Because while barcode scanning is a subset of image processing, it's a use case that's pretty common.

rahimnathwani• 4 years ago

To allow the implementation to take advantage of libraries provided by the operating system[0] (where available), without the web site developer needing to even know that it's happening.

[0] or things like Google Play Services.

PaulHoule4• 4 years ago

At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software. I can tell iOS users to just "scan the code with the Camera" app but I have to tell Android users to find a QR scanner app on the app store.

My experience with an actual bottom-of-the-line prepaid phone was that the first QR scanner app I downloaded from Google Play worked right the first time but I had to try six different apps before I found one that worked with my Samsung Tablet.

A web-based scanner would be a possible answer, though really Google and the phone makers and the carriers should have prioritized a "just works" solution years ago.

Bilal_io• 4 years ago

I've been scanning QR codes with my Android phone for a long time. I know at least two native cameras work, Pixel's and Samsung's since 2019. But I am sure legacy phones with older android versions don't support it natively.

eertami• 4 years ago

Honest question: for what reasons do you ever need to scan a random QR code? I think I've used them to sign in to some apps (eg, WhatsApp) while logged in on a PC, and other than that exactly 0 times.

The Google camera app does support QR codes FWIW, but I don't know if any other vendors except Google use it.

RenThraysk• 4 years ago

It's built into Google Lens. Use it occasionally to send files over wifi to phone, with webwormhole.

SkyPuncher• 4 years ago

At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software

It's hidden, but I believe it's built in the default camera app now.

I've learned that my last 3 phones have had it built in.

easrng• 4 years ago

My Motorola phone from 2017 has QR support in the Camera app.

OJFord2• 4 years ago

As always, some fragment of Android actually got there before Apple (to put it in the Camera app), but no, it's not standard.

My current phone (Nokia 3.4) doesn't have it, but previous one (Motorala One) did. Ironic, since I chose them for being about as close to stock as you can get, both in the 'Android One' programme (so also comparatively LTS).

idealmedtech2• 4 years ago

Native code allows for performance gains that a script (even a performant one) may not be able to match. After all, this is fundamentally an image processing problem, one of the more computationally expensive things a browser has to do.

Gigachad• 4 years ago

Isn't that the point of webassembly? Just seems like we are still moving in the wrong direction on trying to move the whole of NPM in to the browser web apis, rather than building a solid base you can build anything on.

agucova2• 4 years ago

It's a heavily standardized, very recurrent problem that can take a lot of advantage of native code and platform-specific optimization, so it's really an ideal target for a Web API.

afavour• 4 years ago

Barcode detectors can be pretty big dependencies. Given that this will often just be forwarding directly so some kind of OS capability it makes sense to ship in browser.

michaelt3• 4 years ago

does anyone know why this is ”native” API and not just a generic image processing script

Consider the following facts:

1. This was designed by Google

2. Google Play Services provides an API for detecting barcodes, faces and text https://developers.google.com/android/reference/com/google/a...

3. Chrome gets a shape detection API for detecting barcodes, faces and text https://web.dev/shape-detection/

Of course, it's a matter of perspective whether you want to see this as lovers of the free web helping webapps to feature parity with native apps; or Google churning out yet another low-effort standard that's easy for them and difficult for all their competitors.

ape4• 4 years ago

Even thought its on mozilla.org is not supported by Firefox according to the matrix at the bottom.

happytoexplain3• 4 years ago

These docs are a general web developer resource that Mozilla maintains - they are not specific to Mozilla products.

hunterb123• 4 years ago

You'll find many features in MDN docs that applies to.

That's mainly because of the expansiveness of MDN docs, but somewhat due to the funding distribution of Mozilla going to officers, not Firefox devs.

agucova2• 4 years ago

MDN is meant to be a somewhat universal resource for web development. I've even seen official Google web dev resources pointing to MDN.

dblohm72• 4 years ago

And if you look at the badges in the matrix, the API is not a standard yet, either.

PaulHoule4• 4 years ago

... it decodes the barcode too?

soylentgraham• 4 years ago

It does

happytoexplain3• 4 years ago

According to this documentation, yes, it returns the raw value encoded in the barcode.

What it does not appear to do is further decode the raw value into logical data. E.g. data matrices commonly contain GS1 element strings that are not trivial to decode. There are libraries for that, though.

joshua-at-banxa• 4 years ago

Arguably, as a Barcode Detection API that's all it should do.

amelius• 4 years ago

I'm still looking for an open source python library that can decode Data-Matrix barcodes as seen on Digikey bags. Should I give up and switch to JavaScript+browser?

michaelt3• 4 years ago

Is pylibdmtx not up to it?

marcjuul3• 4 years ago

It's not great but libdtmx is the best open source datamatrix decoding as far as I know.

pabs33• 4 years ago

zxing-cpp has a Python wrapper that should work for that.

marcjuul3• 4 years ago

zxing does not have a usable datamatrix decoder last I checked. it's there but it's no good.

schappim7• 4 years ago

For Digikey bags (we make eletronics) we use SwipeTrack app + JS.

Eyght• 4 years ago

I would love to use this for pairing a product with the correct image by decoding barcodes in the image.

jacob019• 4 years ago

I recently made a webapp for scanning barcodes using the phone camera for warehouse use. I used QuaggaJS. I disabled the barcode locator because it was too heavy, instead I just draw a box on the screen that the barcode is to be positioned in. The barcode detection and decoding happens entirely in the browser and it works very well.

egfx2• 4 years ago

I also built a web app for barcode scanning. For testing quaggaJS is fine but it’s highly inconsistent and requires an extreme degree of customization to get adequate results. I recommend scandit. It’s a good production ready library with a very nice web sdk.

silviogutierrez• 4 years ago

Not OP, but I looked into scandit. While excellent, it's quite expensive right?

WesleyJohnson• 4 years ago

My team was trying to get a JS solution working as well, using zxing-js. The barcode basically had to be perfect size, on a flat surface and no glare. we ended up swapping it out for ScandIT's Web SDK and have been very please with performance. As someone else said, it is quite expensive and being a public-facing web application, we had to get a per-scan license vs device license. We're going under contract for 1 year with a "shot in the dark" scan estimate and will renegotiate next year once we have usage statistics.

Would be nice to see if this new spec gets full adoption so we can avoid high licensing fees.

timando• 4 years ago

I wanted this to be a barcode scanning API where a web app can use the camera to scan a barcode without being able to see anything else in the image.

bob10292• 4 years ago

This stuff has been a nightmare for us for a long time. We have to scan pdf417 barcodes from physical identification as part of our offering.

Recently, we stumbled upon some cheap USB keyboard emulator 2d scanners that can pick these up very reliably. For our product/customers, this is a viable path. We have tested probably 30 different webcams by this point. The only 2 that are as reliable as the handheld CCD scanner are the iPad and my Canon DSLR tethered to my PC in automatic mode with a macro lens.

Resolution is a factor, but so is frame rate and environment. You have to consider the full UX stack. Someone is required to physically position the barcode in a particular way with certain lighting constraints, etc. With a handheld CCD scanner, you get a laser guide for precisely what distance to go for, and an ergonomic experience that is much more conducive to successful scans. Most offer on board illumination, so even in a pitch black room you will get a successful scan.

jaxn• 4 years ago

We also scan pdf417 from ID cards. In our case, it was much easier to just use hardware 2d scanners. We are a b2b SaaS and they are scanning IDs all day, so a hardware solution makes sense.

The weird thing about pdf417 is that there is a lot of helpful info our there, up until a point. It's like everyone is collaborative and asking/answering questions up until the point they really figure it out, but those last few steps never much make it into open code.

A couple times a year I go searching to see if anyone has implemented solid pdf417 in WASM yet. It seems like a natural fit to me, and I am sure people have done it, but I have yet to see anything publicly available.

schappim7• 4 years ago

What device are you using to scan? If you can use Android, Chrome supports pdf417.

schappim7• 4 years ago

Here is how you can do it on Android: https://files.littlebird.com.au/barcode9.html

apinstein• 4 years ago

I used the microblink one for a small toy project and it worked well in all browsers.

https://demo.microblink.com/self-hosted-api/pdf417

disillusioned2• 4 years ago

We ended up going with the purpose-built barcode sleds from Infinite Peripherals. The Linea Pro, I think. Extra battery, onboard illumination/laser guide, _really_ fast barcode engine in any orientation, and a few pretty simple SDK options. We ended up using "our custom webkit browser will fire a custom JS event on scan", which worked perfectly for our purposes.

Everything else is measurably slower: waiting for focus in a dim room, low framerate, weird other issues.

schappim7• 4 years ago

We use the Linea Pro too w/ the SwipeTrack browser making them available in our web app. We've previously used both the Infinite Peripherals SDK w/ an iOS app, but SwipeTrack have a pretty good JS API [1].

Funnily enough, the Linea Pros have been going down hill. They swapped out the barcode engine and it is not fast / crisp. Sometimes faster to just use the iOS Camera (w/ Swipetrack) instead of the Linear Pro's sled.

[1] https://files.littlebird.com.au/SwipeTrackBrowserAPI_200-Yre...

unqueued3• 4 years ago

I found the most reliable barcode scanner to be an old Symbol usb scanner.

I also found that I could scan barcodes off of a screen with a laser scanner if I put a translucent white plastic bag over the screen. Maybe someone with more knowledge about optics could give some insight into why that trick works. My understanding is that laser scanners should only be able to scan by reflection, and shouldn't work off of a screen at all.

masklinn• 4 years ago

Note that they’re scanning 2D “pdf417” barcodes.

Reading a linear barcode is quite easy, the shitty webcam of my 2010 macbook pro was enough for Delicious Library to pick most every article instantly, usually faster than I could even present the article.

danachow• 4 years ago

Purpose built barcode scanners typically don’t fumble with autofocus have faster 0-latency autoexposure (adjusted during frame readout) and even the good 2D CMOS sensor ones will start decoding linear or stacked codes such as PDF417 while the frame is being transferred. That can’t be beaten for latency.

fennecfoxy• 4 years ago

Idk but whatever library the NHS website is using to scan the QR codes on LFTs is flawless, as soon as the camera view opens it immediately scans, you'd miss it if you blinked.

djbusby• 4 years ago

There is a full copy of the zxing library in JS. It's been working great for my purposes - scanning QR and C128 Barcodes.

E: https://github.com/zxing-js/library

bob10292• 4 years ago

This is what we use as a fallback if the user doesnt have a handheld 2D scanner. Very fast & accurate in most of my testing.

asyncscrum• 4 years ago

Used this in a b2b saas for a few years. Worked well and helped remove complexity in a previous way we did it that relied on a native app called pic2shop. We lost a bit of functionality specifically a clear target zone, green scanning line and better low light support. It made up for it in ease of support and reduction in perceived jank.

crossroadsguy• 4 years ago

ZXing is/was solid. I used it for Android, then moved to (suggested) Google’s ml-kit offering around a year ago.

ZXing is pretty much abandoned now. It was anyway “kind of” from Google.

gruez2• 4 years ago

Secure context: This feature is available only in secure contexts (HTTPS), in some or all supporting browsers.

What's the point of this? I understand why you might want this for APIs that expose private user information, but this just performs some computation. You could do the same thing with a js library or webassembly. Is this just to punish http sites?

dblohm72• 4 years ago

Basically, yes. Browser vendors have had an agreement for some time now to only expose new APIs in secure contexts in order to encourage HTTPS adoption.

wyager• 4 years ago

I'm not sure I feel great about gating feature A behind completely unrelated feature B just because manufacturers like feature B. Even if feature B is concretely pretty good (which is debatable for https as it exists now), this seems bad on principle.

makeitdouble3• 4 years ago

I am in agreement with wyager, this feature looks like it could be heavily beneficial to a shipping company's internal web app for instance, or other applications that could be running completely isolated from the global network (or running on the machine itself for that matter).

Forcing https in these applications is a waste of time and money for orthogonal purposes. I guess it could be seen as a cost of dev. for any web based application from now on, but it's sad to me to see the barrier to entry get higher for these reasons.

chiph• 4 years ago

pdf417 barcodes are commonly used on US driver's licenses, and contain private information like the person's address, height, eye color, full name, etc. For the API to do its work in detecting a valid barcode, it needs to be able to read the entire barcode to compute a checksum/checkdigit. Which means it has to read the data.

gruez2• 4 years ago

But this API doesn't allow access to the camera. The caller is responsible for passing in an image to be parsed. Saying it needs to be blocked for that reason is like saying parseInt() should be blocked.

imajoredinecon• 4 years ago

There might be more to the privacy characteristics of an API like this than meets the eye. For instance, if a user grants camera permissions scoped to http://insecure.website in order to scan a QR code and get it detected, it might be surprising to the user that on-path attackers would be able to see the user’s camera snaps. Making the API work in a degraded mode in insecure contexts (e.g. by disabling particularly privacy-sensitive data sources) might be a plausible alternative, but at this point it’s arguably cleaner and less risky to just make the interface HTTPS-only in order to avoid making developers and users reason through these nuances.

xg153• 4 years ago

I don't get it. The privacy risk in there is from getUserMedia(), which is already (correctly) https-only. There is no privacy risk coming from the barcode API itself.

eternityforest2• 4 years ago

I'm not a fan of the whole secure context separation thing. A lot of cool P2P stuff is impossible because a service worker can't access an HTTP server, which would be very useful for web sites to talk to IoT devices.

If there was something sensitive in front of my camera, I wouldn't give any random site permission anyway.

File system access should definitely be secure only, the rest might not need it.

Of course if Mozilla actually did FlyWeb we could almost totally deprecate HTTP except for some public legacy sites.

matthewmacleod• 4 years ago

Huh, TIL. Recognition performance is pretty modest (on Chrome/Mac at least) but it's fast and probably more than adequate for most close-up mobile device scans. It basically just wraps MacOS's Vision framework on the Mac, and ML Kit on Android.

pabs33• 4 years ago

Anyone know what the backend for this is? Is it zxing-cpp or similar?

Edit: guess it depends on the browser and OS.

sleepy_keita• 4 years ago

I bought my first Android device to try this feature out. It's so much faster than zxing on iOS.

cyode• 4 years ago

Bit off-topic: can anyone recommend resources (videos/papers/tutorials) that could help me learn how QR barcode detection and scanning itself is implemented?

I've been thinking that building a basic scanner from scratch (i.e. only using vanilla cpp/python/whatever + image processing libraries, but nothing barcode-specific) could teach me a lot.

lgvld• 4 years ago

I guess it helps to understand how to generate a QR code: https://www.nayuki.io/page/qr-code-generator-library (see also the references at the end)

Good luck. ;-)

marcjuul3• 4 years ago

Does anyone happen to know where in the Chromium code base the implementation is hiding?

The android version seems to use a library that's part of Google Play Services:

  /**
   * Implementation of mojo BarcodeDetection, using Google Play Services vision package.
   */

and

  // The vision library will be downloaded the first time the API is used
  // on the device; this happens "fast", but it might have not completed,
  // bail in this case. Also, the API was disabled between and v.9.0 and
  // v.9.2, see https://developers.google.com/android/guides/releases.

From: https://github.com/chromium/chromium/blob/c4d3c31083a2e14812...

The desktop version references a third party library called "barhopper" here:

https://github.com/chromium/chromium/blob/e1e495b29e1178a451...

and barhopper seems to come from a non-public Google repo:

    'src/third_party/barhopper': {
      'url': 'https://chrome-internal.googlesource.com/chrome/deps/barhopper.git' + '@' + 'ad3c4382875afdd0340f1549f8b9c93cbbc16e37',
      'condition': 'checkout_src_internal and checkout_chromeos',
  },

from https://source.chromium.org/chromium/chromium/src/+/master:D...

I didn't realize that Chromium had missing APIs compared to Chrome but maybe that's always been the case? This appears not to be a W3C standard:

https://wicg.github.io/shape-detection-api/

I was interested in this because there are no really good open source DataMatrix decoders (to my knowledge) with the one in ZXing being basically unusable (last I checked) and libdtmx being ok but not comparable to commercial offerings.

sevenf0ur• 4 years ago

Yeah, I haven't been able to find any decent open source DataMatrix libraries as well. If you find one I'd love to know.

This feature uses native libraries and only works on Chrome OS, Android, and MacOS currently according to this page:

https://chromestatus.com/feature/4757990523535360

noodlesUK• 4 years ago

I’d love to see this in safari on iOS as well. The native barcode detection capabilities in iOS are substantially better than libraries like zximg, and I like barcodes as an interface between the real world and tech. I also wish text recognition were in safari.

schappim7• 4 years ago

Here is an example of using the Barcode Detection API w/ WebRTC on Android / Chrome: https://files.littlebird.com.au/barcode20.html

derac• 4 years ago

I forked another person's code and added bounding boxes, if you want to check it out. Seems like it works pretty well, though the bounding boxes are erratic.

https://gist.github.com/derac/9dfa8884bb04df59ac498d3bac4a93...

https://htmlpreview.github.io/?https://gist.githubuserconten...

paxys• 4 years ago

The dream of the early 2000s is finally being realized.

adamrezich• 4 years ago

all of these small features seem good in aggregate but each one makes it that much harder to ever build a competing browser, especially from scratch.

duck-valentine• 4 years ago

There's a couple good datasets with trained models on Roboflow Universe that would pair nicely for a Computer Vision project/app:

Object Detection for QR Code and Bar-Codes - https://universe.roboflow.com/new-workspace-mrc2b/optiscan-l...

another one for just QR Codes - https://universe.roboflow.com/lihang-xu/qr-code-oerhe

numlock862• 4 years ago

Cool. Another feature that Firefox won't support. Same with the WebSerial API and others. I sort of understand their reasoning that they want to make a web browser and not an application platform, but that's where the web is shifting to ... or rather has already for a long time. And people wonder why every new tech/browser is based on Chromium all the time. Well, this is one of the many reasons.

schwartzworld2• 4 years ago

Why would Firefox implement non-standard APIs?

numlock862• 4 years ago

I get what you are trying to bring up as an argument, but even Firefox doesn't argue this way. See their answers on issues related to these kind of APIs.

Scoundreller• 4 years ago

I’m happy to see things like this. My ISP used to ask for a picture of my docsis modem sticker because people kept making mistakes entering in the MAC address.

Unfortunately, the csr was just keying them in and keyed mine in wrong so I was without internet for a few days.

Something to automatically pick it out would be a lot better.

veysiertekin• 4 years ago

A different context but related: a website plugin that I have built years ago for barcode scanning machines: JBarcode - https://github.com/veysiertekin/jbarcode

I didn't made any changes since then. But it can be easily ported into a more modern framework (React.js etc)

xg153• 4 years ago

It's a nice feature, but, honest question, why is this a web API?

I don't see anything this has to do with browsers. This could be implemented perfectly well in a library.

bricss• 4 years ago

In the latest Chrome: > Uncaught ReferenceError: BarcodeDetector is not defined

Barcode Detection API

Related Stories