A DIY near-IR spectrometer

https://caoyuan.scripts.mit.edu/ir_spec.html

by radlad • 3 years ago

365 73 3 years ago

caoyuan.scripts.mit.edu

progbits3• 3 years ago

This is very nice, but most of that $10k probably pays for certification that you won't get for DIY. Great for hobbyist, but you couldn't sell it for $9500 profit.

version_five• 3 years ago

My first thought as well. People tend to misunderstand why things cost what they do, for stuff like this it isn't the raw components.

MostlyStable2• 3 years ago

Possibly, but he also describes pretty cheap and easy calibration methods. It's possible that most of the cost of the commercial options is the fact that these are extremely low-volume devices and the overhead of a business with low-volume sales is quite high. You might be paying less for calibration and more for all the things a business needs to do/have that a lone DIYer doesn't.

Not to mention the fact that a lot of researchers are buying thing with grant money, and so can be, in some cases, somewhat price insensitive.

I don't doubt that the commercial one is pre-calibrated and certified. But I would be quite surprised if that certification/calibration _actually_ cost ~$9000.

progbits3• 3 years ago

Sure I'm not saying that. If you give away the R&D for free and don't seek profit you can build it much cheaper.

But self-calibrating is still something else. Unless I pay some lab with traceable calibration to do it for me I can't use the results to certify other equipment for example. I think it's more like insurance, sure the unit cost is low but you pay extra so in case the lab screws up they pay for the mess.

jacquesm3• 3 years ago

So what? If you need it, now you can build it. As a hobbyist you don't need certified gear, you need gear.

MrBuddyCasino• 3 years ago

The point wasn't "you can get rich selling this", but "you can save $$$ building this yourself as a hobbyist".

progbits3• 3 years ago

Absolutely, you are correct.

ortusdux• 3 years ago

I wonder what 3rd party certification would cost. Industries that use sensors like this frequently require annual recertification, so there are typically multiple testing houses and competitive prices.

jacquesm3• 3 years ago

There is a huge difference between calibration and certification. Calibration means that your instrument gives you absolute rather than relative output. Certification means that your instrument is precise enough to be used for specific procedures.

Calibration can be done in house, certification is usually the domain of some certification institution and can be extremely expensive depending on the kind of gear.

HerculePoirot3• 3 years ago

DIY recertification obviously. What's the lowest cost for the hobbyist ? A high-end, stable, double-wall PH-meter or finding a cheap way to build single-wall PH meters ?

https://www.instructables.com/cheap-DIY-electronic-pH-meter/

You explore the solution-space, and suddenly, out of nowhere it changes your perspective on another aspect, and you realize you may not need a spectrometer at all to measure ion concentartion in aeroponic nutrient solutions.

http://jupiter.plymouth.edu/~jsduncan/courses../2011_Fall/In...

CamperBob2• 3 years ago

If you're using it in a lab, you can calibrate and verify this sort of instrument yourself most of the time. Wavelength is trivial to calibrate. Absolute amplitude not so much, but you usually don't care that much about absolute amplitude. Response flatness across the spectrum may or may not be a concern.

s0rce• 3 years ago

I'm not sure about certification but there is a lot of overhead in designing, producing, selling and servicing products. This is basically a prototype. It will probably work for a while but will be difficult to maintain if something goes wrong and the software won't be very polished. There are a lot of non-BOM costs that go into selling a commercial spectrometer. This is great if you want to DIY one but even a single sensor based visible light spectrometer with cheap parts can't be bought for close to the sum of its parts.

w10-1• 3 years ago

DIY is not just for hobbyists.

Companies face the build-or-buy question every day. This article demonstrates that you can build and validate your build with the knowledge and skills expected of most principal investigators. If I were a lab director, I would want my people to be able to consider building when it makes sense, particularly since we can then integrate systems and tailor validation to our requirements.

That's the stuff of proprietary IP and career advancement.

gaze2• 3 years ago

This write-up and project is from the same guy that discovered superconductivity in bilayer twisted graphene. Pretty impressive.

nielsole• 3 years ago

I don't know how uncommon that is, but more than 14k citations at age 25, most of which as first author sounds pretty darn impressive.

darkclouds• 3 years ago

Not knocking his efforts, but I guess its on a par to writing some code or making something which is used by many people.

SaulJLH• 3 years ago

Aside from pure curiosity and or research purposes... Is there any everyday/practical apps/uses for something like this, for the avg joe? Building something for 500 that would normally be 10k, already has me intruiged.

thatcat• 3 years ago

You could run Quality Control on your amazon chemical purchases if you were willing to put in time to find methods using spectrophotometery in this wavelength range.

HerculePoirot3• 3 years ago

I brought this up in a comment in another thread. I found this page because I wanted to see whether it was feasible to monitor nutrient concentration in an aeroponic solution. Typically the solution circulates between the tank and the root box, so the concentration in nutriments will decrease gradually. Near-IR isn't enough to figure out these details I believe (I'm not a scientist), but it is enough to get a fingerprint of the nutrition "equilibrium". These solutions are probably the most technical aspect of aeroponics, so I wanted to use commercial solutions bundled as three bottles you're supposed to mix in various proportions. I think that if you explore the concentrations space using such a spectrometer you may be able to train a model to interpolate the concentrations from spectrographs measured in "production".

Anyway I think this is best to address the shortcomings of circulating the solution, for instance use a tiny tank, circulate for a few hours/days, refill, dosing as you go. A spectrometer would be still be cool in order to get real-time concentration diffenrential for each solution component. Identifying individual chemical species could be accomplished I think using far-IR spectroscopy with a model trained on data measured using more expansive spectrometry techniques.

I haven't grown anything yet, I want to build a sap flow sensor first.

https://edaphic.com.au/products/sap-flow-sensors/

https://dynamax.com/products/transpiration-sap-flow/dynagage...

analog31• 3 years ago

I read a lot of articles of this nature, not necessarily in this specific subject area, and may be guilty of writing some myself. I rarely build anything exactly to its original plans, but I take away bits and pieces, theory, and techniques, and re-combine them for my own purposes.

The value of the whole thing hanging together as a product is that you know the person has worked out the bugs of the individual parts, to the point where the whole thing is testable and reproducible.

This is actually true of the scientific literature as well. People rarely reproduce entire research projects, but often borrow bits and pieces and adapt them for their own use.

keithnz• 3 years ago

I used to work doing software for fruit sorting machines and we had a NIR Spectrometer to work out the sweetness of fruit, so when I'd go to actual sorting sites, especially in California which have these machines on a massive scale, I'd use it to sort out the biggest sweetest pieces of fruit.

starshadowx2• 3 years ago

I don't know enough to know if this one would work for this kind of usage but cheaper and easily accessible spectrometers would be amazing for harm reduction and drug testing usages.

rolph• 3 years ago

>While a silicon-based camera is as cheap as dirt these days, an one-dimensional InGaAs pixel array already costs upper few thousand dollars. Any full-blown IR spectrometer system goes way over $10k, with their fancy thermoelectric cooling and precision gratings (we actually have one in our lab). The reason why they are so expensive is that the target user group are scientific researchers, not consumers. <<

the expense is a product of intended use. performance guarantee, is responsibility for design failure. scientific instrumentation must be very consistent for the purpose.

SV_BubbleTime• 3 years ago

Yea ok but…

Certain things just need an answer quickly. Like, some science that you do in order to do other science. And you can’t get there without a $10,000 device, you can’t even get started with some other experiment.

There isn’t anyone making hobbiest optical sensors. I looked for a project I needed a laser for. My options were, don’t do it, or pay $4000 for a late 80s sensor I needed off eBay then try and figure out how the thing worked with no software and a proprietary cable. Sound like fun? Not to me. And I didn’t have $10k for the USB version from the early 2000s, again with no software. And obviously far from thr $50k for a UBS3 new model.

Optical equipment is stupid expensive and sometimes you just need “basic tools” for advanced things.

HerculePoirot3• 3 years ago

One aspect in the price of scientific tools is alignment with know physical units and breadth+flatness of measurement. If your goal is to control a dynamic system within a narrow band, you may cut on price to fit for your specific needs.

ajsnigrutin• 3 years ago

Well sure, but with time prices drop, and with lower prices, you get even unscientific uses.

Flir units were for army use, now you can buy cheap chinese ones (without US export restrictions) for a few $100 in a form of a dongle that you plug into your phone and see what part on your PCB is overheating, or where your house insulation is bad.

Keeping liquids at precise temperature was a thing used in labs, and now you can buy a sous-vide machine for $50.

Even medical stuff, such as blood oxygen meters are at single dollar/euro prices now.

So yeah... it's a circle... if prices go down, more people will find more uses for it. If more people find more uses, more companies will produce them and prices will go down more.

lawlessone• 3 years ago

Between this and the DIY radio telescopes and wifi radar, maybe we will have a real tricorder some day.

0xdeadbeefbabe• 3 years ago

hehe no mention of the new iphone 15

JKCalhoun• 3 years ago

I'm just happy with all these "Amateur Scientist" links lately. More of these!

0xbadcafebee• 3 years ago

Articles like these are rare, but there's 30 years of them scattered around the internet. There's got to be a way to catalogue them in a single index somewhere. Like a Wiki, but just articles that are hard to find and extremely interesting.

I'd like an index because HN's articles are often not this caliber of "interesting". If I look back 1 month to the top of HN (https://news.ycombinator.com/front?day=2023-08-12), there's pontifi-posting (editorials), news and editorials from large media companies, blogspam from tech companies and OSS projects, and basic tech How Tos. I don't want to read any of that; I come back hoping there's a single one of this kind of article, and maybe find one a month.

EricMausler• 3 years ago

Sounds like you want a curated feed. There are some email mailing lists that aim to achieve this, usually for a specific domain (like stock market, or AI, etc)

Maybe a group chat or discord of like minded people who casually share things they come across that pass a high standard

One way or another it seems like you're going to need to rely on another person to sift through all the daily published content and mark potentially interesting ones

MayeulC• 3 years ago

I wonder if you could just feed HN titles with more than ~30 upvotes to an LLM and ask it to bring out similar topics. Not perfect, but I think it could work as a start.

mdip• 3 years ago

I have to agree with you. While I do enjoy the OSS projects/physics-/math-related topics (and a lot of the Show HN), these are my absolute favorite things to read.

Curious if you've found other resources that provide more of this sort of content?

I've found the Hackaday blog[0] tends to have a lot of this kind of content (often summarized/linked to the original source) and sometimes it's tagged in a way that makes surfacing others from their archives possible, but I've not found any other sites that are reasonably organized to help surface write-ups of this kind[1].

[0] https://hackaday.com/blog

[1] There are subreddits where this sort of thing can be found, but "deep-dive but accessible technical articles" tend to be placed in a sub-reddit that's dedicated to more narrow topics and a lot are lower quality (there's similar content to this in /r/Optics from 2022, but I'd never have a reason to end up there)

walterbell• 3 years ago

Manually curated list could be crowdsourced via an OSS project, e.g.

  1. Criteria for articles in list.
  2. Example and anti-example articles.
  3. Tag via HN comment with short, unique and human-meaningful phrase.
  4. Query Algolia periodically, triage, PR submission/review/merge.
  5. Syndicate list as RSS feed.

fanick• 3 years ago

Similar project with single photo diode enclosed in sort of a pinhole camera: https://hackaday.com/2016/05/18/using-missile-tech-to-see-li...

client4• 3 years ago

A different design could leverage laser micro ablations on glass over a traditional CMOS sensor.

oceanplexian• 3 years ago

I wonder if this could be used for amateur astronomy? Would be cool to point a telescope at a star, and print out a spectrum corresponding to elements with emission lines.

jacquesm3• 3 years ago

For that a prism and a photographic film would probably be sufficient, the developed film would serve as your print-out.

z3phyr• 3 years ago

Maybe we should have more fundamental science collectives where amateurs can also contribute, in addition to academic and state funded institutions. Sort of like amateur astronomy clubs [Amateur astronomers still contribute widely to their field(s)], in larger scale and contributing in much more significant ways.

Yes to more amateur scientist articles on HN!

KRAKRISMOTT• 3 years ago

The author's clearly not an amateur. He's a condensed matter postdoc.

swores• 3 years ago

I think there's an argument to say that he was doing it as part of his work, but I'll make this point anyway: if he was doing it for fun and to entertain people in a way unrelated to his actual job, wouldn't it still be "amateur science"? The same way that if a premier league footballer or an MLB hall of famer went with their family to play a game in the park, that's amateur football/baseball despite being done by a professional.

And while one could argue that the words used were "amateur scientist" rather than "amateur science", it could also be argued the opposite direction by suggesting that tag might not actually apply to the author but instead to the type of content since it's to be read and perhaps replicated by amateur scientists more than professional ones.

Or we could just celebrate the article regardless of the accuracy of someone's throwaway remark's categorisation of it :p

bkloppenborg2• 3 years ago

As you mentioned, astronomy is a field where contributions by amateurs / citizen scientists are extremely valuable. A few organizations that exemplify this are: (1) the American Association of Variable Star Astronomers (disclaimer, I'm their Executive Director) which collects photometric (brightness) data as well as spectroscopic data on variable stars and kindred objects including exoplanets (2) the Society for Astronomical Sciences which is more broad than AAVSO, but with a moderate focus on instrumentation (3) the Center for Backyard Astrophsyics (hyper specialized on one type of variable star, a good collaborator of the AAVSO) (4) the International Occultation and Timing Association that observe asteroids occulting (blocking) stars to infer their shapes

Most countries have organizations similar to these too.

Edit: There is a group within the SAS working on an automated optical spectrograph for astronomy called the FlexSpec 1 (https://flexspec1.readthedocs.io/en/latest/). It is about $500 in parts. Similar devices sell for about $3,000.

pavel_lishin• 3 years ago

How plausible would it be to contribute if you live in fairly bright suburbs?

derrida• 3 years ago

more fundamental science collectives where amateurs can also contribute

Look anywhere there is field work out in nature, there's amateurs discovering new species, fossils, minerals etc.

ckocagil• 3 years ago

Nice project. A low DA capacitor Cf in parallel with the gain setting resistor Rf can average out more noise if needed. It can even deal with the mains noise if that remains an issue. Another potential improvement: addition of an optical chopper wheel to deal with the entire system's 1/f noise. The downside is this would require higher sampling rate which would then get demodulated + filtered externally. Would also limit Cf to higher frequencies.

jamal-kumar2• 3 years ago

Wow this is the coolest thing. I was looking at trying to build a DIY raman spectroscope and you can make one of those for under 100$ [1], but near-IR spectroscopy opens up more possibilities for sure. From what I understand (I'm not a scientist but I love the idea of having access to these tools) raman spectrscopy is limited to asymmetric crystalline molecules, so you can't really get a good reflection off of say salt to get a reading. (Edit - Looks like Near-IR is complimentary - "Raman active vibrations aren’t visible in the infrared for molecules with a symmetrical stretch. Similarly, infrared active vibrations aren’t visible in the Raman spectra for molecules with asymmetric stretch. This is known as the Principle of Mutual Exclusion and is what makes NIR and Raman Spectroscopy complementary techniques." [2] ) What we're really hoping to find out is if we can detect stuff like pthalates in plastics and heavy metals such as cadmium in metal products, I understand we'll probably just want to send things off to a lab to get GC/MS done for those things after talking to some people who have more experience than I do but I still want to see if I can build a bunch of these raman or near-IR spectroscopes for pharmacies in the third world. Counterfeit drugs are a real problem out there and they don't need to be.

[1] https://www.hackteria.org/wiki/images/a/a0/MobPhone_RamanSpe...

[2] https://www.labmate-online.com/news/mass-spectrometry-and-sp...

djsamseng• 3 years ago

Thanks for the link! I’ve been trying to figure out how to buy/make a Raman spectrometer for cheap (currently in a third world country too!). Have you built one yet? I’m having trouble finding the lenses needed (mostly because of my lack of knowledge). Any chance you know what to use?

Laser ($40): https://a.co/d/0wjNGBz

Diffraction grating ($12): https://a.co/d/6bpO8xm

Laser focusing lens: Not found

Fluorescence collection lens: Not found

Focusing lens: Not found

Collimating lenses: Not found

jamal-kumar2• 3 years ago

I haven't found the lenses yet although supposedly https://www.thorlabs.com/ has them and someone else recommended these guys to me for finding more high quality diffraction gratings than the 'rave goggles' quality you can find on amazon https://www.edmundoptics.com/

herf• 3 years ago

I think this design is usually called a "scanning monochromator" - really nicely done.

pimlottc• 3 years ago

To save time waiting for the content to animate in:

  javascript:document.querySelectorAll(".animate-box").forEach(e => { e.classList.remove('animate-box') })

mensetmanusman• 3 years ago

https://petapixel.com/2019/07/13/shooting-high-res-thermal-p...

One of my favorite photoshoots merges IR and visible.

txnf2• 3 years ago

I don't understand how it scans to record the spectrum with a point detector? did I miss something?

txnf2• 3 years ago

oh n/m it has a lead screw and moves it across the focal plane of the spectrometer..

tambourine_man• 3 years ago

Great content and form. The site is clean, yet still uses tasteful transitions and typography.

vondur• 3 years ago

This is pretty cool. One of the more fun things in O Chem lab was using the IR Spec on liquids. Preparing solids for IR analysis was not fun.

helpfulContrib2• 3 years ago

I have one of these[1] Spectruino spectrometers laying around, and I've occasionally used it for fun projects over the years, most recently in a "jar of pond water" experiment, wherein I attempt to keep a glass of pond water alive so that it grows its own ecosystem[2].

I had some success with this pond water experiment, such that I was able to observe multiple Hydra generations feasting[3] on the fruits of my labors.

I mostly used the Spectruino for 'marvel' factor, as in .. was able to see the spectral signature of the jar experiment change over time. I regret that I don't have my daily spectral log images available at the moment, but its something I always wanted to get back to .. one of these days ..

[1] https://www.myspectral.com/project/spectruino-three/

[2] https://imgur.com/user/waterplusalgaeequalshydra

[3] https://imgur.com/gallery/e5WLDqc

jylam• 3 years ago

Hey nice Atmos and Microdisc btw :)

helpfulContrib2• 3 years ago

Yup! Great machine!

jtagen• 3 years ago

Is nobody going to address why copper winding wire was used for DC power and signaling?

jensgk• 3 years ago

Very cool! NIR is often used to measure protein, fat, water and fiber contents i corn, animal feedstuff. I used to make calibrations for that in the 90's. I wonder if this would be able to do that?

quanto• 3 years ago

The key implementation to save the cost is this:

Next comes the detector. How are we gonna detect light from a range of angles with only a single 'pixel' of photodiode? We mount the photodiode on a motor and scan it across! To do this, I purchased a stepper-driven linear stage from Amazon for $50. This little stage is quite well-built and allows a resolution-per-step of 5 um, more than enough for our resolution.

So instead of using an expensive array of InGaAs sensors, he uses a single sensor ("pixel") and use a motor to physically scan across space.

For my previous work, I used industrial and scientific IR spectrometers costing above 100k USD. While the sensitivity of the physical machines is important (especially for basic scientific research), it's often the software that makes or breaks the usage case. Modern spectrometer software contains advanced signal processing and machine learning.

A DIY near-IR spectrometer

Related Stories