Strange Noise in Gravitational-Wave Data Sparks Debate

https://www.quantamagazine.org/strange-noise-in-gravitational-wave-data-sparks-debate-20170630/

by digital55 • 9 years ago

90 46 9 years ago

quantamagazine.org

justifier• 9 years ago

code links:

original criticism: http://www.nbi.ku.dk/gravitational-waves/correlations.html

response: https://github.com/spxiwh/response_to_1706_04191/blob/master...

definitely click through the quanta article's links.. fascinating reads all the way down

hcrisp• 9 years ago

It is fascinating, and I learned that one of the events was first detected by an unmodeled search for strange events in the signals:

"Finally, LIGO runs 'unmodelled' searches, which do not search for specific signals, but instead look for any coherent non-Gaussian behaviour in the observatories. These searches actually were the first to find GW150914, and did so with remarkably consistent parameters to the modelled searches..."

http://www.preposterousuniverse.com/blog/2017/06/18/a-respon...

unlikelymordant• 9 years ago

I have done lots of signal processing. You cant just subtract out a noisy signal and complain the residual is correlated, there will be bits remaining. If they want to show correlated noise, why cant they use any other part of the signal?

yodon5• 9 years ago

The correlated remainder is only in the vicinity of the signal event.

The 2nd team is arguing the presence of residual correlations after the signal is removed means the total correlation isn't well described by a gravity wave signature.

This is experimental data, so there's clearly going to be noise and there's clearly going to be complexities in the detector response to the event. The question boils down to whether the residuals are significant enough and distinct enough from expected detector response behaviors to call the detection into question or suggest something unexpected is contained inside the detected signal.

chmike• 9 years ago

It all boils down to what they subtract. It is said that they subtract the wave signal. Is this signal known for sure or very predictable ? It also depends on how they subtract it. Is it in the fequency domain or the space doman ? I didn't read the arXiv article and don't plan too. Just commenting and asking.

contravariant2• 9 years ago

If any correlation in the residual signals would be enough to doubt the discovery then I don't understand why they didn't simply combined the signals into one by minimizing the residual correlation.

dnautics6• 9 years ago

“The only persons qualified to analyze this paper are in the LIGO Scientific Collaboration”

What? That's an incredibly dangerous position for a scientist of any stripe to take.

arca_vorago• 9 years ago

I've yet to find a good solution to this increasingly more prevelant sentiment in the current highly-specialized niche type of scientific advancement. I always looked up to the autodidacts of history and wanted to emulate them, but all I hear is "that's impossible these days".

jessriedel3• 9 years ago

It's an empirical claim, not a normative one.

dnautics6• 9 years ago

it's a terrible claim. You can't claim that for all of the subdisciplines in the massive LIGO project, every single field is covered by the 'topmost expert'. Especially for a criticism about signal processing - there are so many adjacent or even distant scientific fields that are likely to have individuals with superior 'expertise', in the same class of algorithms used for the signal processing.

The claim is that only the people working in the project are qualified to criticize; that's almost the opposite. People who are working on the project are more often than not the LEAST qualified to criticize, because they're heads down in the weeds and have too much invested in the particular details and experimental choices that were made.

Moreover, for LIGO specifically, not every scientist is motivated to work for a high-risk, long-time-till-result endeavor. In fact, I'd suspect the people most likely to approach things with the right critical mindset are antiselected from working on the project.

kurthr2• 9 years ago

I think the end of the sentence is interesting as well... ", said Robert Wagoner, a theoretical physicist at Stanford University who is not affiliated with LIGO. “They are the only ones who have had access to the raw data.”

That kind of gives the lie to the conspiracy that many would like to believe... And before you go demanding that they release the raw data (in addition to the processed data), I hope you are aware of how many disk drives you're going to need to fit into that station wagon.

dnautics6• 9 years ago

I mean yeah there's probably lots of instrument telemetry in the data, but let's give LIGO the benefit of the doubt and say that the instrumentation has consistent performance through null control regions of the timeseries and the "interesting" regions of the timeseries (bloop #1, in this case, but apparently now bloops #2, and #3 which were not analyzed by the person doing the criticism).

the 'truckloads' of data are basically all the uninteresting parts of the timeseries. Of course there is statistically going to be random events where the noises correlate at the level seen by the critique, and maybe the truckloads of data show that those are frequent enough that say 1/3 of the bloops should have that correlated noise superimposed on the 'actual signal'; and maybe that bloops 2 and 3 do not have the correlated noise if you apply the same analysis to it.

How much more 'raw data' do you need to reach a conclusions that refutes the criticism?

maxander2• 9 years ago

Do they propose a mechanism for how there could be simultaneous correlations between the two detectors that aren't gravitational waves? There's few enough forces that can act through kilometers of solid earth that I'd be willing to take it as a given that they're uncorrelated, absent a signal... but then, I'm not a physicist.

Even more I'm-not-a-physicist-y speculation; could the additional correlation be some secondary gravitational wave effect hidden under the main signal, pointing to another aspect of the phenomenon?

fleitz• 9 years ago

If gravity is waves, it stands to reason that there would be lots of noise that correlated as various events in the universe reach earth.

posterboy• 9 years ago

gravity isn't waves, I think that was an overly simplistic description, but the explanation for noise does stand to reason, imho.

spuz3• 9 years ago

I think the argument is that any correlation in the noise must be simply coincidental. In other words, the detection could be explained by chance rather than some true physical phenomenon measured by both detectors.

But, LIGO ran their detectors for 8 years at a lower sensitivity with no detections. Almost as soon as they turned on the upgraded detectors in September 2015, they detected the first event (GW150914) which appeared at a level that would have been undetectable previously. Also, they have made two other detections in the 1.5 years since. My guess is that if these events were due to flawed analysis, then we should have seen something during those first 8 years.

mannykannot• 9 years ago

While I think it is unlikely that there is a problem here, one has to consider that upgrading the sensitivity could introduce or reveal a systematic error.

Also, it is not just a correlation that has been observed - it is a correlation that has a specific form, a rising fundamental frequency that happens to in line with what we would expect from merging black holes. I know there's an element of circularity in the argument, but what are the chances of that particular sort of signal arising from a spurious correlation?

smortaz• 9 years ago

If you want to look at the data and run their code w/o installing any software, check out their ready to view/run Jupyter notebooks:

https://notebooks.azure.com/losc/libraries/tutorials

1. click on a notebook to see the html rendering 2. or clone+run to execute/edit the code (need to create login)

sleavey• 9 years ago

Let's wait and see if their article gets through peer review in a similarly respected journal to the three published detection papers'.

raverbashing2• 9 years ago

Wait

Scientists keep saying and repeating how correlation doesn't mean anything to the point it's become a joke

Now they are dabbling about a spurious correlation in a small section of the data? For real?

No, really, if anyone would be trying to prove anything based on that correlation alone they would have been laughed and dismissed by the majority of the scientific community

But suddenly when it's to reject a result then it's important?

points out that Jackson’s team could have misused a common data-processing technique called the Fourier transform... The error, Harry writes, has to do with the technical assumption that the input data signal be “cyclical,”

Correct. Not to mention the sample is too short (and the even is short as well and there's noise and the sampling rate is not too big).

If you play with FFT on these conditions you start seeing things that "aren't there"

phreeza3• 9 years ago

Scientists don't say correlation doesn't mean anything. What they do say is that correlation doesn't imply causation.

If the basic model underlying the original LIGO finding would predict no correlation between the recordings, then a significant amount of correlation should lead scientists to revise the model, and reexamine the findings. Very much standard scientific practice.

raverbashing2• 9 years ago

There are several possible sources of correlation on the "impulse response" of the LIGO detectors (the whole detector, from building construction to the electronics and detectors in itself) so it is not surprising that might happen. Or maybe it's something in the source of the signal

But the signal is still stronger than the noise

Besides the "poking the data in weird ways until it stops meaning anything" issue

snarfy• 9 years ago

Is it really noise or just more signal?

Cosmic background radiation is noise but it can be mapped out into a pattern. Could the correlation be a view of a type of cosmic background gravitational waves?

dboreham• 9 years ago

The pattern is in a different dimension than the noise (time vs spatial). A pattern by definition isn't noise.

kobeya• 9 years ago

Not by definition. There is such a thing as structured noise.

mirimir• 9 years ago

Well, except:

The main claim of Jackson’s team is that there appears to be correlated noise in the detectors at the time of the gravitational-wave signal. This might mean that, at worst, the gravitational-wave signal might not have been a true signal at all, but just louder noise.

If that's an accurate characterization, "louder noise" does seem a lot like signal.

Couldn't the "correlated noise" just be a superposition of numerous small gravitational-wave signals?

shamaku• 9 years ago

where there's smoke...

PhantomGremlin• 9 years ago

Obviously the simplest solution is to have a third detector, located far away from the other two. I'm surprised the article didn't even mention that.

Strange Noise in Gravitational-Wave Data Sparks Debate

Related Stories