After the historic announcement in February 2016 hailing the discovery of gravitational waves, it didn’t take long for skeptics to emerge.
The detection of these feeble undulations in the fabric of space and time by the Laser Interferometer Gravitational-Wave Observatory (LIGO) was said to have opened a new ear on the cosmos. But the following year, a group of physicists at the Niels Bohr Institute in Copenhagen published a paper casting doubt on LIGO’s analysis. They focused their criticism on the experiment’s famous first signal, a squiggly line—representing the collision of giant black holes more than a billion light-years away—that was printed in newspapers worldwide and tattooed on bodies.
Quanta Magazine
About
Original story reprinted with permission from Quanta Magazine, an editorially independent publication of the Simons Foundation whose mission is to enhance public understanding of science by covering research developments and trends in mathematics and the physical and life sciences.
Even as LIGO sensed more gravitational-wave signals and its founders received Nobel Prizes, the Copenhagen researchers, led by professor emeritus Andrew Jackson, claimed to have found unexplained correlations in the “noise” picked up by LIGO’s twin detectors. The detectors — L-shaped instruments whose arms alternately stretch and squeeze when a gravitational wave passes — are located far apart in Livingston, Louisiana, and Hanford, Washington, to ensure that only gravitational ripples from space could wiggle both instruments in just the right way to produce the telltale signal. But according to Jackson and his team, the correlations in the noise data suggested that LIGO might have detected not gravitational waves but some terrestrial disturbance, perhaps an earthquake. They claimed that, at the very least, something was not right with the instruments or with the LIGO scientists’ analysis.
The findings were worrisome. LIGO scientists checked their work again, and a party of experts visited the Niels Bohr Institute last year to dig into the details of Jackson and colleagues’ algorithms. Two groups of researchers set out to independently analyze LIGO’s data and the Copenhagen group’s code.
Now both groups have completed their studies. The new papers explain different aspects of the problem that led Jackson and his coauthors to make their claim. Both analyses definitively conclude that the claim is wrong: There are no unexplained correlations in LIGO’s noise.
“We see no justification for lingering doubts about the discovery of gravitational waves,” the authors of one of the papers, the physicists Martin Green and John Moffat of the Perimeter Institute for Theoretical Physics, said in an email.
The pair has no direct ties to LIGO. “It’s important for science for people to do analysis of data and results independently of the group,” Moffat said, “especially for such a historic event in the history of physics.”
Frans Pretorius, a gravitational-wave expert at Princeton University who was not involved in any of the recent studies, said that for more than a year, he and most of the physics community have been satisfied that LIGO’s analysis, and its discovery, are sound. Nevertheless, he said, “it’s important that finally there is a thorough analysis in the form of a paper,” rather than “media back and forth.”
The spokesperson of the 1,200-person LIGO Scientific Collaboration, David Shoemaker of the Massachusetts Institute of Technology, said by email that the new findings corroborate internal discussions among the team. “Seeing those two non-Collaboration re-analyses does reaffirm my certainty that the detections [of gravitational waves] are genuine,” Shoemaker said, “and also is a reinforcement of our earlier perception of where the Jackson et al. paper has problems.”
In an email, Jackson called Green and Moffat’s paper, which was published in Physics Letters B in September, “absolute rubbish.” When asked to elaborate, he appeared to wrongly characterize their argument and didn’t address the most important issues they raised about his team’s work. Jackson also dismissed the second set of findings by Alex Nielsen of the Max Planck Institute for Gravitational Physics in Hannover, Germany, and three coauthors, whose paper appeared on the physics preprint site arxiv.org in November and is under review by the Journal of Cosmology and Astroparticle Physics. “We are in the process of writing a response to this latest paper,” Jackson wrote, so “I will not explain where they (once again) made their mistakes.”
“The Copenhagen group refuse to accept that they may be wrong,” Moffat said. “In fact, they are wrong.”
Experts say the problem came down to a combination of blunders: several by the Copenhagen physicists, and one by LIGO.
To help tease out the puny wiggle of a passing gravitational wave from a noisy background, LIGO’s algorithms constantly compare the lengths of the twin detectors’ arms, which oscillate when agitated by a passing gravitational wave or background noise, to “template waveforms” — possible gravitational-wave signals calculated from Einstein’s general theory of relativity. When there’s a close match between a signal detected in Hanford and one sensed shortly before or after in Livingston that also fits a template waveform, email alerts fly around the world.
The scientists then carefully determine the “best-fit” gravitational waveform that most closely matches the signal in the two detectors. When this waveform is subtracted from each of the signals, this leaves behind “noise residuals” — the remaining little wiggles in the detectors that should be uncorrelated, since the instruments are about 2,000 miles apart.
In their 2017 paper, the Copenhagen group claimed to have discovered that the noise in Livingston matched the noise in Hanford seven milliseconds later, just as the putative gravitational-wave signal arrived at both detectors. They interpreted this to mean that LIGO either hadn’t cleanly separated their signal from the noise, or correlations in the noise at exactly the right moment were responsible for the entire signal.
However, Green and Moffat identified a series of errors in the Copenhagen team’s data-handling that they say conspired to create a correlation that wasn’t really there.
To look for correlations in the residuals, Jackson and his colleagues picked a 20-millisecond segment of Livingston data and slid 20-millisecond segments of Hanford data across it, registering correlations whenever peaks overlapped with peaks and troughs with troughs. They found that strong correlations happened when the data was offset by seven milliseconds. But Green and Moffat noticed that when they took Jackson and colleagues’ code and reversed the procedure, fixing the Hanford noise data and sliding Livingston data segments across it, the correlation at seven-milliseconds offset went away. “This was a big red flag because it says, OK, you don’t have a calculational method that’s robust,” said Green, an expert in digital signal processing. Rather, the lengths of the data segments and their asymmetric treatment were “tuned to obtain a correlation signal at just about any desired time offset,” he said.
In a separate calculation, Jackson and his team seemed to find non-random, correlated patterns of peaks and troughs throughout the noise records in the two detectors. But Green and Moffat inferred that the Copenhagen physicists had not “windowed” the two sets of noise data. Windowing is a standard technique of smoothly dialing a signal to zero at the beginning and end of a segment of data before doing a mathematical operation called a “Fourier transform” that facilitates comparisons to other data. The Fourier transform treats a data segment as if it is cyclical, looping together the beginning and end. If the segment isn’t windowed, abrupt changes at the endpoints called “border distortions” can wind up looking like correlations when the data is compared with a second data set.
When Green and Moffat windowed the two sets of noise data, the claimed correlations went away. “Our concern is that the calculation that was done by the Copenhagen group was contrived to get the result they wanted to get,” Green said.
Nielsen and his coauthors — Alexander Nitz, Collin Capano and Duncan Brown — also concluded that the claimed correlation in the noise isn’t real, but they say the error can be attributed at least in part to LIGO’s mistake in providing the wrong data in the first figure of their 2016 discovery paper in Physical Review Letters.
Figure 1 is “the thing people have tattooed on their arms,” said Brown, a gravitational-wave astronomer at Syracuse University and a former LIGO member, who left the collaboration this year to pursue independent analyses of the data.
The figure’s top panel shows side-by-side squiggly lines representing the gravitational-wave signal detected in Livingston and Hanford. Below that are template waveforms closely matching the signals and, in the bottom panel, jagged lines representing the “noise residuals” in the two detectors, after the template waveform has been subtracted from each data set.
Brown explained that Jackson’s code, which he examined in detail during a visit to Copenhagen last year, detects an overlap in the residuals at seven milliseconds offset for a mundane reason: The template waveform shown in Figure 1 is not the “best-fit” waveform that LIGO actually used in its rigorous analysis. The figure was created for illustrative purposes, Brown and others explained. The figure-maker had matched a template waveform to the twin signals by eye, rather than using the best-fit signal as determined by careful calculations. Small imperfections in the subtracted waveform meant that there was some gravitational-wave signal left in both data sets that didn’t get subtracted off, and which ended up mixed in with the noise shown at the bottom of Figure 1—producing correlations that could be teased out by Jackson and colleagues’ algorithms. “What they discovered was an imperfect subtraction” of the signal waveform, Brown said. “When we subtract a better waveform than the one used in the PRL paper, we find no statistically significant residuals.”
“If LIGO did anything wrong,” he added, “it was not making it crystal-clear that pieces of that figure were illustrative and the detection claim is not based on that plot.” Jackson, however, accused LIGO scientists in an email of “misconduct” and making “the conscious decision not to inform the reader that they were violating one of the central canons of good scientific practice.”
Which is to blame, LIGO’s sloppy figure or the Copenhagen group’s faulty calculations? “In reality, I think it’s both,” Brown said. If Jackson and his colleagues were able to tune their parameters to create correlations at seven milliseconds offset, as Green and Moffat’s findings suggest, this would have essentially biased their calculations. Then, at the same offset, their biased algorithm picked out the imperfectly subtracted bits of signal in the noise, reinforcing the false impression.
Jackson, however, maintains that the unexplained correlations are present and says he and his colleagues are preparing a rebuttal to the recent work. He still thinks LIGO’s first, most powerful gravitational-wave signal (and all others by extension) might have been something else altogether — perhaps, he said, “a lightning strike in Burkina Faso, seismic, or even one of the mysterious ‘glitches’ that LIGO detectors see about once an hour.”
But both new papers reviewed and reanalyzed LIGO’s raw data and rediscovered the gravitational-wave signals within it, using different algorithms than LIGO’s. Other researchers have done the same.
“I think the pursuit of independent analyses of gravitational-wave data is a very important and valuable thing to do, and we are delighted that more people are getting involved,” said Shoemaker, LIGO’s spokesperson. “That the Jackson et al. work has stimulated some additional independent investigations can be seen as a positive outcome, but I personally think it comes with a fully unnecessary cost of ‘drama.’”
Meanwhile, LIGO’s twin detectors, along with a third instrument in Europe called Virgo that switched on in 2017, have recorded 10 black hole collisions to date and one space-time wiggle from colliding neutron stars. Scientists announced the four latest black hole detections this month and released dazzling graphics showing the universe’s growing population of these mysterious, invisible, super-dense spheres. When the neutron-star collision was detected last year, 70 telescopes swiveled toward the fireworks; their observations indicated the cosmic origin of gold, the expansion rate of the universe and more.
Brown said it isn’t surprising that LIGO’s revolutionary discovery invited skepticism. A powerful event was detected “basically the day we turned it on,” he said, and the rate of black hole collisions in the cosmos has turned out to be at the high end of expectations.
“The universe loves gravitational-wave astronomers,” he said.
Original story reprinted with permission from Quanta Magazine, an editorially independent publication of the Simons Foundation whose mission is to enhance public understanding of science by covering research developments and trends in mathematics and the physical and life sciences.