Source: New Scientist, Oct 2018
We believe that LIGO has failed to make a convincing case for the detection of any gravitational wave event,” says Andrew Jackson, the group’s spokesperson. According to them, the breakthrough was nothing of the sort: it was all an illusion.
The big news of that first sighting broke on 11 February 2016. In a press conference, senior members of the collaboration announced that their detectors had picked up the signature of gravitational waves emitted as a pair of distant black holes spun into one another.
The misgivings of Jackson’s group, based at the Niels Bohr Institute in Copenhagen, Denmark, began with this press conference. The researchers were surprised at the confident language with which the discovery was proclaimed and decided to inspect things more closely.
Their claims are not vexatious, nor do they come from ill-informed troublemakers. Although the researchers don’t work on gravitational waves, they have expertise in signal analysis, and experience of working with large data sets such as the cosmic microwave background radiation, the afterglow of the big bang that is spread in a fine pattern across the sky. “These guys are credible scientists,” says Duncan Brown at Syracuse University in New York, a gravitational wave expert who recently left the LIGO collaboration.
when Jackson and his team looked at the data from the first detection, their doubts grew. At first, Jackson printed out graphs of the two raw signals and held them to a window, one on top of the other. He thought there was some correlation between the two. He and his team later got hold of the underlying data the LIGO researchers had published and did a calculation. They checked and checked again. But still they found that the residual noise in the Hanford and Livingston detectors had characteristics in common. “We came to a conclusion that was very disturbing,” says Jackson. “They didn’t separate signal from noise.
Jackson is suspicious of LIGO’s noise analysis. One of the problems is that there is no independent check on the collaboration’s results. That wasn’t so with the other standout physics discovery of recent years, the Higgs boson. The particle’s existence was confirmed by analysing multiple, well-controlled particle collisions in two different detectors at CERN near Geneva, Switzerland. Both detector teams kept their results from each other until the analysis was complete.
New Scientist has learned, for instance, that the collaboration decided to publish data plots that were not derived from actual analysis. The paper on the first detection in Physical Review Letters used a data plot that was more “illustrative” than precise, says Cornish. Some of the results presented in that paper were not found using analysis algorithms, but were done “by eye”.
Brown, part of the LIGO collaboration at the time, explains this as an attempt to provide a visual aid. “It was hand-tuned for pedagogical purposes.” He says he regrets that the figure wasn’t labelled to point this out.
This presentation of “hand-tuned” data in a peer-reviewed, scientific report like this is certainly unusual. New Scientist asked the editor who handled the paper, Robert Garisto, whether he was aware that the published data plots weren’t derived directly from LIGO’s data, but were “pedagogical” and done “by eye”, and whether the journal generally accepts illustrative figures. Garisto declined to comment.
There were also questionable shortcuts in the data LIGO released for public use. The collaboration approximated the subtraction of the Livingston signal from the Hanford one, leaving correlations in the data – the very correlations Jackson noticed. There is now a note on the data release web page stating that the publicly available waveform “was not tuned to precisely remove the signal”.
Whatever the shortcomings of the reporting and data release, Cornish insists that the actual analysis was done with processing tools that took years to develop and significant computing power to implement – and it worked perfectly.
However, anyone outside the collaboration has to take his word for that. “It’s problematic: there’s not enough data to do the analysis independently,” says Jackson. “It looks like they’re being open, without being open at all.”
Brown agrees there is a problem. “LIGO has taken great strides, and are moving towards open data and reproducible science,” he says. “But I don’t think they’re quite there yet.”
The Danish group’s independent checks, published in three peer-reviewed papers, found there was little evidence for the presence of gravitational waves in the September 2015 signal. On a scale from certain at 1 to definitely not there at 0, Jackson says the analysis puts the probability of the first detection being from an event involving black holes with the properties claimed by LIGO at 0.000004. That is roughly the same as the odds that your eventual cause of death will be a comet or asteroid strike – or, as Jackson puts it,”consistent with zero”. The probability of the signal being due to a merger of any sort of black holes is not huge either. Jackson and his colleagues calculate it as 0.008.