Recent years have seen changes to our theories about what counts as a biosignature and which planets might be habitable, and further turnarounds are inevitable. But the best we can really do is interpret the data we have with our current best theory, not with some future idea we haven't had yet.

This is a big issue for those involved in the search for extraterrestrial life. As Scott Gaudi of Nasa's Advisory Council has said: "One thing I am quite sure of, now having spent more than 20 years in this field of exoplanets … expect the unexpected."

But is it really possible to "expect the unexpected"? Plenty of breakthroughs happen by accident, from the discovery of penicillin to the discovery of the cosmic microwave background radiation left over from the Big Bang.

These often reflect a degree of luck on behalf of the researchers involved. When it comes to alien life, is it enough for scientists to assume "we'll know it when we see it"?

Many results seem to tell us that expecting the unexpected is extraordinarily difficult.

"We often miss what we don't expect to see," according to cognitive psychologist Daniel Simons, famous for his work on inattentional blindness.

His experiments have shown how people can miss a gorilla banging its chest in front of their eyes.

Similar experiments also show how blind we are to non-standard playing cards such as a black four of hearts. In the former case, we miss the gorilla if our attention is sufficiently occupied. In the latter, we miss the anomaly because we have strong prior expectations.

There are also plenty of relevant examples in the history of science. Philosophers describe this sort of phenomenon as "theory-ladenness of observation".

What we notice depends, quite heavily sometimes, on our theories, concepts, background beliefs and prior expectations. Even more commonly, what we take to be significant can be biased in this way.

For example, when scientists first found evidence of low amounts of ozone in the atmosphere above Antarctica, they initially dismissed it as bad data.

With no prior theoretical reason to expect a hole, the scientists ruled it out in advance. Thankfully, they were minded to double check, and the discovery was made.

Could a similar thing happen in the search for extraterrestrial life?

Scientists studying planets in other solar systems (exoplanets) are overwhelmed by the abundance of possible observation targets competing for their attention. In the last 10 years scientists have identified more than 3,650 planets - more than one a day. And with missions such as NASA's TESS exoplanet hunter this trend will continue.

Each and every new exoplanet is rich in physical and chemical complexity. It is all too easy to imagine a case where scientists do not double check a target that is flagged as "lacking significance", but whose great significance would be recognised on closer analysis or with a non-standard theoretical approach.

However, we shouldn't exaggerate the theory-ladenness of observation. In the Müller-Lyer illusion, a line ending in arrowheads pointing outwards appears shorter than an equally long line with arrowheads pointing inwards.

Yet even when we know for sure that the two lines are the same length, our perception is unaffected and the illusion remains.

Similarly, a sharp-eyed scientist might notice something in her data that her theory tells her she should not be seeing. And if just one scientist sees something important, pretty soon every scientist in the field will know about it.

History also shows that scientists are able to notice surprising phenomena, even biased scientists who have a pet theory that doesn't fit the phenomena.

The 19th-century physicist David Brewster incorrectly believed that light is made up of particles travelling in a straight line. But this didn't affect his observations of numerous phenomena related to light, such as what's known as birefringence in bodies under stress. Sometimes observation is definitely not theory-laden, at least not in a way that seriously affects scientific discovery.

We need to be open-minded
Certainly, scientists can't proceed by just observing. Scientific observation needs to be directed somehow. But at the same time, if we are to "expect the unexpected", we can't allow theory to heavily influence what we observe, and what counts as significant.

We need to remain open-minded, encouraging exploration of the phenomena in the style of Brewster and similar scholars of the past.

Studying the universe largely unshackled from theory is not only a legitimate scientific endeavour – it's a crucial one.

The tendency to describe exploratory science disparagingly as "fishing expeditions" is likely to harm scientific progress. Under-explored areas need exploring, and we can't know in advance what we will find.

In the search for extraterrestrial life, scientists must be thoroughly open-minded. And this means a certain amount of encouragement for non-mainstream ideas and techniques.

Examples from past science (including very recent ones) show that non-mainstream ideas can sometimes be strongly held back. Space agencies such as NASA must learn from such cases if they truly believe that, in the search for alien life, we should "expect the unexpected". 

The Conversation 

More about: #alienlife