Why are auroras so hard to predict? And when can we expect more?

Social media was flooded with photos of the vivid pinks, greens and blues lighting up the skies from local beaches and backyards all over the country.

Auroras are normally visible near Earth's north and south poles. In Australia, they are typically only seen in Tasmania. However, due to rare and special space weather conditions, this time people could see them as far north as Queensland.

The Australian Space Weather Forecasting Centre first issued a potential extreme (G5, most severe level) geomagnetic storm warning on Saturday morning.

Lucky Australians who received this warning, and those who happened to look outside that evening, were rewarded with an amazing spectacle. However, by sunset on Sunday, the chance of aurora had subsided, leaving many hopeful viewers in the dark.

What happened? Why are auroras so hard to predict, and how reliable are aurora forecasts? To answer this, we need to know a bit more about space weather.

Auroras on Earth are related to the Sun's magnetic field. The Sun's activity increases and decreases over an 11-year period called the solar cycle. We are currently approaching the solar cycle maximum, meaning there's a higher number of sunspots on the Sun's surface.

These sunspot regions have intense magnetic fields, which can lead to huge explosions of electromagnetic radiation called "solar flares", and eruptions of material into space, called "coronal mass ejections".

When this material is directed towards Earth, it collides with Earth's protective magnetic field, kicking off a series of complex interactions between the magnetic field and the plasma in the ionosphere, part of Earth's upper atmosphere.

The charged particles resulting from these interactions then interact with the upper atmosphere, causing beautiful and dynamic auroras. The conditions in space produced by this chain of events are what we call "space weather".

Everyday space weather generally poses no threat, but these events - known as geomagnetic storms - can impact power supply, satellites, communications and GPS, potentially leaving lasting damage.

Saturday's dazzling display was produced by the most intense geomagnetic storm since November 2003. Fortunately, this time there have been no reports of major disruptions to power grids, but SpaceX's Starlink constellation was reportedly impacted.

Last weekend's geomagnetic storm was caused by a large and complex region of sunspots that launched a series of solar flares and a train of coronal mass ejections.

Space weather prediction is challenging, and the physics is complex. Even when we see an eruption on the Sun, it's not clear if or when it will hit Earth, or how strong the effects might be.

Predicted arrival times can be off by up to 12 hours, and it is only when the eruption arrives at monitoring spacecraft close to our planet that can we can gauge the strength of an impending geomagnetic storm.

As a result, aurora hunters really only have up to a few hours advanced notice to decide whether venturing outside is worthwhile or not.

At the time of writing, the sunspot region responsible for the recent display is still spitting out X-class flares and eruptions, but it's no longer facing Earth directly. It is possible this region will still be active when it faces Earth again, but this remains to be seen.

However, as we approach solar cycle maximum, other large complex sunspot regions are likely to form and, if the conditions are right, produce more spectacular aurora displays.

Thousands of Australians lined the beaches looking towards the horizon on Sunday night hoping for a second show, only to be disappointed. Official space weather and aurora forecasts provide a wide range of possibilities that must be communicated with the appropriate nuance.

The most reliable sources of information about space weather and aurora are agencies such as the Bureau of Meteorology's Australian Space Weather Forecasting Centre or the United States NOAA Space Weather Prediction Center.

Not only do they provide aurora forecasts, but they also play a vital role in safeguarding infrastructure from the negative impacts of space weather.

Looking ahead, scientists in Australia and around the world are working hard to improve our understanding and prediction of space weather events.

By studying the Sun's magnetic activity and developing advanced forecasting models of the complex processes that happen between the Sun and Earth, we can better predict and prepare for future space weather events.

A better understanding will help protect important technologies that we rely upon. It will also alert people to step outside and witness a phenomenon that not only lights up the sky, but ignites a profound sense of wonder.