News for hackers. Stuff that matters.
Re:Square Kilometre Array prototype 'scope achieves first light by colordrops on Wednesday January 31, 2024 @02:18AM (#39200988)
Interesting how docker was profiled as important to the project. Certainly there are hundreds or thousands of other just as important technologies involved.
Re:Square Kilometre Array prototype 'scope achieves first light by defrost on Wednesday January 31, 2024 @04:07AM (#39201640)

    And then there's all the back-end compute and storage needed to handle the streams of data all those observatories will produce – a topic The Register will monitor.
https://pawsey.org.au/

https://pawsey.org.au/supercomputing/

https://pawsey.org.au/systems/data-cloud/

https://pawsey.org.au/visualisation/

Re:Square Kilometre Array prototype 'scope achieves first light by a_bonobo on Wednesday January 31, 2024 @04:35AM (#39201822)
Does Pawsey also look after the South African SKA data? Seems a bit far for data transfer
Re:Square Kilometre Array prototype 'scope achieves first light by _djo_ on Wednesday January 31, 2024 @04:38AM (#39201835)
Re:Square Kilometre Array prototype 'scope achieves first light by SideburnsOfDoom on Wednesday January 31, 2024 @05:01AM (#39201963)
> One of the computers will be based in Cape Town.

FYI, it makes sense for this location as it's still (relatively) close the the SKA telescopes, which are of course in a high, dry, remote and sparsely inhabited area. ( https://en.wikipedia.org/wiki/Sutherland,_South_Africa )

And Cape Town is the nearest large coastal population centre where several sub-sea internet cables connect to the land, so it's closer to the rest of the global network in that regard.

Re:Square Kilometre Array prototype 'scope achieves first light by rantingdemon on Wednesday January 31, 2024 @03:16PM (#39208779)
A small correction :). SKA in South Africa is based closer to Carnavron than to Sutherland.

https://www.skao.int/en/arrange-visit

Not that Sutherland is top be ignored, it does host the South African Astronomical Observatory.

PS, it gets damn cold there in the desert, even during summer :).

https://en.wikipedia.org/wiki/South_African_Astronomical_Obs...

Re:Square Kilometre Array prototype 'scope achieves first light by defrost on Wednesday January 31, 2024 @04:48AM (#39201887)
History wise the SKA concept was put forward and pressed forward from Australia in the (contrary to wikipedia) 1980s.

Then it got internationally political with grant money coming in and backing from China, Britain, etc.

The story today is:

    The Square Kilometre Array (SKA) is an intergovernmental international radio telescope project being built in Australia (low-frequency) and South Africa (mid-frequency).

    The combining infrastructure, the Square Kilometre Array Observatory (SKAO), and headquarters, are located at the Jodrell Bank Observatory in the United Kingdom.

    The SKA cores are being built in the southern hemisphere, where the view of the Milky Way galaxy is the best and radio interference at its least.

    Conceived in the 1990s, and further developed and designed by the late-2010s, when completed sometime in the 2020s it will have a total collecting area of approximately one square kilometre. 
https://en.wikipedia.org/wiki/Square_Kilometre_Array

https://www.skao.int/en

https://pawsey.org.au/projects/australian-square-kilometre-a...

Pawsey has been up and running for some time now, well in advance of the actual physical SKA element build - modelling and computing for other Murchison radio quiet zone projects (cosmic microwave background, etc.)

It's now getting up and running and more and more fully international - but over the past two decades the bulk of SKA material online has come from Australia, the CSIRO and Pawsey.

It won't be the only local supercomputing centre anymore, with others in S.Africa and elsewhere, but it was the leader for PoC and illustrates the kinds of compute systems that will lie behind the project.

Re:Square Kilometre Array prototype 'scope achieves first light by _djo_ on Wednesday January 31, 2024 @06:15AM (#39202451)
> History wise the SKA concept was put forward and pressed forward from Australia in the (contrary to wikipedia) 1980s.

It's not that simple. As one of The Australian CSIRO's own researchers points out, the basic concept of the SKA was independently developed by a number of researchers and institutions including the Project Cyclops proposal by Barney Oliver and John Billingham in 1971. [0]

> Then it got internationally political with grant money coming in and backing from China, Britain, etc.

It was international from the start, the result of merging a number of different international ideas and proposals.

> Pawsey has been up and running for some time now, well in advance of the actual physical SKA element build - modelling and computing for other Murchison radio quiet zone projects (cosmic microwave background, etc.)

> It's now getting up and running and more and more fully international - but over the past two decades the bulk of SKA material online has come from Australia, the CSIRO and Pawsey.

> It won't be the only local supercomputing centre anymore, with others in S.Africa and elsewhere, but it was the leader for PoC and illustrates the kinds of compute systems that will lie behind the project.

While Pawsey is an interesting and remarkable centre in its own right, and will be an important element of SKA and other radio astronomy, it is not the only processing centre for SKA and it's too Australia-centric to mention only it each time this question is raised.

For instance, the South African array has also been operating for some time, beginning with the KAT-7 and MeerKAT precursor arrays, and South African companies have been at the forefront of developing new mid-frequency high-bandwidth FPGA-based signal processing solutions like ROACH-1, ROACH-2, and SKARAB both for them and the full SKA-Mid array. [1][2][3]

I have already shared information about the Cape Town Science Processing Centre in another comment, but it's worth noting that the different frequencies used means that the data rate requirements are much higher for the South African centre, at around 20 terabits per second versus the 8 terabits per second needed to get SKA-Low data to Pawsey. [4]

SKA is amazing in many ways, not least because of how international it was from the very beginning

[0] https://arxiv.org/pdf/1212.3497.pdf

[1] https://mg.co.za/article/2015-12-03-ska-gives-high-tech-firm...

[2] https://indico.ict.inaf.it/event/2367/contributions/14597/at...

[3] https://www.sarao.ac.za/wp-content/uploads/2016/12/ska_sa_te...

[4] https://www.skao.int/en/explore/big-data

Re:Square Kilometre Array prototype 'scope achieves first light by defrost on Wednesday January 31, 2024 @06:40AM (#39202622)
Sure, there was a different big antenna idea of a different size and a different end goal sketched out in the early 1970s.

This antenna project, with a square kilometre target and Lo|Mid frequency ranges was sketched out largely by European and Commonwealth astros at Commonwealth Radio conferences - your link does mention the earlier work by Canadians and parallel|later work by the Dutch to craft small dish arrays to survey the sky (as opposed to the Cyclops SETI goals).

The event that really kicked things along (as I distantly recall) from an idea on the back of envelopes to a project with dot points looking for international backers was the 1988 collapse of the 300 Foot Telescope - whether it was due to load, to vibrations, poor materials, etc. it served to launch several new lines of creating larger telescopes - suspended across natural craters, broken into arrays of smaller dishes, etc.

SKA as a specific project had some back history before it "officially" was launched; before the 2011 legal entity, before the 1998 name adoption, before the 1990 anniversary of VLA's.

My comment was motivated largely by looking back in wonder at just how long it takes to get projects of this scale up and running - I've attended pie in the sky proposals for variations on this since 1982 (same year as my first introduction to the (then) current state of gravity wave detection).

> While Pawsey .. it is not the only processing centre for SKA

I can't see that I made that claim, I did link to Pawsey as an example of other systems that lie behind SKA and another comment linked to the parallel systems being launched in S.Africa.

Re:Square Kilometre Array prototype 'scope achieves first light by pmontra on Wednesday January 31, 2024 @05:52AM (#39202291)
According to the top comment now, which was written two hours after yours, they use two Kubernetes clusters. Details at https://news.ycombinator.com/item?id=39201980
Re:Square Kilometre Array prototype 'scope achieves first light by foobarian on Wednesday January 31, 2024 @09:44AM (#39204374)
Unless they don't use Docker images in those k8s clusters that's not too bad for a news article I guess.