News for hackers. Stuff that matters.
Re:Decoding James Webb Space Telescope by colechristensen on Monday December 27, 2021 @01:21PM (#29705105)
This is just telemetry data which doesn’t have much general or scientific interest, i guess they could publish the protocol spec (honestly it probably is aquirable) but most of the fun for the kinds of people who want this data is going to be doing this reverse engineering themselves.

The real imaging data would require a much more significant dish to even receive (i can’t immediately find what it’s going to use, but I’m guessing something like a 40 meter dish) so there are approximately zero amateurs who could use such open source information.

Re:Decoding James Webb Space Telescope by Twisol on Monday December 27, 2021 @04:33PM (#29707018)
Most of the public specifications are distributed freely by the CCSDS (Consultative Committee for Space Data Systems): https://public.ccsds.org/Publications/BlueBooks.aspx

The mission-specific parameters ("managed parameters") used by any given mission are usually more tightly controlled, as are the payload specifications for each telemetry channel.

> This is just telemetry data which doesn’t have much general or scientific interest

My understanding is that "telemetry" and "telecommand" stand for the downlink and uplink directions of a space link. I mostly worked upstream of telecommand, but I understood "telemetry" to refer to received data of any kind -- e.g. in CCSDS 130.1-G-3, an informational report on the design of the CCSDS telemetry system. https://public.ccsds.org/Pubs/130x1g3.pdf

By the by, I've been continually impressed with the quality of the CCSDS' documents. The "green books" (informational reports, like the one above) are extremely approachable and well-written.

Re:Decoding James Webb Space Telescope by astroflask on Monday December 27, 2021 @05:03PM (#29707364)
Gonna chime in here to comment that most NASA missions (and ESA too) provide the scientific data for download free of charge, under Public Domain or CC licenses. If it's for scientific purposes, it's not just good manners, but rather a requirement to cite the proper dataset (that also gives you the bonus of citing a respected source, so it's a win-win). Thing is that many people doesn't even know where to look for!

And it doesn't help that some missions manage their own archives differently, and there's a lot of terminology to learn on your own. One of the complete opposites of that, which was a joy, was the New Horizons archive which, at one point, you could download from a torrent! For example, if you wanted to see V3 of the Arrokoth encounter from 2019, you'd go to: https://pdssbn.astro.umd.edu/holdings/nh-a-lorri-3-kem1-v3.0...

Again, New Horizons is a bit of a rare case in which they went for super accessible data for everyone. PDS itself is a great system, but many missions will just upload a bit of data to PDS and then manage the rest some other way (Cassini for example has only a couple of instruments on PDS, and you have to go to some other URL if you want uncalibrated but automatically processed images on JPEG format[0], but yet another place (to which I've lost the link to and I can't find on mobile) for the full, science-grade dataset).

A great resource is OPUS[1] too, however I find it's UI a bit difficult, and in the end I prefer to download full datasets and just explore them on my own rather than going with those online browsers. For example, if you wanted to check the Voyager images of Neptune, you'd go to this massive URL[2]. Quick tip: once you've configured the filter you want to apply, the Search button is on the top left -- this is the kind of usability thing I mentioned, buttons and links aren't quite where you'd expect them. Oh and there's a limit to how many things you can select for download at once. And it's all dynamically loaded, and on and on and on. Which is why, as I said before, I generally prefer to just download the full GB sized dataset and explore it on my own.

[0] https://solarsystem.nasa.gov/raw-images/raw-image-viewer/?or...

[1] https://opus.pds-rings.seti.org/opus/

[2] https://opus.pds-rings.seti.org/opus/#/instrument=Voyager+IS...

Re:Decoding James Webb Space Telescope by Twisol on Monday December 27, 2021 @06:11PM (#29708100)
> Gonna chime in here to comment that most NASA missions (and ESA too) provide the scientific data for download free of charge

I thought that was the case, but it's been so long since I've been on a mission proper (Cassini, student co-op) that I didn't want to say so without basis. Thanks!

Re:Decoding James Webb Space Telescope by astroflask on Tuesday December 28, 2021 @10:21AM (#29714563)
Yes, I just explore these archives as a hobby and sometimes do a bit of amateur processing on the files. There tends to be some embargo period (a few months to a couple of years) where only the mission team has access to the data, and they decide how/to whom share it. But then again, they are all scientists and willing to share knowledge 99.9% of the time -- so a fellow researcher will probably be able to get a copy of the data if they are polite and ask for it accordingly.

As I'm just "playing" with the files, I don't mind waiting a few months/years to get access to full "scientific grade" readings from incredible complex machines and systems. And if the "raw" data is not easily available, they also usually do provide processed images as part of the missions public outreach campaigns (usually the ones that are found on Wikipedia).

Re:Decoding James Webb Space Telescope by aunty_helen on Monday December 27, 2021 @02:10PM (#29705695)
From: https://ntrs.nasa.gov/api/citations/20080030196/downloads/20...

>To keep up with the high downlink, the recorder data gets sent directly to the Ka-band transmitter

Currently aws groundstation doesn't support KA band so no luck there. It's apparently going to do a transmission once a day so you would need to time it right with the ground station.

Re:Decoding James Webb Space Telescope by Thorentis on Monday December 27, 2021 @04:09PM (#29706802)
Groundststion as a Service. I had no idea this existed. I am continuously amazed at how many things Amazon churns out "as a service".
Re:Decoding James Webb Space Telescope by aunty_helen on Monday December 27, 2021 @11:05PM (#29710232)
It's pretty amazing. I found out about it a couple months ago from a job posting that wanted a programmer for a cube sat. It seems like this is what they're aiming the service for at the moment.

And really, it's the perfect place for aws to enter, add glue, and provide a service to let you do your core business. Feels like the future man.

Re:Decoding James Webb Space Telescope by hayanno on Monday December 27, 2021 @04:12PM (#29706828)
Nice link, very informative, here's a funny excerpt : "It [JSWT] is currently planned to be launched in 2013 from French Guiana aboard an Ariane 5 launch vehicle".
Re:Decoding James Webb Space Telescope by aunty_helen on Monday December 27, 2021 @11:07PM (#29710240)
I couldn't find a publish date in the pdf but going off the url I'm going to guess that it was from 2008.

It's a little bit annoying finding information about jwst as info could potentially be old and out of date. Another document that I read mentioned using XML as the database format because XML was an emerging standard :S

Re:Decoding James Webb Space Telescope by londons_explore on Monday December 27, 2021 @01:35PM (#29705263)
But in a really open project, the design of the whole lot would be on the web, and the data sent back would be sitting on an FTP server somewhere for anyone to download and use.

In many ways, an open project is cheaper to do than a behind-closed-doors project where every new contractor needs to get access to only the bits of the project they need access to, and misunderstandings happen because not everyone has enough of the big picture.

The only bit that needs to be secret is one private key used to sign the commands sent to the satellite, just so one random Mallory can't 'steal' it.

Re:Decoding James Webb Space Telescope by semi-extrinsic on Monday December 27, 2021 @01:47PM (#29705420)
> the data sent back would be sitting on an FTP server somewhere for anyone to download and use

I'm sure they could actually do that without too much fuss. But it would require significant amounts of scientist time to document those datasets to enable others to use them for any arbitrary dataset. I'm sure we'll see fully open data sets from JWST appear, but lots of the stuff it collects isn't going to be interesting enough that it's reasonable to spend scientist time documenting it.

Re:Decoding James Webb Space Telescope by jandrese on Monday December 27, 2021 @02:40PM (#29705990)
It seems like it should all be automated. Some scientist generates a mission request for the JWST techs. If accepted the mission is added to the timeline with all of the metadata the original scientists had in their proposal. Stuff like the area being imaged, the sensors in use, duration of capture, etc...

Once the data is collected and downloaded it is added to the catalog with all of that metadata attached. Then it's a matter of opening up that catalog to the public, although I'm guessing the downloads will be quite sizeable so the bandwidth could be an issue.

The trick to making this work is to integrate the publishing into the workflow so it doesn't require any additional effort on the part of anyone.

Re:Decoding James Webb Space Telescope by xioxox on Monday December 27, 2021 @03:04PM (#29706192)
The data from the mission will be made public after proprietary periods [1]. They have an archive [2]. I don't expect that the raw telemetry will be made available, but the raw science data in FITS format appears will be available.

[1] https://jwst-docs.stsci.edu/jwst-opportunities-and-policies/... [2] https://archive.stsci.edu/missions-and-data/jwst

Re:Decoding James Webb Space Telescope by londons_explore on Monday December 27, 2021 @02:07PM (#29705655)
I think there needs to be a distinction between something that a project "publishes", and something that is "made available".

Something published has been checked by a few team members, written with care, and represents the opinion of the authors and project.

Something made available has no guarantees of correctness, might not represent the projects opinion, and might just be random matlab scripts made by a JWST scientist in their lunchtime that they thought was fun.

In the open source world, what is 'published' is probably the projects homepage, and code. What is 'made available' is random chatter on their discord or IRC channel.

I hope that more government projects 'make available' everything done by all the workers - every file saved on every PC, with the understanding that there is no guarantee of correctness.

I guess it's the same idea as being able to see into the kitchen from a restaurant. You might see the chef making mistakes or juggling the saucepans, but you'll also see the work being done as it's done, and being able to view doesn't delay the chefs work.

Re:Decoding James Webb Space Telescope by dividedbyzero on Monday December 27, 2021 @04:50PM (#29707215)
That's a pretty bad idea IMO. Putting people in a panopticon has a strong chilling effect, no matter what disclaimer you put on the recordings. Creative, deep work needs space to make blunders in private, scientists are no exception. They'll just use their personal laptops or document every experiment and mistake and script to death, getting done a lot less actual research.

Plus, it will be pretty much useless im practice since you'd have to be an expert in that niche yourself to know what's correct (you're not getting any extra docs or context) and probably most of it will be some kind of incorrect, possibly very subtly. The only people who could profit tremendously are the competition who aim to snipe that particular paper. Science is pretty dirty and ruthless often as not, I totally could see this happen.

Re:Decoding James Webb Space Telescope by michaelwilson on Monday December 27, 2021 @06:28PM (#29708256)
As I recall, with the Hubble, and presumably the JWST, there were scientific "precedence" rules which limited access to the observational data to the scientist requesting the observation for some period of time.

This was to prevent others from also getting the data and publishing before the requesting scientist.

Assuming this continues to be true, they probably wouldn't want to handle the data by publishing it to a FTP server immediately on receipt.

Re:Decoding James Webb Space Telescope by aragilar on Monday December 27, 2021 @10:35PM (#29710038)
The data will be at https://archive.stsci.edu/ (which holds the data from a whole host of space telescopes, and uses standard VO interfaces).

STScI is a major contributor to astropy etc. (https://github.com/astropy), and has their own space with more tools/software: https://github.com/spacetelescope.

It's unclear what else you'd actually want, unless you want to build a clone of the actual satellite (it wouldn't surprise me that a majority of the software for the satellite is open source, just not put together somewhere publicly for people to download).

Re:Decoding James Webb Space Telescope by colechristensen on Monday December 27, 2021 @10:46PM (#29710117)
> It's unclear what else you'd actually want, unless you want to build a clone of the actual satellite (it wouldn't surprise me that a majority of the software for the satellite is open source, just not put together somewhere publicly for people to download).

Much satellite technology is classified or at least export controlled (ITAR). The major difference between a space telescope and a spy satellite is which direction you point.

Re:Decoding James Webb Space Telescope by MAGZine on Tuesday December 28, 2021 @12:54AM (#29710916)
A funny quip but the main imaging cameras on the jwst would be useless pointed at earth.
Re:Decoding James Webb Space Telescope by coldpie on Monday December 27, 2021 @01:57PM (#29705545)
You're assuming it's secret, but the more likely case is it just isn't anyone's job to make it public.
Re:Decoding James Webb Space Telescope by dylan604 on Monday December 27, 2021 @01:33PM (#29705234)
Aren't they using the Deep Space Network for this?
Re:Decoding James Webb Space Telescope by jacquesm on Monday December 27, 2021 @03:08PM (#29706227)
Yes, all communications are routed through DSN.