Close

Hack Chat Transcript, Part 1

A event log for SatNOGS Update Hack Chat

Do you know where your satellite is?

Dan MaloneyDan Maloney 10/30/2019 at 20:020 Comments

Pierros Papadeas12:00 PM
Pierros here indeed :) Also @fredy and @elkos from the core team

Pierros Papadeas12:01 PM
(apart from @cshields)

OK, great - hard to keep track sometimes. Welcome all! Can we start off with a little introduction on what SatNOGS is for those who don't know?

Pierros Papadeas12:02 PM
Sure!

alan.b.johnston joined  the room.12:02 PM

Pierros Papadeas12:02 PM
SatNOGS is an open source stack of technologies for tracking, communicating and gathering data about satellites

manaschhabra12:03 PM
Nice

manaschhabra12:03 PM
What is the technology behind ?

Pierros Papadeas12:03 PM
That includes Ground station designs, (antennas, rotators etc), Client software, Radio software, and then on the server side many applications to orchestrate this data gathering (network, db, dashboards and others)

RichardCollins12:04 PM
So the intention is data sharing, and the hardware side is the enabler?

manaschhabra12:04 PM
What is the motivation behind ?

Corey Shields12:04 PM
with a goal of improving access to data coming from space (mostly in the ham radio bands)

Pierros Papadeas12:04 PM
@manaschhabra depends on the part of the stack ;) For client running the satellite observations locally it is C/GNuradio and Python, for server side it is mostly Django

Hexastorm12:04 PM
how is the business sustained? like a charity?

manaschhabra12:05 PM
How are users getting benefitted ?

manaschhabra12:05 PM
And isn't it a niche area

Pierros Papadeas12:06 PM
@manaschhabra motivation is to have open data for everything space (check our foundation : Libre Space Foundation) and to promote usage of open source technologies in the space domain

Corey Shields12:06 PM
Let me set a little background/context,

manaschhabra12:06 PM
Sure

Pierros Papadeas12:07 PM
@Hexastorm Non profit foundation behind it (Libre Space Foundation), we do have grants for development and do provide services, also winning prizes (hackaday 2014) helped kickstart this!

Corey Shields12:08 PM
typical satellite operators doing low-cost LEO (cubesats, etc) are often universities or research groups who may see their satellite overhead a couple of times a day. There's a lot of data they may miss out on inbetween. SatNOGS is building a global network of receivers (and the network software) to automate the collection of data for these satellites using SDR devices and raspberry pi, thus increasing the amount of data *collected* from these satellites

RichardCollins12:09 PM
What about from "regular" satellites - images, datasets, etc?

Corey Shields12:09 PM
we have hundreds of ground stations contributing now, and an automated path to go from RF to telemetry dashboards (dashboard.satnogs.org)

ted12:09 PM
Would you say satnogs/libre space is at production state, or "beta", or well before that?

anfractuosity12:09 PM
Is satnogs, rx only?

Pierros Papadeas12:10 PM
@ted I would say in production for most Telemetry oriented usages, and alpha on other.

Pierros Papadeas12:11 PM
@ted for Libre Space projects some are well in production (or proven in space) (check UPSat mission and others in https://libre.space )

Pierros Papadeas12:12 PM
@anfractuosity It's Network side yes. Locally client can TX as well. but we are trying to sort out the legal constrains around TX'ing though the Network

anfractuosity12:12 PM
gotcha cheers

Pierros Papadeas12:13 PM
(It is a deliverable for us for an ESA project to do TX to, so it should be solved by end of 2020 any case ;) )

anfractuosity12:13 PM
ooh nice

Ethan Waldo12:13 PM
Is Satnogs in any way associated to AWS Ground Station?

Corey Shields12:14 PM
consider it the "free tier", open source alternative ;)

fredy12:14 PM
@RichardCollins What do you mean by regular? NOAA, ISS METEOR etc... ?

RichardCollins12:14 PM
yes

anfractuosity12:15 PM
there seems very little information on the hardware of the AWS ground station stuff

anfractuosity12:15 PM
at least that i could find

Corey Shields12:15 PM
AWS ground station is focused on their commercial partners operating in commercial frequencies today.. So, "very similar" for sure. SatNOGS is more community oriented

Corey Shields12:15 PM
they run their own ground stations at their data centers

Ethan Waldo12:15 PM
Sure, so completely separate

Corey Shields12:16 PM
yes, absolutely.. just very similar in use cases

fredy12:16 PM
@RichardCollins yes we track them, check for example some METEOR obs here https://network.satnogs.org/observations/?future=0&norad=40069

fredy12:16 PM

https://network.satnogs.org/observations/?future=0&norad=40069&results=d1

SATNOGS

SatNOGS Network - Observations

Read this on Satnogs

fredy12:17 PM
This for example has a good decoded image on the data tab https://network.satnogs.org/observations/1149072/

RichardCollins12:17 PM
@fredy I review global data networks. NASA put a huge amount of data online that is undocumented and lossy. Useless for scientific purposes, then they happily said, "Oh how good we are." This is more common then you might imagine. The exception is that any data offered "free" is actually immediately useful. High cost of learning and getting access, proprietary formats and software. That sort of thing.

Ethan Waldo12:18 PM
Has there been any thoughts in diverting Satnogs stations part-time for use in long baseline interferometry since there is a unique geo-diversity?

RichardCollins12:19 PM
@Ethan Waldo Excellent question! Thanks!

Corey Shields12:19 PM
There's a somewhat related project that just kicked off recently called the personal space weather project. We've spoken with them many times about sharing code, etc

Corey Shields12:20 PM

https://hamsci.org/basic-project/personal-space-weather-station

HAMSCI

Personal Space Weather Station

By Nathaniel A. Frissell (W2NAF), Philip J. Erickson (W1PJE), Ethan S. Miller (K8GU), William Liles (NQ6Z), Kristina Collins (KD8OXT), David Kazdan (AD8Y), and Nathaniel Vishner (KB1QHX) Photo by Laura Gooch (N8NFE) The Ham Radio Science Citizen Investigation (HamSCI) is an international collective of professional researchers and amateur radio operators working together to simultaneously advance the fields of space science and amateur (ham) radio activities.

Read this on Hamsci

Corey Shields12:20 PM
oh, I may have misunderstood where you were coming from though

Tyler Berezowsky joined  the room.12:20 PM

RichardCollins12:20 PM
The GPS stations are all also collecting ionospheric and atmospheric data. Those networks are growing about as fast as yours. There are hundreds of overlapping efforts.

Pierros Papadeas12:21 PM
@Ethan Waldo for radio astronomy usages (assuming this was your question) we have investigated it, but there are a few key constrains :

Pierros Papadeas12:21 PM
1. Bandwidth limitations for IQ data transfer (to be able to somewhere do the processing combined)

2. Common reference source for all those stations

RichardCollins12:21 PM
They use SDR, but most of the tradiational astronomical and radio astronomy groups are going "global network"

RichardCollins12:22 PM
@Pierros Papadeas So partnering with your local supercomputer network or correlation network is the usual strategy.

Ethan Waldo12:22 PM
Yes, you'd need sufficient GPSDOs and software-based signal combination

Pierros Papadeas12:23 PM
@RichardCollins unless we want to do this in an async kinda one-off way, which might be possible. But looking this from a perspective of always-on always-observing there are those limitations

Pierros Papadeas12:23 PM
@RichardCollins yeap :) always looking for collaboration opportunities

ted12:24 PM
Will there be stronger linkage from an observation to the human-readable content (results)? Is it ultimately just the 5 completed ones listed on 'dashboards'? (which isn't linked from an observation)

RichardCollins12:25 PM
@Pierros Papadeas Because the networks are all fragmentary right now, I have accepted that things are not going to be clean and simple. So I deal with whatever datastreams that come out, and encourage groups to document the datastreams so users know where it comes from, who is doing it, how to get involved, what software is available -- the ususal global collaborative effort stuff.

Corey Shields12:25 PM
@ted: yes.. in fact, we have a draft UI to try and pull the individual sites together better, as a stop-gap to a more inclusive "single app" experience for users

Pierros Papadeas12:26 PM
@RichardCollins then you might be interested in our collaborative project with Harvard-Smithsonian observatory around establishing metadata structures for describing (among others) observations and data from space

Corey Shields12:26 PM
(always looking for python/django/js help!!)

Pierros Papadeas12:26 PM
@RichardCollins check out schema.space

Tyler Berezowsky12:27 PM
1) Is their any efforts in the pipeline to use observations from multiple ground stations for error correction?

2) How is data being managed? Are duplicate observations removed? Are they compressed or is raw IQ data stored? How often is it purged?

Tyler Berezowsky12:28 PM
Also! It's been awesome to follow you guys since you won the prize. Great work :)

RichardCollins12:29 PM
@Pierros Papadeas Yes I was just reading that. But they are not users themselves. And may not be creating new algorithms, visualizations, user groups, links to scientific groups in their universitiies, nor connections globally. That kind of group usually stagnates fairly quickly. Not to be pessimistic, but I seldom see good come of it. I will look at schema.space Thanks for the link.

Pierros Papadeas12:30 PM
@Tyler Berezowsky :

1) There are some R&D projects on signal detection and signal classification (check sdrmaker.space ) that in theory would be benefited from training on large observation datasets. Is that close to what you are referring to?

Pierros Papadeas12:31 PM
(btw SDRMakerspace is a set of activities that Libre Space Foundation is developing, open source, SDR and space related. Funded by the ESA)

Corey Shields12:32 PM
@Tyler Berezowsky 2) right now the client will generate compressed ogg audio of the observation, a waterfall image, and any decoded data (based on the decoder used). All of this gets uploaded to network.satnogs.org (where if we have telemetry decoders, will decode into db and dashboard). There is an option to save raq IQ data locally on the client but we aren't automatically storing that online today (too much data). Right now there's no deduplication of data or observations, in fact we welcome the duplication to make comparisons between ground stations and receiving conditions. Currently no data is purged, though that may change in the future especially where data is known to be "bad" (even then, it is valuable to some). Also, in the near future waterfall info will be sent as data instead of an image, to allow for better processing options

Pierros Papadeas12:32 PM
@RichardCollins couldn't agree more on the stagnation risk unless you have the actual users within your community. That's why we try to be as hands-on as possible with all the missions (including ours ;) )

RichardCollins12:32 PM
@Pierros Papadeas No content at schema.space, no people committed, no drafts, no examples, no goals for science.

Tyler Berezowsky12:33 PM
@Pierros Papadeas and @Corey Shields understood.

Pierros Papadeas12:34 PM
@RichardCollins absolutely true. This started 3 months ago. Regarding commitment this is a 2-year project between the two teams and we will be having actual examples through our own missions soon

RichardCollins12:34 PM
@Tyler Berezowsky I hope by "waterfall infor will be sent as data", you mean the complete FFT (amplitude and phase) that can be used to recreate the time series, but in a conventient frequency form for analysis?

Ethan Waldo12:35 PM
Besides building a station and adding to the network, what are the ways people can get involved?

Corey Shields12:36 PM
@Tyler Berezowsky once data is online, it depends (and some of the spiderweb is to allow modularity, as we allow non-satnogs clients to contribute data as well)

ted12:37 PM
gotta go, but thx as a station operator, nice to hear from ya.

RichardCollins12:38 PM
The Large Hadron Collider is throwing away most of the low energy data because they are too focused on the agenda of a few PI's. The massive amount of low energy data is useful to every university and nuclear related effort on the planet, and they just throw it away. Not archived, not summarized, not shared, not studied and shared. The exact same thing is happening at LIGO. They throw away all the earth-based data because it interferes with looking far away. But if they understood it better, encouraged and strengthened communities to understand it, then they could more easily remove it to see far away. Most large data collection efforts are like that. A shame. A terrible shame.

Corey Shields12:38 PM
@Ethan Waldo there are lots of opportunities for code contributions (python, web, gnuradio, kaitai.io) as well as "citizen science" ideas around the data collected. If you're a grafana fan we have a whole lot of data waiting to be visualized by people who want to contribute crowdsourced dashboards

fredy12:38 PM
@RichardCollins about the new waterfall data, the plan is to transmit timestamp and signal power that will allow us to draw the waterfall but also do some further analysis, like TLE matching and identification.

Corey Shields12:39 PM
@Ethan Waldo see "contribute" on https://wiki.satnogs.org/Main_Page , our forums at https://community.libre.space, or #satnogs on matrix

Ethan Waldo12:39 PM
Is there a Slack or Discord chat Satnoggers hang out on?

fredy12:40 PM
@Ethan Waldo I'm also adding here the repo of all Libre Space projects https://gitlab.com/librespacefoundation and for SatNOGS https://gitlab.com/librespacefoundation/satnogs

Ethan Waldo12:40 PM
+1

Pierros Papadeas12:40 PM
Some stats and status currently of the Network:

Pierros Papadeas12:40 PM

Pierros Papadeas12:41 PM
Those are all the Online and Testing stations of the Network (300+)

Corey Shields12:41 PM
@Ethan Waldo just the matrix channel, currently bridged to freenode but we're hoping to consolidate to matrix. https://wiki.satnogs.org/Get_In_Touch

RichardCollins12:41 PM
@fredy I would like to take full frequency FFT power levels over time to make a global map of the electromagnetic field. No two SDRs have the same gain algorithms. There is no real standard. Not even two of identical make and manufacture. But it is possible to taKe these from every SDR or receiver and combine to map the rise and fally of the electromagnetic background. And for calibrating transmissions and reception pathways.

RichardCollins12:42 PM
@Pierros Papadeas Can you deliberately try to encourage complete global coverage?

Ethan Waldo12:42 PM
Thanks @Corey Shields!

Pierros Papadeas12:43 PM
@RichardCollins we do our best, but some locations are hard (far away from cheap internet and power etc) or culturally less accessible

Corey Shields12:43 PM
or covered by an ocean :D

Pierros Papadeas12:43 PM
(e.g. we are a primarily English speaking project)

fredy12:44 PM

Discussions