Close

Hack Chat Transcript, Page 2

A event log for Software for Satellites Hack Chat

It's rocket AND computer science

dan-maloneyDan Maloney 07/12/2023 at 20:410 Comments

Dan Maloney12:52 PM
Transcript coming up at the end, btw -- no need to copy and paste links

Jacob Killelea12:52 PM
@Dan Maloney thanks!

Jacob Killelea12:53 PM
I suppose I should hit a couple other things that I mean to mention

Jacob Killelea12:53 PM
Sensors: Your core spacecraft might not need sensors if it's Sputnik, but anything else will want to know where it's pointed and where it's heading!

stu12:53 PM
Jacob, where are you looking to go next? Amazon's project Kuiper is hiring vigorously.

Jacob Killelea12:54 PM
Sensors range from simple gyros to full star trackers that automatically take images of the star field nearby and use an onboard map to find out exactly where they're pointing. When you're close to earth, sun sensors, horizon sensors, and magnetometers can all be used.

Jacob Killelea12:55 PM
@stu I'm not sure yet. Still evaluating my options.

Thomas Shaddack12:57 PM
Any lessons from space that can be extra-useful on the ground?

Jacob Killelea12:57 PM
Communications: There's a lot of communication standards in use onboard a spacecraft! At the most basic level, RS-422 and RS-485 are common and that's what I used to talk to most of my payloads! However, MIL-STD-1553 is also popular, although boutique. Spacewire gets used when you want a lot of bandwidth. It's an LVDS serial protocol, although in my experience the signal integrity is a nightmare. Big NASA projects, like the Lunar Gateway, use Time Triggered Ethernet, which I think is the future.

Carl12:58 PM
What about using distributed systems to avoid or mitigate SEUs and other faults?

Jacob Killelea12:58 PM
@Thomas Shaddack I think we hackers don't often appreciate that one day our creations might leave our bench! Things have to be able to boot, communicate, and recover on their own.

Dan Maloney12:58 PM
No CANbus? Seems like it has a lot to offer

Jacob Killelea12:59 PM
@Dan Maloney CAN is definitely used in some systems! I've never had the opportunity myself however. The issue with CAN is the low data rate. It's great for basic sensors, but not when you have lots of data to shuttle

Thomas Shaddack1:00 PM
...that's why I am asking. Down here sending a technician to a device is easier than Up There but still can be both costly and annoying. (And sometimes the extra telemetry can be handy even between bench and couch, when the hacker is extra lazy...)

Jacob Killelea1:00 PM
The NASA SCALPSS payload is going to spend _hours_ dumping data back to the spacecraft over a 921600 baud serial connection once Nova-C lands. Lots of other interesting instruments are similar, and spacecraft are always hurting for the extra bandwidth, storage, and processing power!

Jacob Killelea1:01 PM
That said, it's amazing how much terrestrial computing is wasted. Spacecraft always keep available resources in mind!

Thomas Shaddack1:02 PM
What about two buses? One for critical realtime control, one for just the payload data that can wait a millisecond?

Thomas Shaddack1:02 PM
YES. Compare the computing power needed to get man to the Moon vs how much is needed today to open an email. Something somewhere went wrong.

Jacob Killelea1:03 PM
@Thomas Shaddack yes, absolutely. It's more complicated to have two busses, but it's common to have a control bus where everyone communicates using fixed size messages at a fixed rate for predictability, and a second bus for handling large amounts of data.

Nicolas Tremblay1:03 PM
Yes, but GUI look better now

Dan Maloney1:03 PM
@Jacob Killelea - we're an hour in, but wrap time is totally up to you.

Jacob Killelea1:04 PM
@Dan Maloney I've got a few more things I can throw out there, let me get through a few more subsystems!

Dan Maloney1:04 PM
Absolutely!

Jacob Killelea1:04 PM
Great, ok. Telecom: Your spacecraft needs radios! How else will it phone home! A lot of systems are QPSK modulated and run between 2 and 10 GHz for the high bandwidth communication.

daniel.fryling1:05 PM
Is there always/often a method of performing firmware updates to the satellites remotely? I'd assume this is a critical component?

Thomas Shaddack1:05 PM
I saw cubesats with ISM LoRa downlinks.

Jacob Killelea1:05 PM
The Consultative Consortium for Space Data Systems (CCSDS) creates standards for space data communication. These are fascinating and run in parallel with the terrestrial OSI model

Jacob Killelea1:07 PM
@daniel.fryling Some systems can be reprogrammed, some can't. It's kind of a new development for spacecraft to have that ability. It definitely makes sense for things like core science and guidance applications, but a lot of embedded controllers might even be made with write-once memory to make them more reliable. If programming fails, you've just wasted a very expensive ($1K to $100K) chip

Thomas Shaddack1:08 PM
Better waste a chip than a whole probe...

Jacob Killelea1:09 PM
@Thomas Shaddack for sure! A low bitrate, highly available radio is crucial. Your high gain antenna is highly directional as well, so it's usually complimented by omnidirectional, low bit rate radios

Jacob Killelea1:10 PM
However, this means that your spacecraft needs to be able to gracefully go from high bandwidth to low or no bandwidth and back up again. I've written applications with a telemetry budget of 0.2 bits per second. Get used to packing your bitfields tight and think carefully about exactly what data you need to send to the ground.

Jacob Killelea1:10 PM
A lot of times, a simple flag or a few bits for a counter is enough to show that a system is working, or at least cycling correctly.

Carl1:10 PM
How do you get an initial pointing with the high-gain antenna(s) after landing?

Jacob Killelea1:11 PM
@Carl We have to control our rotation as we touch down on the moon. There's a certain tolerance for this, and it's important for getting the solar panels pointed correctly, making sure heat is distributed properly, and that our instruments will see what we want them to

Jacob Killelea1:12 PM
Other spacecraft will have gimbaled antenna platforms, but that's a moving part that can break.

Carl1:13 PM
Ohh, it is fixed to the structure?

Jacob Killelea1:14 PM
In many cases it is. This can create an issue if you need to point your antenna at earth, your solar panels at the sun, and your instruments at a point of interest.

Jacob Killelea1:14 PM
Computers: Your spacecraft will include at least one computer. It's kinda mandatory these days. This can be as simple as an ARM microcontroller or as complicated as custom silicon implementing a custom architecture. Vorago, AiTech, BAE systems and others all produce space hardare

Jacob Killelea1:14 PM
The lead times for these parts are punishing, the prices are astronomical, and it probably won't have all the support you need!

Jacob Killelea1:15 PM
Software that runs on these systems can range from commercial RTOSes and hypervisors down to arduino and python code.

Jacob Killelea1:16 PM
Even the high end RTOSes like VxWorks are often missing features you might expect, like a disk driver for your special rad-hard SSD. Others are even missing virtual memory, so your processes can overwrite each other's memory!

Jacob Killelea1:16 PM
All this is done in the name of determinism and light-weight computing. You just don't have a lot of resources to go around.

Nicolas Tremblay left  the room.1:17 PM

Jacob Killelea1:17 PM
If you all want to take a look at a nicely made RTOS with a lot of spaceflight heritage, check our RTEMS, it's an open source RTOS originally developed for the US military. It supports a POSIX C environment, as well as things like heterogeneous / distributed compute and real time scheduling.

Jacob Killelea1:19 PM
Payloads: Your payloads are probably some kind of special science or communication instrument made by a vendor. They might be late, or over-budget, or just a pain to integrate with your spacecraft. Things are generally made to order, and they will probably all use a different packet structure for their commands and telemetry. It's therefore a lot of work to make sure they're thoroughly tested and properly integrated.

Jacob Killelea1:20 PM
(You didn't hear it here, but I actually worked with a payload that uses Windows Embedded as its base operating system).

Dan Maloney1:20 PM
Did you have to say that just as I was taking a sip of water?

Jacob Killelea1:21 PM
I hope I didn't cause a spill!

Dan Maloney1:21 PM
Hot and dry today, no worries

Jacob Killelea1:21 PM
Glad to hear!

Jacob Killelea1:22 PM
Let's see, spacecraft guidance is also an important software area

Thomas Shaddack1:22 PM
An area where it is generally unadvisable to mix together imperial and metric units?

Jacob Killelea1:23 PM
@Thomas Shaddack It's kind of inevitable that you'll have to do some conversions, at least here in the US. My school work was mostly metric, but we certainly did imperial as well

Jacob Killelea1:23 PM
Fun fact, the unit of mass in the imperial system is not the pound, but rather the slug

Thomas Shaddack1:24 PM
I was referring to that case of the Mars probe that turned into a meteorite.

Jacob Killelea1:24 PM
Just as 1 kg weighs about 9.81 N on the surface of the earth, 1 slug weighs 32.2 lbf

Jacob Killelea1:25 PM
@Thomas Shaddack Yep, Mars Climate Orbiter I think? It wasn't an issue with the spacecraft, but rather with navigation software running on the ground.

Thomas Shaddack1:25 PM
GROUND??? I missed that!

Jacob Killelea1:26 PM
Yep, IIRC Lockheed wrote a navigation tool in imperial, and missed converting it to metric before sending it up to the spacecraft, or vice-versa

Thomas Shaddack1:26 PM
(facepalm)

Thomas Shaddack1:27 PM
That's in the same league as the Hubble's myopia. How could that pass testing?

Jacob Killelea1:27 PM
This is an important point - your commands up to the spacecraft are just as important as the onboard software! A lot of missions will test all their commands against a flat-sat or simulator before sending them up

Jacob Killelea1:27 PM
A flat-sat is an important tool for testing by the way, it's quite literally all the systems of your spacecraft laid out on a bench in a lb

Jacob Killelea1:27 PM
in a lab*

Thomas Shaddack1:27 PM
Sometimes the last comm with the sat is a bad command...

Jacob Killelea1:28 PM
Absolutely. Even if your commands are CRC'd and are decoded correctly, is it telling the spacecraft to do the right thing?

Jacob Killelea1:29 PM
Hopefully, your flatsat will catch a lot of the issues, but it's very hard to simulate your gyros and star trackers on a bench. We use a hybrid approach where certain sensors were actually simulated in software with a physics engine, and others could be made with real-world hardware

Thomas Shaddack1:30 PM
All the data and signal integrity in the world can't save you from an issue originating between the chair and the keyboard. All it then can do is guarantee proper delivery of an improper command.

Jacob Killelea1:30 PM
This is where your test coverage is important. Test at every level, ideally to 100% test coverage

Jacob Killelea1:32 PM
Yep! That's why the most important part of a space mission is the team of humans! People who know what's going on are invaluable. It's been shown that the success of cubesat missions is directly related to the continuity of the team. Too many students cycling through a university led mission means that a lot of institutional knowledge is lost along the way.

Jacob Killelea1:32 PM
That's part of what amazes me most about the Hackaday community. So many brilliant people who want to stick around and show and teach others cool new things

Carl1:33 PM
Also you flatsat can't run arbitrarily fast and provide far-reaching insight into the future. Or can time be accelerate in it?

Jacob Killelea1:33 PM
@Dan Maloney, I think that's a good closing point. Anything else you think I should address?

Thomas Shaddack1:33 PM
Long-term stability of the team is important also for terrestrial projects.

Jacob Killelea1:33 PM
@Carl We can certainly propagate physics several years into the future using accelerated simulation. However, you lose some fidelity in the process.

Dan Maloney1:34 PM
No, I think you've given us a lot to chew on! Great stuff, I was really looking forward to this one and it didn't disappoint!

Jacob Killelea1:34 PM
Good, glad to hear that! I'm available to answer anyone's follow-up questions as well. Send me an email at jacobkillelea@gmail.com any time!

Dan Maloney1:34 PM
Thanks so much for your time today, I really appreciate it. And thanks to everyone for all the excellent discussion!

Jacob Killelea1:34 PM
Huge thanks for putting this together, Dan!

Discussions