Close

Hack Chat Transcript, Part 1

A event log for Hacking Apollo Hack Chat

Apollo was a hack from the Earth to the Moon

dan-maloneyDan Maloney 04/22/2020 at 20:200 Comments

Hello every, welcome to Hack Chat! Today we've got not one but four hosts - CuriousMarc, Ken Shirriff, Mike Stewart, and Carl Claunch. They've done a lot of cool retro hacks, chief among them getting a real Apollo Guidance Computer working again for the first time in 50 years. Welcome all!

allem.1 joined  the room.12:00 PM

Maybe you can each tell us a little about yourself to kick things off

carlclaunch5112:01 PM
I am retired and able to devote all my time to restoring old mainframes and other vintage technology

Steve joined  the room.12:02 PM

hb9xar joined  the room.12:02 PM

allem.112:02 PM
I'm retired and looking for new projects.

@carlclaunch51 - From what I understand, mainframe skills are a growth industry right now. At least for the short term ;-)

scruzphreak joined  the room.12:02 PM

Mike Stewart12:02 PM
I currently write flight software for satellites Capella Space, and devote pretty much all of my free time to Apollo digital archaeology in various forms

Ken Shirriff12:02 PM
Hi. I'm a retired programmer and I've been researching vintage computing, reverse-engineering old microprocessors, mining Bitcoin by hand, and studying old aerospace computers.

Pablo Z. joined  the room.12:02 PM

curiousmarc12:03 PM
Hello CuriousMarc here. In real life work on fiber optics at Samtec. Former Intel Fellow. In fantasy life restore old electronics with Carl and Ken and Mike in my over-equipped basement lab.

KisImre joined  the room.12:03 PM

carlclaunch5112:04 PM
@Dan Maloney I ajm focused on the hardware side now, while the current short term interest is in software

Ken Shirriff12:04 PM
Marc, Carl, and I have been working together at the Computer History Museum for a while, helping to keep the old IBM 1401 punch-card mainframe running. We also restored a Xerox Alto in Marc's basement. We joined up with Mike to restore the Apollo Guidance Computer.

Ken Berkun joined  the room.12:05 PM

curiousmarc12:05 PM
Actually that's how we all met (save for Mike), working at the Computer History Museum in Mountain View, CA

carlclaunch5112:06 PM
We are all volunteers helping with restoration and digital archeology projects

Ken Shirriff12:06 PM
Stop by the museum (one quarantine is lifted) to see a punch-card machine running.

Aaron S. Jackson joined  the room.12:06 PM

Tomasz Stochmal joined  the room.12:07 PM

That's still a bucket list trip for me. We had Dag Spicer on last year to talk about CHM and preserving our history. CHM is such a valuable resource.

Mike Stewart12:07 PM
And Ken and I were introduced through a mutual friend, because I was about to try to restore an AGC, but didn't yet have any experience working with old hardware at all

Mike Stewart12:07 PM
yeah for sure

curiousmarc12:07 PM
And if you have an old 1960's mainframe in your basement please let us know

carlclaunch5112:07 PM
How about my garage?

Tomasz Stochmal12:08 PM
I enjoyed watching your Apollo restoration videos and Soyuz clock.

Peter Bosch12:08 PM
I've always been a bit envious of the CHM and the people around it, here in the Netherlands there is no such pool of retro repair knowledge and resources

Ken Shirriff12:08 PM
It's amazing how much technology progressed from the IBM 1401 at the museum (1959) to the Apollo Guidance Computer (mid-1960s). The 1401 uses transistors (germanium, not silicon) while the Apollo Guidance Computer uses integrated circuits. They both use magnetic core memory for storage, though.

Alex Heyes joined  the room.12:08 PM

curiousmarc12:08 PM
@carlclaunch51 Yes you have quite a few but you won't let me film ;-)

DrG12:08 PM
From https://blog.samtec.com/post/restoring-agc-part1/ "When powered, the system came to 4.99 +/- 0.01 volts. Think about that: this is 50 plus year old equipment which had been in storage for decades, and it was running within 0.20% of the original specification! "

DrG12:09 PM
That is a m a z i n g !

@curiousmarc , if I had a teletype in my garage I'd want to restore it. That last video gave me a nostalgia attack

carlclaunch5112:09 PM
@curiousmarc yes I have been a bit obstructionist towards filming until my garage looks better.

Jeremy Weatherford12:09 PM
Can you recommend any books for people who can't get enough Apollo-era technology?

Mike Stewart12:09 PM
Sunburst and Luminary by Don Eyles is fantastic

carlclaunch5112:09 PM
Rocket Ranch and related books are excellent for a KSC oriented view

curiousmarc12:10 PM
+1 on Don Eyles book

Peter Bosch12:10 PM
when the series first started i was wondering whether a modern replica of the AGC would be possible, but I couldn't find gates with a similar open collector output and logic function (not in the 74 series at least). are there any chips still in production that could be used one-for-one without significantly altering the design?

Peter Bosch12:10 PM
especially the bit where they form logic by tying outputs of gates together

Ken Shirriff12:10 PM
Eldon Hall's Journey to the Moon: The History of the Apollo Guidance Computer is another good book.

Mike Stewart12:10 PM
I have a plan for that for my replica -- one sec

curiousmarc12:11 PM
Open collector gates? You can still get them in the 74xx series

brianpagephotog12:11 PM
Digital Apollo by David A. Mindell

Mike Stewart12:11 PM
not in the dual three-input NOR variety though

curiousmarc12:11 PM
But the tough bit is the memory tray B

carlclaunch5112:11 PM
not sure about triple input NOR with open collectors

Peter Bosch12:11 PM
been a while since i looked at it but yep think the 3 input NOR was the problem

Just a note, I'll be pulling a transcript at the end and posting it publically, so don't worry about capturing links or titles of books - it'll all be there for you later.

curiousmarc12:11 PM
Dang we have to make a new wafer of it

carlclaunch5112:12 PM
new project!

Mike Stewart12:12 PM

https://www.digikey.com/product-detail/en/texas-instruments/SN74LVC06APWR/296-8437-2-ND/377383

Peter Bosch12:12 PM
i though about spinning small pcbs with real tiny SMD transistors

Ken Shirriff12:12 PM
Peter: you could probably use surface-mount transistors and emulate the integrated circuits directly without increasing the size much. The integrated circuits in the AGC were just 2 transistors + 6 resistors.

Mike Stewart12:12 PM
those were my plan -- a hex open-drain inverter behaves much like a dual open-drain three-input nor gate with outputs tied together in triples

Ken Shirriff12:13 PM
Sorry, 8 resistors in the AGC's IC.

Peter Bosch12:13 PM
thats a good one :) couldn't get the layout small enough for a size accurate PCB module, but might just not have found small enough packages

brianpagephotog12:13 PM
being core memory, was the program still intact when you got the AGC running?

curiousmarc12:13 PM
What are you going to do for memory Mike? Wanna use that big IBM FAA core memory I have at home?

Ken Shirriff12:13 PM
Yes, Mike was able to read the memory from the AGC.

CJ Keithley12:14 PM
quad input 4002b? push-pull output

brianpagephotog12:14 PM
Any idea of what mission it was planned for?

Mike Stewart12:14 PM
my plan was always to attempt to weave my own, with those cores I bought, but it's unclear how far into that I would get before giving up

carlclaunch5112:14 PM
@brianpagephotog we had a dead bit but due to having parity it was possible to rewire and extract the data. One of Marc's videos details the effort

Ken Shirriff12:15 PM
We could tell exactly what the AGC was running and displaying when it was shut off. It was doing an IMU alignment, with coordinates matching the Houston space center. It's in one of Marc's videos.

brianpagephotog12:15 PM
Nice. I haven't seen the videos. Do you have a quick link?

Mike Stewart12:15 PM
our AGC was never intended for a mission -- it was a prototype model, but it was installed in the LTA-8 lunar test article, and underwent the full LM thermal vacuum testing in Houston

curiousmarc12:15 PM
@Mike Stewart That would be awesome. You might just be able to get some woven ones off 1970's boards like DEC.

Peter Bosch12:15 PM
have you guys ever thought about wiring up more parts to it? like sensors or a radiomodem?


https://www.youtube.com/watch?v=2KSahAoOLdU&list=PL-_93BVApb59FWrLZfdlisi_x7-Ut_-w7

YOUTUBE CURIOUSMARC

brianpagephotog12:16 PM
Thanks!

Jeremy Weatherford12:16 PM
The AGC video series was fantastic -- it made for great viewing, and showcased each of your unique talents beautifully.

That's 31 videos, though - not sure which one Ken was talking about. They're all worth watching, of course ;-)

Ken Shirriff12:16 PM
Peter: the AGC is back in Houston now, so we don't have an opportunity to wire up more things to it.

Mike Stewart12:17 PM

https://archive.org/download/S68-21511/S68-21511.jpg

ARCHIVE

carlclaunch5112:17 PM
@Peter Bosch we did hook up some inputs and outputs including simulating various IMU inputs, but we don't have access to the AGC which stopped any future experiments

Mike Stewart12:17 PM
^^ there's a picture of our AGC installed in the Lunar Module, taken in the thermal vacuum chamber in 1968

carlclaunch5112:18 PM
and Ken got a picture of LTA 8 hanging in Johnson Space Center with a gaping hole where the AGC belonged, shot through the open hatch

Ken Shirriff12:18 PM
Dan I'm not sure which video shows the original contents. Probably around number 13. Maybe Marc remembers.

curiousmarc12:18 PM
@Peter Bosch I miss that machine so much. It's back with its owner. But between all of us we'd have a lot to attach to it: the radio telemetry, Carl is working on a real DSKY display, I have a gyro, Mike has the ground control box for uploads...

Peter Bosch12:19 PM
Wow, besides your gyro video, any other videos in the making on those?

carlclaunch5112:19 PM
I also built a telemetry interface so that I could stream out the data that would have been downlinked to mission control

Niklas Beug joined  the room.12:19 PM

curiousmarc12:19 PM
Give me a moment I'll try to dig the video out.

Mike Stewart12:19 PM
we can still take trips out to work with it, for what it's worth -- I've flown out with more rope modules from Don Eyles to dump their contents using it. it's just that flying back and forth to Houston to work with it is a bit prohibitive

carlclaunch5112:20 PM
Plus a frivilous Lego LM model with tiny LEDs that were lit by the AGC as it commanded movement of the spacecraft, while we were running mission simulations

cprossu12:20 PM
This is a very strange question, but do we know exactly how the telemetry/data from the AGC went into the RTCC and then into mission control on their high resolution monitors? Everything I've researched about RTOS comes up blank.

Andy Geppert12:20 PM
Thank you guys for putting your energy and talent into all of this restoration work, especially the AGC. It's incredibly inspiring and fun to follow the project. So much to be [re-]learned!

j lewis joined  the room.12:20 PM

Mark VandeWettering12:20 PM
Lego + AGC for the win! :-) Two great things I never imagined would be together.

Mike Stewart12:21 PM
@cprossu the expert on the RTCC is a guy by the name of Niklas Beug, who is currently the primary maintainer for NASSP -- if you have IRC, we have a freenode channel #nassp, and I'm sure he would be happy to talk about it

Niklas Beug12:21 PM
Who happens to be here

carlclaunch5112:21 PM
We know exactly what data was downlinked, but there doesn't seem to be anything on the RTOS on the 360/75 computers. Probably the screen layouts in mission control are known

Mike Stewart12:21 PM
oh hello

Mike Stewart12:21 PM
lol

carlclaunch5112:21 PM
Hi Niklas

Mike Stewart12:21 PM
hey Niklas!

curiousmarc12:22 PM
This video: . That's the first thing we did when we got it fully running for the first time before rewiring the memory.

cprossu12:22 PM
That is awesome @Mike Stewart

Niklas Beug12:22 PM
Hey guys. But I don't really know much about telemetry. It was downlinked from the AGC through tracking stations, processed in the RTCC computers and then available for display.

carlclaunch5112:23 PM
Marc built a bit of hardware to produce the pulses that would have flooded the AGC with spurious movement data from the resolver for the LM rendezvous radar on Apollo 11.

cprossu12:24 PM
What amazes me is I found an old philco ford document that has all the numbers and descriptions of how mission control was built @carlclaunch51 . No pictures though

brianpagephotog12:24 PM
and you got the 1201/1202 alarms?

Ken Shirriff12:24 PM
As far as the mission control displays, it wasn't easy to project images onto a big screen. They used an early projection technology called Eidophor. It used electrostatic charges to deform an oil surface(!) A scanning electron beam "drew" on the surface and they reflected light off the oil. https://en.wikipedia.org/wiki/Eidophor

curiousmarc12:24 PM
That's why it would be really cool to get the telemetry stream out of the real thing with Carl's contraption.

Andrew Bingham12:24 PM
This Ars article from a while ago has a lot of information on the tech in the MCC - https://arstechnica.com/science/2012/10/going-boldly-what-it-was-like-to-be-an-apollo-flight-controller/2/

carlclaunch5112:24 PM
We are all a bit obsessed with getting to the exact conditions and hardware behavior that led to the 1201/1202 errors

Mike Stewart12:24 PM
yeah, one of the main things I'm working on right now is understanding/simulating *exactly* what happened with those alarms. so far I haven't managed to replicate the described behavior exactly

Mike Stewart12:25 PM
Marc's pulse generator overloads the computer too much -- we get 1201s and 1202s, but far too many of them

carlclaunch5112:25 PM
it is more complicated that it seems. Not just a stready stream of pulses at maximum rate.

Peter Bosch12:25 PM
how long did it take the original team to figure it out?

cprossu12:25 PM
And just like that managed to find this one.

cprossu12:25 PM

https://cdn2.hubspot.net/hubfs/413105/Apollo%20Press%20Kits/Philco%20Ford.pdf

HUBSPOT

Read this on Hubspot

cprossu12:26 PM
^ Still though we need to find out more info on the system 360/75 rtos

Jeremy Weatherford12:26 PM
Wasn't it something to do with two AC power supplies that could be in different phases?

Discussions