Close
0%
0%

Central Scrutinizer: a serial adapter for M1/M2/M3

Open source serial adapter / reboot controller for Apple Silicon Macs

mazmaz
Public Chat
Similar projects worth following
Starting from
$15.00
mzyngier has 61 orders / 0reviews
Ships from United Kingdom
Serial adapter / reboot controller for Apple M1/M2/M3

The Central Scrutinizer exposes the serial lines available on Apple M1/M2/M3's USB-C DFU port. It does so over a convenient USB connection (USB CDC -- no special driver needed).
It is also capable of performing a hard-reset of the machine by talking to the Mac's own PD controller using VDMs (Vendor Defined Messages).

Finally, it offers a USB2.0 pass-through that can be used to boot the machine over USB (tethered mode) using the m1n1 bootloader (part of the Asahi project). With some additional hacking, it can be used to route the SWD signals and other barely documented things.

The full project (hardware, firmware, and even 3D-printable case) is open-source, under the MIT license. If you cannot be bothered to build you own and yet want to play, I have a few boards from previous batches on Tindie.

The Central Scrutinizer provides a low-level serial port over USB to operating system developers hacking on fruity hardware, which is useful when you cannot count on display/keyboard to be available (headless machine), or need to log information but can't use any local storage.

If you're using MacOS on your Apple Silicon machine, you probably don't need the serial console aspect, but the reboot feature can prove useful if you crash the machine and need to remotely reset it.

It is build with a small PCB of my own design, coupled to a RaspberryPi Pico that runs the control firmware which supports various modes, including bridging to an actual serial RS232 port.

Plenty of extra information in both git repositories (see the left sidebar for links to the KiCad files as well as the firmware source code)!

Want to see an M1 booting Linux over the serial port? Here's one!

Being an open-source project, I'm always happy to take patches and suggestions.

  • Of PCB assembly quality control...

    maz04/05/2024 at 15:06 3 comments

    It now has been about 18 months since I have started designing and producing the Central Scrutinizer. It has been a fun trip into building something physical, as opposed to just writing software.

    I have learned a lot in the process (see my first log entry), and I'm really glad that I managed to build something that people do find useful.

    However, a constant source of frustration has been the PCB assembly process. I only own a basic soldering iron, and my eyesight is not what it once was, so while I'm happy to assemble the through-hole stuff, I leave the SMD work to the PCB shop, aka JLCPCB. The whole project is configured to make it super easy to upload the design to their website and get working boards by mostly accepting the proposed default options.

    I'm also not a huge operation, far from it. I produce boards in small batches (about 10 to 20 at a time), giving most of them for free to fellow open-source developers, and selling whatever is left on Tindie. This makes the buyers indirect sponsors of the open-source projects that use the boards, something I quite like. I don't make any money out of it, other than the odd beer with fellow hackers.

    So it's all good, Sort of. What really gets me is the poor level of quality control that gets applied to the produced PCB. And poor doesn't quite describe it.

    Here's the latest example:

    U5 is off. Not a bit off due to the component having drifted. It is exactly off by a whole pin. Something that any automated checking should spot immediately. And that's the issue. I'm pretty sure that given the rock-bottom price for the PCBA  process, there isn't any margin left for actual quality control. So whatever crap that has been produced ships.

    I've seen other similar issues, such as micro-USB connectors soldered at a funny angle, solder bridges and the odd dead component. For each of those, that's another board on the scrap pile. The planet thanks you. Not. Maybe I should invest in rework equipment...

    Frankly, I'd happily pay significantly more for a reliable service with  an ordering procedure as good as JLCPCB's. If you know of any, please let me know! I wonder how other people with my level of experience (close to zero) and equipment (similarly null) cope with this...

    On the plus side, I've grown a solid enough test procedure, so I'm able to spot a bad board within a few minutes, and only ship the good'uns. I guess that's as good as it gets for an outcome.

  • M3, baby!

    maz11/29/2023 at 20:59 0 comments

    As I found out today, the Central Scrutinizer is working out of the box on a M3-Max machine (and no, that's not mine):

    https://oftc.irclog.whitequark.org/asahi-dev/2023-11-29#32702519;

    I guess the good folks in Cupertino had no reason to change something that was working just fine!

  • Shape of things to come?

    maz11/19/2023 at 12:27 0 comments

    People who know me know that I don't really care about non-ARM architectures. I have been involved with it for a long time, and I'm yet to be excited by what has been coming from the other side of the fence (on the other hand, if you know of a new CPU architecture that doesn't repeat all the combined  mistakes of x86+ARM+Power, please let me know).

    That was one of the reasons to pick the RPi Pico as the uC for the Central Scrutinizer: an architecture I was comfortable enough with and can debug easily.

    The same people also know that my OS of choice is Linux. I have been doing kernel stuff for a few decades, and this is the only OS I have used since 1993. Yeah, it's been that long.

    Of course, picking the Pico meant that running Linux (*real* Linux, not uC/Linux) was out of the question. And for something that is essentially a set of state machines, it really doesn't matter much.

    But then I saw this. The form factor is of course a direct nod to the Pico, but the characteristics are on a different scale: a 64bit CPU that is capable of running Linux, plenty of peripherals (including Ethernet), and a price that is close enough to that of the Pico that I start thinking of new possibilities... What if, instead of plugging the CS board into a console server, the CS was becoming *the* console server? And make use of the MIPI interface to do HDMI capture? Does anyone else think "full BMC"?

    Of course, people paying attention will say "but this is a RISC-V CPU, what's wrong with you?". And they'd be right to ask. But maybe that's the sort of incentive I need to start looking at other architectures... ;-). And if you know of an interesting arm64 board with the same form factor, please shout!

    Now, this thing is as far as you can imagine from being usable. There is a v5.10 BSP for it, which I am not going to touch with a barge pole (it mandates fetching unknown x86 binaries and running them locally, which isn't going to work anyway on my arm64 box). So I've started looking at the upstream support for it, and as of v6.7-rc1 we have interrupts, timer and UART working. Nothing else. Not going to ship anything with it any time soon.

    But I definitely like the idea of it, and I plan to spend some time improving the upstream support. Maybe in a year or so, I'll be able to get to the point where the firmware for the CS is only a userspace process, and nothing else. That'd be cool... and would open the door to more interesting experiments!

    Stay tuned!

  • I didn't have this in mind, but...

    maz11/17/2023 at 14:32 0 comments

    It happened a while ago, but I thought I'd mention this hack on top of my own hack:

    Note that I'm not affiliated with this in any shape or form, other than being pleased to see someone making an unexpected use of it!

  • Everything looks better in gold

    maz11/01/2023 at 18:06 0 comments

    A couple of  weeks ago, I ran out of Central Scrutinizer boards. Between those that went on Tindie and the ones that I handed to fellow developers, that's about 50 boards that I distributed. Not bad for a pretty niche project!

    I hesitated making another run, as I'd prefer to see people building their own. But since I kept getting requests, I ended up producing another small run of v3.2 boards. This time using the ENIG process instead of the standard HASL that I used so far. So instead of the silvery and slightly bumpy finish, I get these gold-colored and super-flat pads.
    It doesn't seem much, but it makes quite a difference at assembly time. Soldering them becomes almost an enjoyable experience!

    Other than that, they are still the same! I took this opportunity to retire my last v0 and v1 boards, and treat myself to some quality gold stuff! I'll still maintain the firmware for these early revisions though.

  • Behold the new test jig monster...

    maz10/26/2023 at 19:49 0 comments

    If  you have read my first (and way too long) log entry, you will be familiar with my first attempt at building a test jig, which looked like this:

    Not exactly pretty, but functional enough: slap a board to be tested on top, secure it with the bolts, and let the Pico that lives underneath do the work. I was able to test a number of boards this way without having to solder anything.

    However, this is a one off, and I thought I'd build something a bit less hackish. Also, I would like to be able to test the dual-board setup, which requires to be more inventive.

    So I settled on a PCB design that would give me some flexibility: a jig PCB can either hold a board or a Pico, but not both. If you want to connect it to anything else, I have an additional connector for that in the middle:

    Of course, this results in twice the number of PCBs for the same thing as my original contraption:

    The (modest) advantage of this version is that I can swap the top interposer for one that has a different configuration. And this is not restricted to a Central Scrutinizer board either.

    Now, back to my earlier plan of being able to test two boards at once. I found this carrier board (https://thepihut.com/products/pico-omnibus-dual-expander) that allows me to plug two boards. With two test jigs plugged in, and two CS boards to boot, that's quite a setup! I have since replaced the metal screws and bolts with nylon equivalents, which is much safer.

    Of course, the PCB design is pushed out in the repo. Feel free to reuse it for something else!

  • Serial to serial

    maz10/17/2023 at 09:54 0 comments

    These days, a serial adapter is widely understood as a way to convert a TTL serial stream to a more "modern" (read: more pointlessly complicated) protocol. And that's what the Central Scrutinizer does in its most basic incarnation by converting the serial stream to USB CDC. 

    However, the good old RS232 is still alive and kicking, specially outside of the embedded environments My test farm, for example, is largely composed of machines that have RS232 interfaces which I connect to my (home made) console server.

    There is also a trust issue: It is easier to integrate a pure serial device conforming to the RS323 spec than a USB device running untrusted firmware and potentially capable of tripping interesting bugs on the host side. BadUSB anyone? Of course, *I* trust my code. Who else does? Rhetorical question!

    For all these reasons, it is sometimes desirable to trade the convenience of a USB serial port for the robustness and trust of a RS232 connection.

    Exposing a Mac serial port over RS232 shouldn't be a big deal: a couple of level shifters, and that'd be it. However, if you want the current behaviour of multiplexing control (such as the reset capability) and data planes over the same channel, you need some on-board processing. Which is fine, as the Pico has two UARTs, and we can use the second one for the upstream connection.

    I was looking at fitting a MAX3232 on the Central Scrutinizer when I discovered this add-on board that seems designed for it: https://www.waveshare.com/wiki/Pico-2CH-RS232 A MAX3232, all the required passives, and a bunch of visually annoying LEDs. What's not to like? And it is available almost everywhere (eBay, Pi Hut...).

    A couple of software hacks later (all pushed out onto the git repo), we have a full serial to serial bridge. Look Ma, no USB! Well, you still need to apply some 5v somewhere, so you actually need a USB connection. But it can be to a dumb (and trusted) power supply, without any data involved. Oh, and the SW automatically guesses that the RS232 board is connected, so the same build works in all conditions.

    A slightly different hack would be to use both RS232 channels and split the control plane from the data stream. Not sure if there is an interest for that though.

    In any case, it is pretty amusing to see a modern Mac being accessible over an RS232 connection!

  • Building the Central Scrutinizer, one mistake at a time

    maz10/08/2023 at 11:43 0 comments

    What is this?

    I've been building this serial adapter for M1/M2 Macs for some time now, and I thought that I'd document how it all came to life. There is nothing amazing in it. Only that it demonstrates the power of open source software and hardware. I didn't invent anything. I just put two and two together, and end-up with something that people found useful. It doesn't get better than that.


    February 2021

    M1 mini has landed. Screw macOS, I have other plans.

    The Asahi project already has some basics in place (the m1n1 firmware), and thankfully describes a rough method to get a serial port out of the machine: https://github.com/AsahiLinux/vdmtool . That's where everything starts.

    I'm rather adverse to soldering, so aiming for a Lego-like setup by using a bunch of things sourced from Tindie/Amazon/Whatever as well as making liberal use of my junk box:

    A couple of hours of  "work" assembling everything, tons of flying leads all over the place (and probably the reliability of a Morris Minor), and I end-up with this:


    Of course, nothing works, and while the Arduino seems to correctly talk to the FUSB302, that sucker keeps connecting/disconnecting. After some debugging (and Marcan's help), I realise that both CC lines are connected all the way, which isn't quite what USB-C expects... RTFM!

    Multiple solutions:

    • hack the FUSB302 board
    • hack the cable
    • hack the pass-through board

    It turns out that the pass-through board has a number of vias to route all the signals across. Which is great news, as I have a very small screwdriver that I can apply to one of these vias (yes, this is terrible -- I don't care):

    ... and that's one CC line gone for good. Hopefully I won't regret this. And guess what, it all starts working. We have lift off! Both serial and reboot are working, and I can hack on the M1 remotely, without having to hear the stupid "Dong" each time the machine reboots (about 20 times an hour). I narrowly avoided taking the same screwdriver to the speaker...


    Now that's what I call engineering! Note the unused micro-USB connector on the FUSB302 board. Eventually, I used that connection to upload payloads to the Mac. To this day, this machine still boots entirely from USB.

    And frankly, this should have been it. End of the story.


    November 2022

    Almost two years later. The M1 is still going strong. I wrote a PCIe driver for it, KVM/arm64 is up and running (although I had to work around some of ugly deviations from the architecture as well as some silly HW bugs), and it is my main KVM development machine.

    Only problem is that I now have a *second* M1. And I need another serial contraption. Yes, first world problem. Frantically trying to buy the ReclaimerLabs board again, but it is sold out, and the seller doesn't reply to my anxious messages. At no point it occurs to me that I could just upload the design to a PCB shop website and get it done. No.

    Instead, I decide to reinvent the wheel. Just because I can. And my square wheel looks like this:

    The big square in the middle is a bare FUSB302 soldered on a DIP adapter, the USB-C breakout board is a much cheaper version that exposes the SBU signals (just chop these pesky resistors off the CC lines), and dangling is the same FTDI adapter. And it works! Well, as long as you don't try to move the ball of wires around. Or breathe next to it.

    OK, I...

    Read more »

View all 8 project logs

  • 1
    Build it!

    Everything is over here:

    https://git.kernel.org/pub/scm/linux/kernel/git/maz/cs-hw.git/tree/README.txt


    * Building your own:
    
    This is the preferred option, really. I've used JLCPCB for all the
    revisions above, both good and bad. If you're not good at soldering
    very small stuff, get them to do the heavy lifting. The process is
    very straightforward, but you will have to produce at least 5
    boards. Find some fellow hackers and share the costs!
    
    The 'production' directory contain the Gerber files in a single ZIP,
    BOM and positions in CSV format (produced by the JLCPCB fabrication
    toolkit plugin). You only need to upload those to the JLCPCB website,
    check the orientation of the components, and let it rip. I've used the
    basic FR-4 with HASL finish, Economic PBCA type, and the result is OK.
    
    Things to be aware of:
    
    - The assembly side is at the *bottom* of the board. It is only the
      Pico that goes on the top side. Make sure you pick the bottom side
      in the web interface.
    
    - Warning about JP[1-8] being absent from the position file can be
      safely ignored. The web interface gets confused about having pads
      without anything soldered to them.
    
    - Components sometimes are out of stock. While you can often quickly
      find a direct replacement in the LCSC library, be very careful about
      the footprint, specially with connectors. You may have to amend the
      PCB design and regenerate the production files.
    
    - The orientation bit is absolutely crucial. While I do my best to
      reconcile what KiCad and JLCPCB respectively think of the
      orientation of components by adding rotation offsets to the
      metadata, you absolutely need to check this carefully before
      starting the build. Look for the orientation markers on the PCB and
      use the web editor to align the component positional markers with
      them. Do not expect JLCPCB to catch these mistakes for you.
    
    Of course, JLCPCB isn't the only game in town, and their QC is dodgy
    at best. If you know of a decent PCB+assembly shop and can help with
    making the PCB easy to get produced, let me know. I'm happy to add
    metadata to the schematic to drive a fabrication plugin/framework if
    there is one.
    
    Ideally, I'd like to have a collection of setups that allow people to
    build boards locally instead of having shipped around the world...
    
    You can also use one of these PCB shops to only produce the PCB and
    populate the board yourself. If you are in this category, I assume you
    know what you are doing and you need no further advice from me!
    
    * Final assembly:
    
    Once you have managed to get your hands on an populated CS board, you
    must solder a Pico to it. Make sure that:
    
    - the two boards are back to back (all components are on the outside
      of the board sandwich).
    
    - the two micro-USB connectors on the same end of the assembly -- if
      you have the USB-C and Pico micro-USB close to each other, you're
      doing it wrong.
    
    - the two boards are far apart enough that the two micro-USB
      connectors can be plugged without interfering with one another.
      I've used male turned pin strips as the connector, and they are
      great.  Normal header pins are also fine if you're not bothered with
      one side sticking out more than it normally should. You may need to
      add extra spacers to keep the boards apart.
    
    - you use a bog standard RPi Pico. Not a Pico W, not a one of the many
      variants with a creative pinout... It may work, it may not. Be
      cheap, don't use anything fancy. The Pico W is known to have a
      different GPIO assignment, which interferes with the current
      SW. Nothing terrible, but enough to spend some time debugging it.
    
    

View all instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates