Project Logs

Collapse

• tl;dr - self-hosted AI Yoda Speaks, Listens

  savant42 • 10/03/2025 at 19:02 • 0 comments

  My Mom said to go make a friend. So I made a friend.
  
  LOTS of updates, I'll just get to the link first: https://github.com/thesavant42/chainloot-Yoda-Bot-Interface

  That is the chat Interface for Yoda's AI stack. 

  He can now hold conversations on his own, using speech recognition, text to speech, and voice cloning.

  
  And did I say, locally, self-hosted? All of the parts are OpenAI API compatible, but open source / open models, running on my desktop PC with the graphics card it came wtih. (Ok, so the graphics card isn't terrible yet. Let's do the rundown.
  
  
   Hardware 

  
  - RTX 4070, 12 Gigs of VRAM), 

  - 64 Gigs System Ram
  

  - i7 somethinorother 
  
  

  Software

  
  - Docker, via WSL 2 on Windows 11 for hosting the containers
  - TTS-WebUI with Chatterbox TTS for Text to Speech and Whisper STT services

  - LM Studio for GGUF Model hosting, will likely swap out for llama.cpp once I finalize on the base model, so I can tune for performance. I have an ollama docker container I will test, as well.

  Model(s)

  - Using Qwen3 as a baseline, as it offers a decent mix of tool usage capabilities, reasonably decent at instruction following, and the Q4 quants are pretty ok for under 10 gigabytes. 
  - phi4 performs very well at chatting and role playing but is less reliable at tool usage in my testing, but I concede I have not tuned it whatsoever.
  - Smollm3 from Huggingface is shockingly capable for how smol it is. The unsloth quantization clocks in at ~ 1.8 gigabytes for 128k context. Not a typo, 128k context, under 2 gigs. WAT. If I can get this to use tools reliably, this is the one.
  

  - Voice Cloning utilizes the chatterbox model + some optimizations, but best of all is how it utilizes compiled flash_attn for speed boosts.
  
  Lesson learned on tts latency: Don't stream the text, stream the audio. Even if the model supports streaming characters in "real time", don't make the mistake of thinking that will help text to speech times. The TTS engine needs complete sentences to avoid audio artifacts, or even worse, waiting forever for the stream to end.
  
  https://github.com/rsxdalv/TTS-WebUI The real hero of the show
  
  https://huggingface.co/unsloth/SmolLM3-3B-128K-GGUF  How'd they do that?
  
  

• GitHub Link

  savant42 • 09/18/2025 at 18:47 • 0 comments

  I'm still trying to figure out how to make this site work with my brain, but it seems to me this link got buried. So, here's where the code lives.

  
  https://github.com/thesavant42/y0da
  
  

• Portability (Throwback Thursday)

  savant42 • 09/18/2025 at 18:46 • 0 comments

  Throwback to early experiments in portability.
  
  
  

• LayerOne 2025 Video

  savant42 • 09/18/2025 at 18:45 • 0 comments

  LayerOne
  
  Firstly, here's the LayerOne Talk. I am pretty happy with how it turned out, but I will admit I was working until the 11th hour on trying to complete the drive-train, and didn't leave myself enough time to polish the slides. As a result, I feel I rushed it, and glossed over some important details. Next time it will be better.
  

• Video: Seagulls! (Stop it NOW!)

  savant42 • 04/18/2024 at 04:52 • 0 comments

• The story so far

  savant42 • 04/18/2024 at 04:19 • 1 comment

  Current Status: 

  Main goal is functional, the Robot can be animated in real time, with an xbox (or similaer) controller, in animated sequences via Bottango, or in Bottango's "standalone driver" mode, which more closely mimics a seasonal decoration, and all animation lives on the microcontroller.

  ### Why did I do this?
  - big fan of combining technology and story to create magic
  - to learn the technologies involved
  - to see if I could
  - to have my own robot friend

  ### Who am I?
  - 15+ years in Information Security as an "penetration tester" (hacked applications and networks managed by my employers)
  - Raised in SoCal, big fan of animatronics, and in particular of the audioanimatronics featured in Disney Parks.

  # How does it work?
  ## How it works (unmodified)
  Yoda's animations are triggered by a passive infrared sensor mounted to his robes, which can also be bypassed to use a contact-closure trigger (step-pad, remote, whatever) or be set to "lights only" mode, wherein he just kinda...wiggles around. I didn't spend much time in this mode. 

  ### Control Panel
  Inside his control panel, yoda's animations are driven by a microctroller SoC from FMD. This soc is responsible for driving the left arm motor, the lightsaber LED, and passes off signals from audio recordings on the control board, off to Yoda's head via a 4-wire bus to his head. 

  ## Surgery: Opening Yoda's Head
  There is not a way that I've found to get around this step: You're going to have to open up Yoda's head. This requires getting to 4 cross-head screws that are obscured by the silicone on the back of Yoda's head. 

  The two screws nearest the neck's opening are within channels that can be seen through the opening of the neck, so it's pretty straight forward to follow them to their opening. The two near the top of Yoda's head are a little trickier, but you can feel them with your fingers if you're patient.

  I was able to take a phillips screwdriver, with about a 3" reach, and pierce the silicone over each screw. I then unscrewed them, and then fought the stupid screws to make sure I didn't lose them.

  With the 4 screws removed, the silicone covering is all that holds the two halves of the skull together. 

  ### Silicone Skin, ugh
  Silicone is neat, and skin like in several ways. It's also a pain to have to glue things to. As everyone is quick to tell me, "Only silicone sticks to silicone". To get the skull halves apart, you can delicately separate the pieces along the seam, which is visible from the outside of the Skull, and travels along the path of the shell's seam. The silicone is glued together and is jammed within channels of the head shell. Larger pieces of silicone are bridged together to form a tab within the skull. If possible, try and keep those tabs intact, as they are important in reattaching the skin to the shell. 

  [If you would like you glue the skin back together, you're going to need to buy a specialty silicone adhesive, such as Silpoxy. Note, this stuff is pricey, so have a plan when you use it.]

  [[ EDIT: No longer true! I found an inexpensive silicon glue that will work just fine. ]]

  With some patience and practice, eventually you will be able to get the two halves of the shell apart. BE DELICATE WHEN YOU PULL THE TWO HALVES APART, THERE ARE WIRES CONNECTING THEM TOGETHER!

  Now you can see the inner working of Master Yoda's 900 year old noodle: a smattering of servo wires, connecting up to a breakout board, which is also joined by a 2 wire motor connection for driving Yoda's mouth, and a 4 wire bus back to the main control panel. Sadly, this connector is NOT in fact i2c. It is +3v3, GND, Motor -, Data. Connecting my logic analyzer confirmed PWM on the Data wire, and these pins matched the silkscreen on the OEM control panel.

  ### Breakout within Yoda's Head

  There, it connects to a breakout board, and is joined by 8 3-wire servos, which connect to the board in a **JST PH 2.0mm** 2.0 plug-end connector, as opposed to...

  Read more »

View all 6 project logs