Smart Speaker from Scratch

Make an open source smart speaker for daily use

Similar projects worth following
The project is about to make an affordable, open source smart speaker. It should be able to run Amazon Alexa, Google Assistant, MyCroft, Home Assistant, Mopidy and etc. A case is required to make it suitable for daily use. The resources of hardware and software is also at

Why should we make an open source smart speaker when there are so many smart speakers on the market? Because these smart speakers are designed from vendors' perspective, not from users' perspective. These smart speakers have two issues:

+ privacy - these speakers are always-listening black boxes which scare lots of people
+ functionality - these speakers can be more powerful but are limited by the vendors

In 2014, the Amazon Echo was released. The new species of smart speakers is rapidly expanding since then. Although there are tons of smart speakers made by a variety of vendors, but they are basically the same as the Echo. All these smart speakers use a keyword to wake up, do speech recognition in the cloud and extend their skills in the cloud. Their software and software updates are controlled by the vendors. They all have the two issues.

One best way to eliminate privacy concerns is open source.

Most of smart speakers have very powerful ARM processors. Some even run Android OS just like an Android phone.  The smart speakers only is limited to run only one app, but they have the ability to run thousands of applications.

I want an open source smart speaker which can run Amazon Alexa and Google Assistant at the same time, and also is able to run Home Assistant, Mopidy, Snapcast, Nextcloud and etc. Maybe you want one too.


3d preview of the open source smart speaker, created with 3D Builder of Windows 10

3mf - 1.19 MB - 04/11/2019 at 10:36


Smart Speaker from Scratch.pptx

A presentation for a hackaday event at Chaihuo Maker Space

presentation - 9.54 MB - 04/09/2019 at 03:17



CAD DXF file for the paper case

AutoCAD DXF - 123.55 kB - 03/20/2019 at 14:43


  • 1 × Raspberry Pi
  • 1 × A microphone Array - ReSpeaker 4 Mic Linear Array Use a mic array to support algorithms such as DOA and Beamforming to get better audio quality
  • 1 × A Speaker
  • 1 × A case Use paper, wood or 3d print to make a case

  • Get started VOICEN OS: hey_wifi + ttyd + jupyterlab = awesome!

    Yihui10/10/2019 at 11:24 0 comments

    Last time I showed a web app of sending data over sound to setup Wi-Fi. A further step is to integrate it with a web terminal - ttyd and JupyterLab. These web apps are included in VOICEN OS which is based on Armbian OS. Have a look at the video to find out how it works.

    Steps to get started with VOICEN OS:

    1. Press and hold the touch key of the device for 4+ seconds to turn the device into Wi-Fi setup mode

    2. Boardcast WiFi SSID and password through sound on the hey_wifi web app

    3. After the device received the SSID and  password and connected to the Wi-Fi network, it sends its IP address to the web app, and then the web app generates a link which is linked to the device dashboard.

    4. Click the link with the device's IP address, we will get the dashboard of the device.

    5. We can open a web terminal (ttyd) to add a new user. The default user is root. The default password is 1234.

    6. We can also open JupyterLab which is a very powerful web IDE. It allows you to create and share documents that contain live code, equations, visualizations and narrative text. In this document, you can run python or shell scripts to play sound, plot graph and analyze data.

    That's it. Do you like it? It would be great if you kindly give me some feedback.

  • Send WiFi settings through sound wave

    Yihui06/21/2019 at 13:39 0 comments

    When we get a smart speaker, the first thing we need to do is to connect it to the WiFi network. There are a lot of ways to setup the WiFi connection, such as SmartConfig, Bluetooth Setup. While we have a cool way to setup the WiFi connection for a smart speaker. As the title says, we can transmit WiFi settings through sound. The smart speaker decodes WiFi settings by analyzing sound, and then connect to the WiFi network.

    The application contains two parts:

    1. web app, based on quiet.js + pure.css

    2. python app on the smart speaker, based on

    The code is at

  • Learn to use blender for 3D modeling

    Yihui04/16/2019 at 04:18 0 comments

    Last time I used Microsoft's 3D Builder to make a 3D model. The 3D Builder is a handy tool and very easy to learn, but it lacks some functions. So I switched to Blender. 

    Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation.

    The story of Blender is very interesting and inspiring (

  • Created a simple 3D model for the smart speaker

    Yihui04/11/2019 at 11:01 0 comments

    We designed a new mic array to build an open source smart speaker, you may wonder what it looks like. So we  created a simple 3D model to show a preview of the smart speaker. The model is created with 3D Builder of Windows 10. You can find the model at

    Top view - 4 R/Y/G/B LEDs, 1 touch pad

    Side view

    Front view

  • A presentation of Smart Speaker from Scratch

    Yihui04/09/2019 at 05:00 0 comments

    This is a presentation of  this project for a hackaday event at Chaihuo Maker Space.  It is worth sharing it here. You can also find the .pptx file at

    As an engineer, I always enjoy finding ways to complete a task. For me, the current task is making an open source smart speaker.

    Why we should make an open source smart speaker? What's the to-do-list? How to make it? The presentation is to answer these three questions.

    There are lots of existing smart speakers in the market. They are well-designed products. But they have two common issues. One is that they are always-listening black boxes, which scares many people. The other is that their functions is limited by their vendors. Smart speakers have very powerful ARM processors. They are just like smart phones with touch screens. Smart phones have so many apps, but smart speakers only run one app. Amazon Echo only has Alexa, Google Home only gets Google Assistant. We have no access inside of the smart speakers. We can't install any app on them.

    Maybe we can get some ideas from the history of routers and the OpenWrt project. Before we have OpenWrt, routers were just black boxes. After that, we replace routers' firmware with OpenWrt, and then install a variety of apps, such as web servers, proxies, file storage and etc.

    Maybe we can do the same to a smart speaker.

    But it's a little bit different. Although Amazon released Echo's source code, it's extremely hard to hack a Echo device. For the 2015 and 2016 edition of the Echo, we can boot it from an external SD to hack it. For the latest Echo, it have been fixed.

    For most of us, hacking an existing smart speaker is harder than making a new one. So why not make an open source smart speaker from scratch.

    To find out what we need to do, we should learn how a smart speaker works.

    These can be divided into 3 parts: hardware, software and document.

    Hardware is kind of simple. We need a Single-Board-Computer (Raspberry Pi, ...), a mic array and a speaker. Software is the main part. Fortunately, there are lots of open source resources we can use.

    Figure out what we need to do, then we can just find a way to do it.

    The first prototype is made with Raspberry Pi, ReSpeaker 4 Mic Array and a 45mm Speaker. The cost is a little high.

    So next step is to find some more affordable hardware. It seems NanoPi Neo Air is a good choice. It runs Armbian which is based Debian and is very similar with Raspbian.  A new mic array is being designed to reduce the cost.

    Looking forward to your feedback! We can also follow or join the project on or github.

  • New design: Linear 4 Mic Array & Amplifier

    Yihui03/25/2019 at 15:39 0 comments

    The first prototype based on ReSpeaker 4-Mic Linear Array has an issue that its microphones and FPC connector are on top of the PCB. It's not easy to attach the mic array to the case.

    Besides, we want a cheaper, more compact and lower power consumption option.

    So we just designed a new mic array. It works with NanoPi Neo Air, which is cheaper and lower power consumption than Raspberry Pi

    The initial thought was like:

    There are 4 microphones, 3 LEDs and 2 mounting holes. The mounting holes are also touch pads which can be used with metal screws as touch buttons. While it was hard to place all the components with this design, so we made some adjustments.

    The new design is:

    There are 4 microphones, 4 LEDs and 2 mounting holes and 1 touch pad + mounting hole.

    You may notice that "VOICEN - Linear 4 Mic Array" is on the PCB, which is the product name.

    We got the design work done at March 19th, and sent the gerber files to makerfabs, which provides PCB prototyping service at March 20th.

    Today, we got the PCBs!!!

    Based on the date on the packages, we can find that the PCB manufacturing process was actually super rapid. Its delivery took most of the time.

    Do you like the design or the name "VOICEN - Linear 4 Mic Array"? Just leave a comment here.

  • Prototype #1 based on Raspberry Pi + ReSpeaker 4 Mic Linear Array

    Yihui03/20/2019 at 14:18 0 comments

    This is a paper case smart speaker made with Raspberry Pi and ReSpeaker 4 Mic Linear Array. See also on Github


    The hardware includes:

    • Raspberry Pi 3B (or 3B+)
    • ReSpeaker 4 Mic Linear Array
    • 45mm Speaker
    • Laser-Cut Paper Case
    • + 2 Nylon Rivets (R3035 or R3045), the diameter of mounting holes is 3mm (ф3). Screws will also work, while rivets is easier to use.

    Raspberry Pi and ReSpeaker 4 Mic Linear Array can be found at Seeed Studio, and we get the 45mm speaker and nylon rivets from Alibaba or Taobao.

    Design a Paper Case

    To use a laser cutter, we need a CAD design file. I learned how to use a CAD software at college, but have not used CAD for many years. Fortunately, it is easy to design a paper case. We only have to know how draw lines and circels. Commercial CAD tools are quite expensive, but for this simple task, the open source CAD tool - LibreCAD is enough to get it done. You can design a unique paper case based on my work.

    If you have not used any CAD software, it will take some time to be familiar with a CAD software like LibreCAD. You can also use Python script to genrate the CAD DXF file. Manfred Moitzi's handy Python library - ezdxf will help us to do the design work. For example, to draw an array of speaker holes, using Python script is much easier.

    The Python script is like:

    import ezdxf
    dwg ='R2007')
    msp = dwg.modelspace()
    d = 40.
    n = int(40 / 1.5)
    delta = d / n
    start = (100, 100)
    msp.add_circle(start, 20)
    for x in range(-n, n):
        for y in range(-n, n):
            if y & 1:
                offset = 0.5
                offset = 0
            rx = ((x + offset)**2 + y**2)**0.5
            if rx <= (n/2 + 0.1):
                r = 0.35 # (0.25 * delta * (n/2 - rx) / (n/2) + 0.15 * delta)
                msp.add_circle((start[0] + (x + offset) * delta, start[1] + y * delta), r)

     The output will be:

    We can use both LibreCAD and Python script to make a complex design.


    For a smart speaker, the software is the key point. The easiest way is using Alexa Voice Service or Google Assistant SDK. If you want to know how a voice assistant works, you should try MyCroft. If you prefer to offline voice assistant, is a good option. You can find a full list of resources at voice-engine/make-a-smart-speaker.

  • Make a wood case for the smart speaker

    Yihui03/18/2019 at 06:42 0 comments

    With ( written by Florian Festi, make a laser-cut wood case is so easy.

  • Record & Play Audio on Linux

    Yihui03/11/2019 at 05:58 0 comments

    To make a smart speaker, the first thing to do in terms of software is to learn how to record & play audio. See the detail on Github

  • Offline + Online Voice Assistant

    Yihui03/08/2019 at 04:38 0 comments

    A smart speaker should not only work with Internet, but also work when offline. The ARM processors of most smart speakers are powerful enough to recognize voice commands offline. Here is a demo of how an offline + online voice assistant (Chinese version) looks like. 

    The conversation is:

    Human : 开灯  ( lights on )

    Speaker : ( recognized it offline and turned on the Mi LED Desktop Lamp through local network )

    Human : 几点了 ( what's the time )

    Speaker: 现在是19点34分 ( it's 19:34 )

    Human : 小呆小呆,20/3 等于多少  ( "a hotword", how much is equal to 20/3 )

    Speaker: 20/3等于6.667 (20/3 is equal to 6.667)

    "开灯 ( lights on )" and "几点了 ( what's the time )" are voice commands which are recognized offline. When a hotword is detected, it will enable online voice service to handle the conversation.

    In this demo, I use chumenwenwen's SDK for offline speech recognition and use dueros as the online voice service.

    The source code is

    Thanks to that we can control  the Mi LED Desktop Lamp through local network.

    Next step is to make an English version. Stay tuned!

View all 10 project logs

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates