Close
0%
0%

Smart Speaker from Scratch

Make an open source smart speaker for daily use

Similar projects worth following
The project is about to make an affordable, open source smart speaker. It should be able to run Amazon Alexa, Google Assistant, MyCroft, Home Assistant, Mopidy and etc. A case is required to make it suitable for daily use. The resources of hardware and software is also at https://github.com/voice-engine/smart_speaker_from_scratch

Why should we make an open source smart speaker when there are so many smart speakers on the market? Because these smart speakers are designed from vendors' perspective, not from users' perspective. These smart speakers have two issues:

+ privacy - these speakers are always-listening black boxes which scare lots of people
+ functionality - these speakers can be more powerful but are limited by the vendors

In 2014, the Amazon Echo was released. The new species of smart speakers is rapidly expanding since then. Although there are tons of smart speakers made by a variety of vendors, but they are basically the same as the Echo. All these smart speakers use a keyword to wake up, do speech recognition in the cloud and extend their skills in the cloud. Their software and software updates are controlled by the vendors. They all have the two issues.

One best way to eliminate privacy concerns is open source.

Most of smart speakers have very powerful ARM processors. Some even run Android OS just like an Android phone.  The smart speakers only is limited to run only one app, but they have the ability to run thousands of applications.

I want an open source smart speaker which can run Amazon Alexa and Google Assistant at the same time, and also is able to run Home Assistant, Mopidy, Snapcast, Nextcloud and etc. Maybe you want one too.

speaker1.3mf

3d preview of the open source smart speaker, created with 3D Builder of Windows 10

3mf - 1.19 MB - 04/11/2019 at 10:36

Download

Smart Speaker from Scratch.pptx

A presentation for a hackaday event at Chaihuo Maker Space

presentation - 9.54 MB - 04/09/2019 at 03:17

Download

prototype1.dxf

CAD DXF file for the paper case

AutoCAD DXF - 123.55 kB - 03/20/2019 at 14:43

Download

  • 1 × Raspberry Pi
  • 1 × A microphone Array - ReSpeaker 4 Mic Linear Array Use a mic array to support algorithms such as DOA and Beamforming to get better audio quality
  • 1 × A Speaker
  • 1 × A case Use paper, wood or 3d print to make a case

  • Learn to use blender for 3D modeling

    Yihui2 days ago 0 comments

    Last time I used Microsoft's 3D Builder to make a 3D model. The 3D Builder is a handy tool and very easy to learn, but it lacks some functions. So I switched to Blender. 

    Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation.

    The story of Blender is very interesting and inspiring (youtu.be/qJEWOTZnFeg).

  • Created a simple 3D model for the smart speaker

    Yihui04/11/2019 at 11:01 0 comments

    We designed a new mic array to build an open source smart speaker, you may wonder what it looks like. So we  created a simple 3D model to show a preview of the smart speaker. The model is created with 3D Builder of Windows 10. You can find the model at https://cdn.hackaday.io/files/1642217026031264/speaker1.3mf

    Top view - 4 R/Y/G/B LEDs, 1 touch pad

    Side view

    Front view

  • A presentation of Smart Speaker from Scratch

    Yihui04/09/2019 at 05:00 0 comments

    This is a presentation of  this project for a hackaday event at Chaihuo Maker Space.  It is worth sharing it here. You can also find the .pptx file at https://cdn.hackaday.io/files/1642217026031264/Smart%20Speaker%20from%20Scratch.pptx

    As an engineer, I always enjoy finding ways to complete a task. For me, the current task is making an open source smart speaker.

    Why we should make an open source smart speaker? What's the to-do-list? How to make it? The presentation is to answer these three questions.

    There are lots of existing smart speakers in the market. They are well-designed products. But they have two common issues. One is that they are always-listening black boxes, which scares many people. The other is that their functions is limited by their vendors. Smart speakers have very powerful ARM processors. They are just like smart phones with touch screens. Smart phones have so many apps, but smart speakers only run one app. Amazon Echo only has Alexa, Google Home only gets Google Assistant. We have no access inside of the smart speakers. We can't install any app on them.

    Maybe we can get some ideas from the history of routers and the OpenWrt project. Before we have OpenWrt, routers were just black boxes. After that, we replace routers' firmware with OpenWrt, and then install a variety of apps, such as web servers, proxies, file storage and etc.

    Maybe we can do the same to a smart speaker.

    But it's a little bit different. Although Amazon released Echo's source code, it's extremely hard to hack a Echo device. For the 2015 and 2016 edition of the Echo, we can boot it from an external SD to hack it. For the latest Echo, it have been fixed.

    For most of us, hacking an existing smart speaker is harder than making a new one. So why not make an open source smart speaker from scratch.

    To find out what we need to do, we should learn how a smart speaker works.

    These can be divided into 3 parts: hardware, software and document.

     
    Hardware is kind of simple. We need a Single-Board-Computer (Raspberry Pi, ...), a mic array and a speaker. Software is the main part. Fortunately, there are lots of open source resources we can use.

    Figure out what we need to do, then we can just find a way to do it.

    The first prototype is made with Raspberry Pi, ReSpeaker 4 Mic Array and a 45mm Speaker. The cost is a little high.

    So next step is to find some more affordable hardware. It seems NanoPi Neo Air is a good choice. It runs Armbian which is based Debian and is very similar with Raspbian.  A new mic array is being designed to reduce the cost.

    Looking forward to your feedback! We can also follow or join the project on hackaday.io or github.

  • New design: Linear 4 Mic Array & Amplifier

    Yihui03/25/2019 at 15:39 0 comments

    The first prototype based on ReSpeaker 4-Mic Linear Array has an issue that its microphones and FPC connector are on top of the PCB. It's not easy to attach the mic array to the case.

    Besides, we want a cheaper, more compact and lower power consumption option.

    So we just designed a new mic array. It works with NanoPi Neo Air, which is cheaper and lower power consumption than Raspberry Pi

    The initial thought was like:

    There are 4 microphones, 3 LEDs and 2 mounting holes. The mounting holes are also touch pads which can be used with metal screws as touch buttons. While it was hard to place all the components with this design, so we made some adjustments.

    The new design is:

    There are 4 microphones, 4 LEDs and 2 mounting holes and 1 touch pad + mounting hole.

    You may notice that "VOICEN - Linear 4 Mic Array" is on the PCB, which is the product name.

    We got the design work done at March 19th, and sent the gerber files to makerfabs, which provides PCB prototyping service at March 20th.

    Today, we got the PCBs!!!

    Based on the date on the packages, we can find that the PCB manufacturing process was actually super rapid. Its delivery took most of the time.

    Do you like the design or the name "VOICEN - Linear 4 Mic Array"? Just leave a comment here.

  • Prototype #1 based on Raspberry Pi + ReSpeaker 4 Mic Linear Array

    Yihui03/20/2019 at 14:18 0 comments

    This is a paper case smart speaker made with Raspberry Pi and ReSpeaker 4 Mic Linear Array. See also on Github

    Hardware

    The hardware includes:

    • Raspberry Pi 3B (or 3B+)
    • ReSpeaker 4 Mic Linear Array
    • 45mm Speaker
    • Laser-Cut Paper Case
    • + 2 Nylon Rivets (R3035 or R3045), the diameter of mounting holes is 3mm (ф3). Screws will also work, while rivets is easier to use.

    Raspberry Pi and ReSpeaker 4 Mic Linear Array can be found at Seeed Studio, and we get the 45mm speaker and nylon rivets from Alibaba or Taobao.

    Design a Paper Case

    To use a laser cutter, we need a CAD design file. I learned how to use a CAD software at college, but have not used CAD for many years. Fortunately, it is easy to design a paper case. We only have to know how draw lines and circels. Commercial CAD tools are quite expensive, but for this simple task, the open source CAD tool - LibreCAD is enough to get it done. You can design a unique paper case based on my work.

    If you have not used any CAD software, it will take some time to be familiar with a CAD software like LibreCAD. You can also use Python script to genrate the CAD DXF file. Manfred Moitzi's handy Python library - ezdxf will help us to do the design work. For example, to draw an array of speaker holes, using Python script is much easier.

    The Python script is like:

    import ezdxf
    
    dwg = ezdxf.new('R2007')
    msp = dwg.modelspace()
    d = 40.
    n = int(40 / 1.5)
    
    delta = d / n
    
    start = (100, 100)
    
    msp.add_circle(start, 20)
    
    for x in range(-n, n):
        for y in range(-n, n):
            if y & 1:
                offset = 0.5
            else:
                offset = 0
            rx = ((x + offset)**2 + y**2)**0.5
            if rx <= (n/2 + 0.1):
                r = 0.35 # (0.25 * delta * (n/2 - rx) / (n/2) + 0.15 * delta)
                msp.add_circle((start[0] + (x + offset) * delta, start[1] + y * delta), r)
    
    dwg.saveas('speaker_hole.dxf')

     The output will be:

    We can use both LibreCAD and Python script to make a complex design.

    Software

    For a smart speaker, the software is the key point. The easiest way is using Alexa Voice Service or Google Assistant SDK. If you want to know how a voice assistant works, you should try MyCroft. If you prefer to offline voice assistant, Snips.ai is a good option. You can find a full list of resources at voice-engine/make-a-smart-speaker.

  • Make a wood case for the smart speaker

    Yihui03/18/2019 at 06:42 0 comments

    With Boxes.py (https://github.com/florianfesti/boxes/) written by Florian Festi, make a laser-cut wood case is so easy.

  • Record & Play Audio on Linux

    Yihui03/11/2019 at 05:58 0 comments

    To make a smart speaker, the first thing to do in terms of software is to learn how to record & play audio. See the detail on Github

  • Offline + Online Voice Assistant

    Yihui03/08/2019 at 04:38 0 comments

    A smart speaker should not only work with Internet, but also work when offline. The ARM processors of most smart speakers are powerful enough to recognize voice commands offline. Here is a demo of how an offline + online voice assistant (Chinese version) looks like. 

    The conversation is:

    Human : 开灯  ( lights on )

    Speaker : ( recognized it offline and turned on the Mi LED Desktop Lamp through local network )

    Human : 几点了 ( what's the time )

    Speaker: 现在是19点34分 ( it's 19:34 )

    Human : 小呆小呆,20/3 等于多少  ( "a hotword", how much is equal to 20/3 )

    Speaker: 20/3等于6.667 (20/3 is equal to 6.667)

    "开灯 ( lights on )" and "几点了 ( what's the time )" are voice commands which are recognized offline. When a hotword is detected, it will enable online voice service to handle the conversation.

    In this demo, I use chumenwenwen's SDK for offline speech recognition and use dueros as the online voice service.

    The source code is https://github.com/voice-engine/wenwen/blob/master/alpha.py

    Thanks to https://github.com/skorokithakis/python-yeelight that we can control  the Mi LED Desktop Lamp through local network.

    Next step is to make an English version. Stay tuned!

View all 8 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates