Close
0%
0%

Voice Control RaspberyPi Smart AR Glasses

A smart ar glasses base on Raspberry and cloud service

Public Chat
Similar projects worth following
316 views
Ar glasses is a very interesting topic. There are products such as Google glasses and Microsoft Hololens. I see many people share their achievements. I also want to make my own smart AR glasses. Because I am Chinese, I use a lot of Chinese resources and services. You can also use resources and services suitable for you. First of all, I would like to thank all geeks who share their designs and achievements. Their achievements have inspired me.


First of all, I think my project is useful and interesting. So I first made a list of scenarios and set a priority based on my abilities. First, I used it to show the abilities of AR glasses and to identify the modules needed for the project. Current functions include: Current Time Query, Line Query (By Bus), animal and plant identification, qrcode recognition, and installing new functions based on qrcode.

首先,我认为我的项目是有用和有趣的。因此,我首先列出了一个场景列表,并根据我的能力设置了优先级。首先,我用它来展示AR眼镜的能力,并确定项目所需的模块。当前功能包括:当前时间查询、线路查询(通过总线)、动植物识别、二维码识别、基于二维码安装新功能。

The project is divided into hardware part and software part. At present, in order to highlight the characteristics of AR glasses, only voice control is supported. 

项目包含硬件部分和软件部分。目前为了突出AR眼镜的特色,仅支持语音控制。

The hardware part is mainly controlled by raspberry pi. Input and output of voice control by using USB sound card. Use the camera to capture pictures and display them with LCD screen and atomic mirror to ensure that the photos we take are what we need.In the future, from a practical point of view, a rocker or even a Bluetooth keyboard may be added. In fact, I have included a Bluetooth keyboard / mouse in my project, but I can't make good use of them. As the basic spectacle frame, I chose the head mounted magnifying glass as the foundation, which temporarily meets the current needs and is very easy to make. This also simplifies the current hardware selection. In the future, I hope to make it light enough to make it more like a pair of glasses. If you can have a better design, I hope you can share it.

硬件部分主要由树莓派作为主控。通过使用USB声卡进行语音控制的输入输出。使用摄像头捕获图片,并使用LCD屏幕和原子镜进行显示以确保我们拍下的照片是我们所需要的。将来从实用的角度,可能会加入摇杆甚至蓝牙键盘。实际上目前我的项目中已经包含了一个蓝牙键盘/鼠标,只是我还不能很好的利用它们。作为基础的眼镜架,我选择了头戴式放大镜作为基础,暂时满足目前的需求,而且非常容易制作。这也简化了目前的硬件选择。将来我希望能够做得足够轻量化,使之更像一副眼镜。如果大家能有更好的设计,也希望能够分享。

The software part mainly uses cloud services, including offline wake-up, instruction recognition and a simple l process engine based on data bus.

软件部分主要使用云服务,本地包括离线唤醒、指令识别和一个简单的基于数据总线的l流程引擎。...

Read more »

  • 1 × RaspberryPi Zero Main control
  • 1 × RaspberyPi 400(optional) Development environment main control
  • 1 × USB sound card Capacitors / Ceramic
  • 1 × LCD 2.0inch Capacitors / Ceramic
  • 1 × RaspberyPi Camera

View all 10 components

View all 4 project logs

  • 1
    Setup off-line speech recognition

    Now there are fewer and fewer open source speech recognition modules, and the test results are not ideal. The latest one that can be found is vosk, but vosk does not support raspberry pie 0. Finally, I chose Speechrecogonition + pocketsphinx as the solution for offline speech recognition. In order to ensure the recognition success rate, after several rounds of tests, I chose "my good friend" as the wake-up word, and made a special recognition file. You can also choose your favorite wake-up words. The only suggestion is to consider the recognition success rate.

    现在开源语音识别模块越来越少,而且经过测试效果并不理想。能够找到的最新的是VOSK,但是VOSK不支持树莓派0。最终我选择 Speechrecogonition + pocketsphinx作为离线语音识别的解决方案。为了确保识别成功率,经过多轮测试,我选择“My Good Friend”作为唤醒词,并且制作了专门的识别文件。你也可以选择你喜欢的唤醒词,唯一的建议就是考虑一下识别成功率。

  • 2
    Regiest Baidu AI

    In the field of cloud computing, I choose Baidu AI Open Platform as the basis. Baidu AI provides many aspects of cloud service support, such as voice, image, text, map, translation, etc. Baidu AI not only supports Chinese, but also other languages such as English. So you can use Baidu AI directly if you like.

    在云计算领域,我选择百度AI开放平台作为基础。百度AI提供了很多方面的云服务支持,比如,语音、图像、文字、地图、翻译等。百度AI不仅支持中文,也支持英文等其它语言。所以如果你愿意,也可以直接使用百度AI。

    Before using Baidu AI, you need to register users, create applications, and then add the AI services you need to your applications. Record the APPID, API_KEY, and SUCRETY_KEY of your applications and use them later.

    使用百度AI前,需要先注册用户,创建应用然后将你需要的AI服务加入到你的应用中。记录好应用的APPID、API_KEY和SECURTY_KEY,在后面将会用到它们。

    Baidu AI can be invoked using HTTP or directly using SDK provided by Baidu.

    百度AI可以使用HTTP的方式调用,也可以直接使用百度提供的SDK进行调用。

    For testing purposes only, most of Baidu AI APIs offer free trials from half a year to one year.

    如果仅仅为了测试,百度AI大多数API提供了半年到一年的免费试用。

  • 3
    Record voice

    There are two ways to record a sound: one is to trigger the recording by means of buttons, and the other is to record it fully automatically. Because I chose the full voice control mode, I chose the latter.

    录音有两种方式:一种是通过按钮等方式去触发录音;另一种方式是全自动录音。因为我选择的是全语音控制模式,所以我选择的是后一种方式。

    The key to automatic recording is when to start recording and when to end recording. The strategy used is to set a threshold based on the volume of the sound read through pyaudio to determine when to start and end.

    自动录音的关键是何时开始录音和何时结束录音。使用的策略是根据通过pyaudio读取到的声音的音量来设置阈值,用来确定何时开始和结束。

View all 8 instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates