Close
0%
0%

Room Assistant

A project for an voice and gesture controlled

Similar projects worth following
Things this *thing* should do:
0.) Be unobtrusive, be helpful, be self explanatory, be VALUABLE.

1.) Have a microphone, we want to talk to it
2.) Have a speaker system of some sort, it needs to talk to us
3.) Have a camera capable of at least 15fps and 640x480(minimum), it needs to see us, and react to both our presence, and our gestures
4.) PIR Sensor, that way we can get a fast motion detection event, and in the event privacy mode is enabled, still respond to motion
5.) It should have sensors for Temp/Humidity, CO2/Smoke would be nice as well, because it should know the state of the room its in.
~~~~~~~~~~~~~THINGS I *HOPE* IT WILL DO~~~~~~~~~~~~`
6.) netboot, with no sd card, big hope
7.) pull power via real POE, or at least fake POE for now (+12 on 4/5 Gnd on 7/8)
8.) boot *FAST*, it should never reboot in an ideal world, but if it does have to reboot, it should be FAST

All parts of the project can be found by clicking here:

https://hackaday.io/projects?tag=JAW.HA

So far I have picked the orange pi one as the SBC of choice for this application.

Orange Pi One (and camera):

https://www.aliexpress.com/item/Orange-Pi-One-SET-9-Orange-Pi-One-and-Camera-Camera-with-wide-angle-lens-for/32660642513.html

I will be using the PAM8403 as the amplifier, the boards I got for prototyping are these:

https://www.aliexpress.com/item/10PCS-PAM8403-Super-mini-digital-amplifier-board-2-3W-Class-D-digital-amplifier-board-efficient-2/1547727101.html

0.16usd each? Holy cow batman...

Hands on photos coming soon to the logs

  • CPU stable!

    James Wynhoff03/28/2018 at 22:39 0 comments

    As per the last post I changed the CPU to run at a fixed speed, and here is the output of cpufreq-info now :)

    cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009
    Report errors and bugs to cpufreq@vger.kernel.org, please.
    analyzing CPU 0:
      driver: cpufreq-sunxi
      CPUs which run at the same hardware frequency: 0 1 2 3
      CPUs which need to have their frequency coordinated by software: 0 1 2 3
      maximum transition latency: 2.00 ms.
      hardware limits: 1.20 GHz - 1.20 GHz
      available frequency steps: 60.0 MHz, 120 MHz, 240 MHz, 312 MHz, 408 MHz, 480 MHz, 504 MHz, 528 MHz, 576 MHz, 600 MHz, 624 MHz, 648 MHz, 672 MHz, 720 MHz, 768 MHz, 816 MHz, 864 MHz, 912 MHz, 960 MHz, 1.01 GHz, 1.06 GHz, 1.10 GHz, 1.15 GHz, 1.20 GHz, 1.25 GHz, 1.30 GHz, 1.34 GHz, 1.44 GHz, 1.54 GHz
      available cpufreq governors: interactive, conservative, ondemand, powersave, userspace, performance
      current policy: frequency should be within 1.20 GHz and 1.20 GHz.
                      The governor "performance" may decide which speed to use
                      within this range.
      current CPU frequency is 1.20 GHz (asserted by call to hardware).
      cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:0.00%, 504 MHz:0.00%, 528 MHz:0.00%, 576 MHz:0.00%, 600 MHz:0.00%, 624 MHz:0.00%, 648 MHz:0.00%, 672 MHz:0.00%, 720 MHz:0.00%, 768 MHz:0.00%, 816 MHz:0.00%, 864 MHz:0.00%, 912 MHz:0.00%, 960 MHz:0.00%, 1.01 GHz:0.00%, 1.06 GHz:0.00%, 1.10 GHz:0.00%, 1.15 GHz:0.00%, 1.20 GHz:100.00%, 1.25 GHz:0.00%, 1.30 GHz:0.00%, 1.34 GHz:0.00%, 1.44 GHz:0.00%, 1.54 GHz:0.00%  (1)
    analyzing CPU 1:
      driver: cpufreq-sunxi
      CPUs which run at the same hardware frequency: 0 1 2 3
      CPUs which need to have their frequency coordinated by software: 0 1 2 3
      maximum transition latency: 2.00 ms.
      hardware limits: 1.20 GHz - 1.20 GHz
      available frequency steps: 60.0 MHz, 120 MHz, 240 MHz, 312 MHz, 408 MHz, 480 MHz, 504 MHz, 528 MHz, 576 MHz, 600 MHz, 624 MHz, 648 MHz, 672 MHz, 720 MHz, 768 MHz, 816 MHz, 864 MHz, 912 MHz, 960 MHz, 1.01 GHz, 1.06 GHz, 1.10 GHz, 1.15 GHz, 1.20 GHz, 1.25 GHz, 1.30 GHz, 1.34 GHz, 1.44 GHz, 1.54 GHz
      available cpufreq governors: interactive, conservative, ondemand, powersave, userspace, performance
      current policy: frequency should be within 1.20 GHz and 1.20 GHz.
                      The governor "performance" may decide which speed to use
                      within this range.
      current CPU frequency is 1.20 GHz (asserted by call to hardware).
      cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:0.00%, 504 MHz:0.00%, 528 MHz:0.00%, 576 MHz:0.00%, 600 MHz:0.00%, 624 MHz:0.00%, 648 MHz:0.00%, 672 MHz:0.00%, 720 MHz:0.00%, 768 MHz:0.00%, 816 MHz:0.00%, 864 MHz:0.00%, 912 MHz:0.00%, 960 MHz:0.00%, 1.01 GHz:0.00%, 1.06 GHz:0.00%, 1.10 GHz:0.00%, 1.15 GHz:0.00%, 1.20 GHz:100.00%, 1.25 GHz:0.00%, 1.30 GHz:0.00%, 1.34 GHz:0.00%, 1.44 GHz:0.00%, 1.54 GHz:0.00%  (1)
    analyzing CPU 2:
      driver: cpufreq-sunxi
      CPUs which run at the same hardware frequency: 0 1 2 3
      CPUs which need to have their frequency coordinated by software: 0 1 2 3
      maximum transition latency: 2.00 ms.
      hardware limits: 1.20 GHz - 1.20 GHz
      available frequency steps: 60.0 MHz, 120 MHz, 240 MHz, 312 MHz, 408 MHz, 480 MHz, 504 MHz, 528 MHz, 576 MHz, 600 MHz, 624 MHz, 648 MHz, 672 MHz, 720 MHz, 768 MHz, 816 MHz, 864 MHz, 912 MHz, 960 MHz, 1.01 GHz, 1.06 GHz, 1.10 GHz, 1.15 GHz, 1.20 GHz, 1.25 GHz, 1.30 GHz, 1.34 GHz, 1.44 GHz, 1.54 GHz
      available cpufreq governors: interactive, conservative, ondemand, powersave, userspace, performance
      current policy: frequency should be within 1.20 GHz and 1.20 GHz.
                      The governor "performance" may decide which speed to use
                      within this range.
      current CPU frequency is 1.20 GHz (asserted by call to hardware).
     cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:0.00%, 504 MHz:0.00%, 528 MHz:0.00%, 576 MHz:0.00%, 600 MHz:0.00%, 624 MHz:0.00%, 648 MHz:0.00%, 672 MHz:0.00%, 720 MHz:0.00%, 768 MHz:0.00%, 816 MHz:0.00%, 864...
    Read more »

  • Asic Errors, USB Woes, and !winning!

    James Wynhoff03/28/2018 at 16:09 0 comments

    I think the next thing I'm going to work on will be SPI / netboot

    I'm going to start with a newly compiled uboot on the SD card to get netbooting working, then I will transfer that working uboot to the SPI flash, cant wait to be done flashing and reading sd cards all day.

    The results of that are this correctly configured fex though (opi one), use fex2bin and replace /boot/script.bin

    this gets rid of the asic CPU set errors, but it also sets the CPU to a constant speed (which is ok with me), and fixes USB issues

    --------------UPDATE-----------------

    I was just told that the site malformed this file, DO NOT COPY/PASTE THIS

    I will post the file in the files section

    [product]
    version = "100"
    machine = "Xunlong Orange Pi One"
    
    [platform]
    debug_mode = 1
    eraseflag = 1
    next_work = 2
    
    [target]
    boot_clock = 1008
    storage_type = -1
    
    [key_detect_en]
    keyen_flag = 0
    
    [fel_key]
    fel_key_max = 7
    fel_key_min = 2
    
    [card_boot]
    logical_start = 40960
    sprite_work_delay = 500
    sprite_err_delay = 200
    sprite_gpio0 = port:PL10<1><default><default><default>
    next_work = 3
    
    [box_start_os]
    used = 1
    start_type = 1
    irkey_used = 1
    pmukey_used = 1
    pmukey_num = 3
    led_power = 0
    led_state = 0
    
    [boot_init_gpio]
    used = 1
    gpio0 = port:PL10<1><default><default><1>
    gpio1 = port:PG11<1><default><default><1>
    
    [recovery_para]
    used = 1
    mode = 2
    recovery_key = port:PL04<0><default><default><default>
    
    [pm_para]
    standby_mode = 1
    
    [card0_boot_para]
    card_ctrl = 0
    card_high_speed = 1
    card_line = 4
    sdc_d1 = port:PF00<2><1><2><default>
    sdc_d0 = port:PF01<2><1><2><default>
    sdc_clk = port:PF02<2><1><2><default>
    sdc_cmd = port:PF03<2><1><2><default>
    sdc_d3 = port:PF04<2><1><2><default>
    sdc_d2 = port:PF05<2><1><2><default>
    
    [card2_boot_para]
    card_ctrl = 2
    card_high_speed = 1
    card_line = 8
    sdc_cmd = port:PC06<3><1><2><default>
    sdc_clk = port:PC05<3><1><2><default>
    sdc_d0 = port:PC08<3><1><2><default>
    sdc_d1 = port:PC09<3><1><2><default>
    sdc_d2 = port:PC10<3><1><2><default>
    sdc_d3 = port:PC11<3><1><2><default>
    sdc_d4 = port:PC12<3><1><2><default>
    sdc_d5 = port:PC13<3><1><2><default>
    sdc_d6 = port:PC14<3><1><2><default>
    sdc_d7 = port:PC15<3><1><2><default>
    sdc_2xmode = 1
    sdc_ddrmode = 1
    
    [twi_para]
    twi_port = 0
    twi_scl = port:PA11<2><default><default><default>
    twi_sda = port:PA12<2><default><default><default>
    
    [uart_para]
    uart_debug_port = 0
    uart_debug_tx = port:PA04<2><1><default><default>
    uart_debug_rx = port:PA05<2><1><default><default>
    
    [force_uart_para]
    force_uart_port = 0
    force_uart_tx = port:PF02<3><1><default><default>
    force_uart_rx = port:PF04<3><1><default><default>
    
    [jtag_para]
    jtag_enable = 0
    jtag_ms = port:PA00<3><default><default><default>
    jtag_ck = port:PA01<3><default><default><default>
    jtag_do = port:PA02<3><default><default><default>
    jtag_di = port:PA03<3><default><default><default>
    
    [clock]
    pll_video = 297
    pll_ve = 402
    pll_periph0 = 600
    pll_gpu = 576
    pll_periph1 = 600
    pll_de = 864
    
    [dram_para]
    dram_clk = 624
    dram_type = 3
    dram_zq = 0x3b3bfb
    dram_odt_en = 1
    dram_para1 = 283377664
    dram_para2 = 0
    dram_mr0 = 6208
    dram_mr1 = 64
    dram_mr2 = 24
    dram_mr3 = 2
    dram_tpr0 = 0x48a192
    dram_tpr1 = 0x1c2418d
    dram_tpr2 = 0x76051
    dram_tpr3 = 0x0
    dram_tpr4 = 0x0
    dram_tpr5 = 0x0
    dram_tpr6 = 0x64
    dram_tpr7 = 0x0
    dram_tpr8 = 0x0
    dram_tpr9 = 0x0
    dram_tpr10 = 0x0
    dram_tpr11 = 0x6aaa0000
    dram_tpr12 = 0x7979
    dram_tpr13 = 0x800800
    
    [wakeup_src_para]
    cpu_en = 0
    cpu_freq = 48
    pll_ratio = 273
    dram_selfresh_en = 1
    dram_freq = 36
    wakeup_src0 =
    wakeup_src_wl = port:PG10<4><default><default><0>
    wakeup_src_bt = port:PL03<6><default><default><0>
    
    [twi0]
    twi_used = 1
    twi_scl = port:PA11<2><default><default><default>
    twi_sda = port:PA12<2><default><default><default>
    
    [twi1]
    twi_used = 1
    twi_scl = port:PA18<3><default><default><default>
    twi_sda = port:PA19<3><default><default><default>
    
    [twi2]
    twi_used = 0
    twi_scl = port:PE12<3><default><default><default>
    twi_sda = port:PE13<3><default><default><default>
    
    [uart0]
    uart_used = 1
    uart_port = 0
    uart_type = 2
    uart_tx = port:PA04<2><1><default><default>
    uart_rx = port:PA05<...
    Read more »

  • I need a beer (or 6)

    James Wynhoff03/27/2018 at 06:54 0 comments

    So, after bashing my head against the wall for a long time I finally got the camera working, I had to start with a 5.20 image because hte fex in later build is wrong.....

    Ill be posting the image later, it will save you much much trouble....

  • Soup is done!

    James Wynhoff03/27/2018 at 00:21 0 comments

    That's a lie.

    I came back and realized I didnt add  linux-headers-sun8i to my first apt-get, as they are not installed by default.

    Edit:

    4 hours later, still futzing to get it to load the drivers and detect the CSI camera.......

    Sigh

  • Lunch time

    James Wynhoff03/26/2018 at 19:23 0 comments

    The first few packages, ran this in a file named step1.sh:

    sudo apt-get update -y
    sudo apt-get upgrade -y
    sudo apt-get install python-dev python-pip python-opencv libv4l-dev -y
    git clone https://github.com/umlaeute/v4l2loopback
    cd v4l2loopback/
    make && make install
    cd ..
    git clone https://github.com/lhelontra/vidcopy
    cd vidcopy/
    gcc vidcopy.c -o vidcopy
    cd ..
    echo "v4l2loopback" | sudo tee -a /etc/modules
    echo "gc2035" | sudo tee -a /etc/modules
    echo "vfe_v4l2" | sudo tee -a /etc/modules

     That will take a while, time for lunch and to start the the next project!

  • Let there be sound?

    James Wynhoff03/26/2018 at 18:45 0 comments


    root@orangepione:~# speaker-test speaker-test 1.0.28 Playback device is default Stream parameters are 48000Hz, S16_LE, 1 channels Using 16 octaves of pink noise Rate set to 48000Hz (requested 48000Hz) Buffer size range from 256 to 524288 Period size range from 128 to 65536 Using max buffer size 524288 Periods = 4 was set period_size = 65536 was set buffer_size = 524288 0 - Front Left

    Oops, looks like I flipped the speakers though!

  • Armbian, and the need for speed

    James Wynhoff03/26/2018 at 18:32 0 comments

    First thing was first, I had done some research on reading temp sensors and they fail at 408mhz, which is the "slow" speed, this board doesn't have a proper voltage selectable voltage regulator, just a VR that is settable via resistors, and that's 1.2ghz or 408mhz

    sudo nano /etc/default/cpufrequtils

    ENABLE=true
    MIN_SPEED=1200000
    MAX_SPEED=1200000
    GOVERNOR=performance
    

    sudo watch -n 1  cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_cur_freq

    Every 1.0s: cat /sys/devices/system/cpu/cpu0/cpufre...  Mon Mar 26 20:31:38 2018
    
    1200000
    1200000
    1200000
    1200000
    

    Much better :)

  • WHY?!?

    James Wynhoff03/26/2018 at 17:47 0 comments

    Why is it that yesterday, when I didn't need them, I came across at least 3 usb > 3.3v serial adapters... now that I need ONE to debug this I cant find any.....


    Prop plug to the rescue!

    U-Boot 2017.09-armbian (Nov 22 2017 - 16:47:31 +0100) Allwinner Technology
    
    CPU:   Allwinner H3 (SUN8I 1680)
    Model: Xunlong Orange Pi One
    DRAM:  512 MiB
    MMC:   SUNXI SD/MMC: 0
    *** Warning - bad CRC, using default environment
    
    In:    serial
    Out:   vidconsole
    Err:   vidconsole
    Net:   phy interface0
    eth0: ethernet@1c30000
    230454 bytes read in 195 ms (1.1 MiB/s)
    starting USB...
    USB0:   USB EHCI 1.00
    USB1:   USB OHCI 1.0
    scanning bus 0 for devices... 1 USB Device(s) found
    scanning bus 1 for devices... 2 USB Device(s) found
           scanning usb for storage devices... 0 Storage Device(s) found
    Autoboot in 1 seconds, press  to stop
    =>
    
    U-Boot SPL 2017.09-armbian (Nov 22 2017 - 16:47:31)
    DRAM: 512 MiB
    Trying to boot from MMC1
    
    
    U-Boot 2017.09-armbian (Nov 22 2017 - 16:47:31 +0100) Allwinner Technology
    
    CPU:   Allwinner H3 (SUN8I 1680)
    Model: Xunlong Orange Pi One
    DRAM:  512 MiB
    ▒
    U-Boot SPL 2017.09-armbian (Nov 22 2017 - 16:47:31)
    DRAM: 512 MiB
    Trying to boot from MMC1
    
    
    U-Boot 2017.09-armbian (Nov 22 2017 - 16:47:31 +0100) Allwinner Technology
    
    CPU:   Allwinner H3 (SUN8I 1680)
    Model: Xunlong Orange Pi One
    DRAM:  512 MiB
    MMC:   SUNXI SD/MMC: 0
    *** Warning - bad CRC, using default environment
    
    In:    serial
    Out:   vidconsole
    Err:   vidconsole
    Net:   phy interface0
    eth0: ethernet@1c30000
    230454 bytes read in 194 ms (1.1 MiB/s)
    starting USB...
    USB0:   USB EHCI 1.00
    USB1:   USB OHCI 1.0
    scanning bus 0 for devices... 1 USB Device(s) found
    scanning bus 1 for devices... 2 USB Device(s) found
           scanning usb for storage devices... 0 Storage Device(s) found
    Autoboot in 1 seconds, press  to stop
    switch to partitions #0, OK
    mmc0 is current device
    Scanning mmc 0:1...
    Found U-Boot script /boot/boot.scr
    3708 bytes read in 198 ms (17.6 KiB/s)
    ## Executing script at 43100000
    U-boot loaded from SD
    Boot script loaded from mmc
    115 bytes read in 150 ms (0 Bytes/s)
    3812646 bytes read in 447 ms (8.1 MiB/s)
    4751880 bytes read in 509 ms (8.9 MiB/s)
    Found legacy kernel configuration
    34704 bytes read in 494 ms (68.4 KiB/s)
    ## Loading init Ramdisk from Legacy Image at 43300000 ...
       Image Name:   uInitrd
       Image Type:   ARM Linux RAMDisk Image (gzip compressed)
       Data Size:    3812582 Bytes = 3.6 MiB
       Load Address: 00000000
       Entry Point:  00000000
       Verifying Checksum ... OK
    
    Starting kernel ...
    
    [sun8i_fixup]: From boot, get meminfo:
            Start:  0x40000000
            Size:   512MB
    ion_carveout reserve: 160m@0 256m@0 130m@1 200m@1
    ion_reserve_select: ion chipid  [0x2c00081!
    ion_reserve_common: ion reserve: [0x56000000, 0x60000000]!
    Loading, please wait...
          Begin: Loading essential drivers ... done.
    Begin: Running /scripts/init-premount ... done.
    Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
    Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
     done.
     Begin: Will now check root file system ... fsck from util-linux 2.25.2
    [/sbin/fsck.ext4 (1) -- /dev/mmcblk0p1] fsck.ext4 -a -C0 /dev/mmcblk0p1
    /dev/mmcblk0p1: clean, 34994/72720 files, 199522/290816 blocks
     done.
    done.
    Begin: Running /scripts/local-bottom ... done.
    Begin: Running /scripts/init-bottom ...   done.
    
    Welcome to Debian GNU/Linux 8 (jessie)!
    
              Expecting device dev-ttyS0.device...
    [  OK  ] Reached target Remote File Systems (Pre).
    [  OK  ] Reached target Encrypted Volumes.
    [  OK  ] Reached target Paths.
    [  OK  ] Set up automount Arbitrary Executable File Formats F...utomount Point.
    [  OK  ] Reached target Swap.
    [  OK  ] Created slice Root Slice.
    [  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
    [  OK  ] Listening on Delayed Shutdown Socket.
    [  OK  ] Listening on Journal Socket (/dev/log).
    [  OK  ] Created slice User and Session Slice.
    [  OK  ] Listening on udev Control Socket.
    [  OK  ] Listening on udev Kernel Socket.
    [  OK  ] Listening on Journal Socket.
    [  OK  ] Created slice System Slice.
             Starting Increase datagram queue length...
     Starting Restore / save the...
    Read more »

  • Not that easy, OFC

    James Wynhoff03/26/2018 at 17:43 0 comments

    So as expected, NOT that easy, its boot looping, mashing space bar got the boot to stop

    -------ALSO-------

    An annoyance with documentation, taking pictures of stuff stuff when its dirty, I'm going to be forced to clean everything during this....

  • OS Image

    James Wynhoff03/26/2018 at 17:04 0 comments

    Ok, im going to start with this image and then explore more from there.....

    https://dl.armbian.com/orangepione/Debian_jessie_default.7z.torrent

    Should be easy... Right?

View all 13 project logs

  • 1
    Enabling Camera and hardware h264 encoding

    From a 5.35 raspbian image for the orange pi one

    sudo apt-get install libgd2-xpm-dev libjpeg-dev libpng-dev libva-dev libv4l-dev libpulse-dev libx264-dev libmp3lame-dev libfreetype6-dev -y
    
    git clone https://github.com/avafinger/ffmpeg_cedrus264_H3_Armbian_5.23.git
    sudo dpkg -i ./ffmpeg_cedrus264_H3_Armbian_5.23/ffmpeg-3.1.4_1.0-3.deb
    
    modprobe -r -v vfe_v4l2
    
    modprobe -r -v gc2035
    
    modprobe gc2035 
    
    modprobe vfe_v4l2
    
    echo "gc2035" | sudo tee -a /etc/modules
    echo "vfe_v4l2" | sudo tee -a /etc/modules
    

    now we can start playing with the webcam in a meaningful way

    sudo ffmpeg-3.1.4 -f v4l2 -channel 0 -video_size 1920x1080 -i /dev/video0 -pix_fmt nv12 -r 22 -c:v cedrus264 night_video_test4_1920x1080.mp4

View all instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates