The hairball is alive

A project log for RPi WiFi

Fast RPi WiFi without USB

ajlittajlitt 01/10/2016 at 05:1837 Comments

The mess above is an off-the-shelf ESP-03 module connected to the Pi much like my very first attempt. What's different is that in this case the SPI flash has not been removed, and not all SDIO pins being used. This has been clocked at ~20Mbits/s to a nearby router.

What makes this interesting? It proves that there is a way to connect some unmodified ESP8266 modules to the Pi. It also proves that 1 bit SDIO at 50MHz does not throttle the WiFi connection beyond usability. Most importantly, it means that anyone is free to make and sell a Pi WiFi board using the ESP-12E and -12F modules without going through FCC/CE certification.

There were two problems: a bug in the SD host driver for the Pi's processor that prevents forcing 1 bit SD mode, and a problem where tying /HOLD to ground to keep the SPI chip inactive was causing the associated ESP signal (SD_D2) to set some sort of strap value that gets latched at reset and seems to affect the loading of the WiFi firmware.


jacksonliam wrote 01/17/2016 at 16:35 point

Hi @ajlitt I was wondering if you can help me out. I've hooked up the ESP-12F CMD/CLK/D0/D1 lines to the raspberry pi GPIO pins and D2 is connected to CMD.

I rebuilt the kernel (man, cloning, cross-compiling and installing that kernel takes half a day!) Built and installed the esp8089 module. 

I flashed new FW to the esp module with the dio bit set. But I'm not sure if that clears the QE bit in the flash or just tells the ESP not to use those instructions. I've tried to hold CH_EN and bit bang SPI to read the status register from the flash (with HOLD NC) but it doesn't seem to respond. 

Right now the card appears/dissapears in dmesg as i toggle CH_EN with no errors or further info. No driver gets loaded. I'm not sure where to debug next as I don't have a scope. Is this symptomatic of the flash still getting in the way (e.g. QE bit still set)?

[  410.018091] mmc1: card 0001 removed

[  415.172592] mmc1: queuing unknown CIS tuple 0x01 (3 bytes)
[  415.181865] mmc1: queuing unknown CIS tuple 0x1a (5 bytes)
[  415.185147] mmc1: queuing unknown CIS tuple 0x1b (8 bytes)
[  415.189556] mmc1: queuing unknown CIS tuple 0x80 (1 bytes)
[  415.189675] mmc1: queuing unknown CIS tuple 0x81 (1 bytes)
[  415.189790] mmc1: queuing unknown CIS tuple 0x82 (1 bytes)
[  415.189853] mmc1: new high speed SDIO card at address 0001

pi@raspberrypi:~ $ sudo cat /sys/kernel/debug/mmc1/ios
clock:          50000000 Hz
actual clock:   41666666 Hz
vdd:            21 (3.3 ~ 3.4 V)
bus mode:       2 (push-pull)
chip select:    0 (don't care)
power mode:     2 (on)
bus width:      0 (1 bits)
timing spec:    2 (sd high-speed)
signal voltage: 0 (3.30 V)

pi@raspberrypi:~ $ cat /sys/bus/sdio/devices/mmc1\:0001\:1/device
pi@raspberrypi:~ $ cat /sys/bus/sdio/devices/mmc1\:0001\:1/vendor

pi@raspberrypi:~ $ cat /sys/bus/sdio/devices/mmc1\:0001\:1/modalias

  Are you sure? yes | no

jacksonliam wrote 01/17/2016 at 21:31 point

I think I had MISO/MOSI on the module (D1/D0) hooked up reversed. It does make me question if the other pins on the module are questionably labeled. Exactly the same output with them swapped though.

  Are you sure? yes | no

duke wrote 01/17/2016 at 22:03 point

This is what I found when trying to access the spi flash on an ESP-12E (seems to have same pin layout as 12F).
Pin 9 CS flashchip
Pin 10 MISO
And PIN 14 seems to be /HOLD (not tested).   I can access the spi flash when keeping RESET low to avoid conflicts. Dunno if that helps you (it should for the bitbang test). Not using a Pi.

  Are you sure? yes | no

jacksonliam wrote 01/17/2016 at 22:47 point

@duke Is that's what marked on the silkscreen? The 12F has pin 14 marked as clk.

Since swapping MISO/MOSI I seem to be getting somewhere bit banging the flash, ID from 'wake up' 0xAB instruction repeats 0x2A after 3 dummy bytes. Not sure about the ID but the behaviour is as per datasheet.

Status reg 1 returns 0, status reg 2 returns 0x04. Think those are sane values?

Ill try swapping CLK tomorrow though, can't hurt to try! 

  Are you sure? yes | no

duke wrote 01/17/2016 at 22:51 point

Yes unlike the wiki, PIN 12 is CLK of SPI flash and PIN 14 is connected to hold of the spi flash. Atleast I can read/write/id the flash using that from an ARM MCU :)

  Are you sure? yes | no

David Lowe wrote 01/17/2016 at 22:58 point

I've been trying to make sense of  the labeling on the 12E/F. I'm waiting on modules to arrive, but it seems to me that pins 9-14 map out to SDIO_CMD, SDIO_D0, SDIO_D2, SDIO_D3, SDIO_D1, SDIO_CLK in that order. That's after much consideration, confusion & at times despair, so I may stand to be corrected!

  Are you sure? yes | no

ajlitt wrote 01/18/2016 at 02:43 point

@jacksonliam: The SDIO ID process only relies on SD_CLK and SD_CMD to be functional, so likely the SPI flash still has the QE bit is set, in which case /HOLD is ignored.  That means SD_D0 clashes with the SPI's MISO pin and and non-command traffic gets stomped.

@duke: Are you sure about your pinout?  There are some people who have reworked the ESP-12E to bring GPIO9 and GPIO10 out to those pads by lifting the /HOLD and /WP pins on the SPI and tying them high.  That wouldn't be possible if the PCB were wired the way you suggest.

  Are you sure? yes | no

ajlitt wrote 01/18/2016 at 03:00 point

@duke:  Looking at forums, it seems there is a questionably "fake" ESP-12E variant that has a significantly different pinout from other ESP-12E: 

ESP-12F and the ESP-12Es from Adafruit supposedly have the correct pinout with GPIO9/GPIO10 adjacent in the center of the bottom edge.

  Are you sure? yes | no

duke wrote 01/18/2016 at 07:47 point

@ajlitt  ahh thanks for the link. Yes the ones I got look like the "fake", that explains a lot then!

  Are you sure? yes | no

jacksonliam wrote 01/18/2016 at 20:33 point

OK so the definitive pinout of the ESP-12F is:

[Updated due to below comments]

CMD (CS), D0 (MISO), D2 (HOLD), D3 (WP), D1 (MOSI), CLK. 

Its in the pcb layout in the docx datasheet attached here

And I've managed to read/clear/verify the QE bit. It wasn't working previously because I was reading bits out of sync with the clock while bit banging. Now I can connect the HOLD pin to CS and see the flash stop responding. Feel free to take my python!

However, I still don't seem to be able to comminicate with the module over SDIO. dmesg gets no further than above. I've tried rebooting.

I'm pretty sure my wiring to the pi is correct, CMD->23, D1->25, D3->23, D2->NC, D0->24, CLK->22. I'm wondering if my dupoint cables are a bit long? anyone know how to step down to 25MHz mode?

  Are you sure? yes | no

David Lowe wrote 01/18/2016 at 21:11 point

Are you sure about the 12F pinout, I see you have D0 & D1 swapped vs what I posted. My reference was the 8266 datasheet table 8:

  Are you sure? yes | no

ajlitt wrote 01/18/2016 at 21:18 point

Great job!  That's a great way to commission a new board.  You've also verified that I got the pinout right on the board I just sent to manufacture.

I think your pinout to the Pi is correct.  Did you pull /WP high on the module?  If /WP and /HOLD aren't both high when the ESP is powered up then the driver can't seem to complete the firmware load.

If you want to force the ESP into the lower speed mode, tie GPIO0 low.  That will put it in "lowspeed V1 I/O" mode which seems to convince the host to run the clock at 25MHz.  Smaller resistors may also help.

Edit: @David_Lowe is right about D0/D1: On the ESP side, MISO maps to D0 and MOSI maps to D1.  That's the first thing to try.

  Are you sure? yes | no

jacksonliam wrote 01/18/2016 at 21:47 point

Thanks @ajlitt and @David Lowe I was pretty sure I got D0/D1 right as I worked back from the winbond datasheet but I'll check!

Ill also try pulling WP high rather than leaving it floating! 


Hmm none of those seemed to help, would I see a message in dmesg if the card was working but the driver wasn't installed properly? 

  Are you sure? yes | no

jacksonliam wrote 01/19/2016 at 13:33 point

I'm confused because the W25Q32FV datasheet shows flash pin 5 as being DI (Data In, MOSI) but also shows its SDIO IO0. 

The PCB layout in the docx shows it connected to the modules pin 13, MOSI pin, but also to pin 23 on the Esp8266 SD_DATA_1.

  Are you sure? yes | no

ajlitt wrote 01/19/2016 at 15:21 point

@jacksonliam  Ah, there's the problem.  25-series SPI flash chips don't do SDIO at all.  The "IOx" alternate functions in the W25Q32 datasheet are for QIO/QSPI mode.  That is, the IOx pins become part of a bidirectional data bus in quad mode.  QSPI has a chip select (/CS) for framing and uses in-band control commands, while MMC and SDIO use a separate bidirectional command line (SD_CMD) for out-of-band control and data synchronization.

The ESP8266 has alternate functions for all of its pins, in this case the SPI master that's capable of booting when the boot mode strap = 011, and an SDIO slave when the boot mode strap = 1xx.  The alternate functions for SD_Dx and the SPI MISO/MOSI/WP/HOLD signals don't line up sequentially for QSPI.

You should go by the ESP8266 pin mapping spreadsheet in Espressif's docs section (warning Excel ahead): 

  Are you sure? yes | no

jacksonliam wrote 01/20/2016 at 21:29 point

Oh Yes! Thanks everyone I got a wlan0 interface, it was a kernel sources problem. I'll put the details in the instructions comments.

  Are you sure? yes | no

haddyhad wrote 01/13/2016 at 08:15 point

Is it possible to send the power down command to the flash chip instead of removing it and then use quad sdio mode? Or dual?

  Are you sure? yes | no

floe wrote 01/13/2016 at 09:38 point

Nice idea, but AFAICT, the flash will still react to a power-up instruction which might be sent accidentially during operation?

  Are you sure? yes | no

ajlitt wrote 01/13/2016 at 15:34 point

We don't care that the flash is "reacting".  At some points it may successfully latch the first data bit in the command word before /HOLD takes effect.  But with this scheme /HOLD asserts along with /CS so it can never get beyond that.

The only signal in single-bit SPI that can possibly drive an output is DO, and it never leaves tristate until after all 8 bits of a command word are successfully clocked in.  The /HOLD prevents this from happening.

(edit, read the comment you're replying to...) But you're right.  4-bit SDIO will drive patterns on all 6 signals that it shares with the SPI flash.  So even if a hypothetical wakeup sequence is nontrivial it's possible that normal SDIO traffic could wake the part up.

That said, the WiFi end-to-end throughput at 1-bit-SDIO doesn't seem to be that bad, considering the Pi Zero doesn't have much grunt to start with.  The numbers I gave come from my board with the ESP-03 module, which had questionable connectivity in my first prototype.  I need to redo the throughput tests with one of my custom boards running in 1-bit mode to rule that out.

  Are you sure? yes | no

haddyhad wrote 01/14/2016 at 02:28 point

To release the device from the power-down state, the instruction is issued by driving the /CS pin low, shifting the instruction code "ABh" and driving /CS high as shown in Figure 38a & 38b. Release from power-down will take the time duration of tRES1 (See AC Characteristics) before the device will resume normal operation and other instructions are accepted. The /CS pin must remain high during the tRES1 time duration.

I am not sure this pattern could occur accidentally so seems like it is worth investigating.

  Are you sure? yes | no

floe wrote 01/12/2016 at 19:26 point

Just to clarify, if you're able to physically disconnect the SPI flash on whatever ESP module you're using, then you can also use the (faster) 4-bit bus mode?

  Are you sure? yes | no

ajlitt wrote 01/12/2016 at 22:21 point

That's correct.

Let's say you built a board with the ESP-12E and routed SD_D2 and SD_D3 from the Pi to the external GPIO9 and GPIO10 pads.  You could do 4-bit SDIO if you cracked the lid and desoldered the SPI flash.

  Are you sure? yes | no

floe wrote 01/13/2016 at 08:57 point

OK, thanks for clearing that up. I guess I have to do a bit of catching up on SPI, is there a reference schematic somewhere for the internals of the ESP-12E?

  Are you sure? yes | no

deʃhipu wrote 01/13/2016 at 09:01 point

I wonder if you could use a "high current short removal method" to, um, "disable" the flash chip without desoldering it... ;)

  Are you sure? yes | no

nistvan.86 wrote 01/11/2016 at 23:28 point

Awesome! I assume this should work with the earliest ESP-01 model as well. I have a spare one which I'm not using. Oh well, i don't have a Pi0 though to test with yet  :)

Waiting for the instructions to include the keep-the-flash-intact method :) Really good job, I do hope some nice RPi0 HATs will show up soon.

  Are you sure? yes | no

ajlitt wrote 01/12/2016 at 22:19 point

Maybe, but it would be difficult.  For this to work, the boot mode straps (MTDO, GPIO2, GPIO0) must be left floating high.  But on the ESP-01 MTDO is shorted to ground by a trace going to the ground slug underneath the chip.  You would have to desolder the ESP8266, cut the trace, and solder it back for this to work on an ESP-01.

ESP-12Es and ESP-12Fs are pretty cheap direct from China though.

  Are you sure? yes | no

nistvan.86 wrote 01/13/2016 at 16:20 point

I see, thank you for the answer! I'm not in hurry anyway since I'm waiting for the RPi0 to be purchasable again. :)

  Are you sure? yes | no

jacksonliam wrote 01/11/2016 at 10:56 point

This is very cool, my ESP-12F should arrive this week so looking forward to trying out 1 bit mode with an FCC approved module. 

What do you do with the HOLD pin now if not tying to GND?

What did you do to fix the driver? 

  Are you sure? yes | no

davedarko wrote 01/11/2016 at 12:30 point

side question - where is the difference between 12e and 12f?

  Are you sure? yes | no

ajlitt wrote 01/11/2016 at 14:55 point

12E is a 2 layer PCB and has (what looks to my untrained eye like) a suboptimal folded inverted F antenna.  12F is a 4 layer PCB with a folded F antenna that takes better advantage of the PCB area and has much more ground plane stitching.  Both modules have passed FCC.  -12E and -12 are under the same ID (2ADUIESP-12), while -12F has a new one (2ADUIESP-12-F).

  Are you sure? yes | no

ajlitt wrote 01/11/2016 at 14:48 point

Thanks!  I have an ESP-12E and few ESP-12Fs on the way too.

The solution was surprisingly simple: connect together /HOLD and /CS.  For reference /CS is mux'd with SD_CMD.  That way /HOLD is nominally inactive high when the bus is inactive, but gets driven active low when the SPI flash might accidentally be convinced to take DO out of tristate.

I don't know how long the hold period is for the ESP's latching of strap bits, so there's a nonzero chance the ESP will be put in a bad state if the module is powered up or enabled while the host is probing.

The ESP and the Pi both advertise 4-bit SDIO through the handshake that happens over the SD_CMD channel.  Unlike buses like PCIe which can detect and automatically degrade lanes if they're not physically present, the MMC/SD standard assumes that the devices are honest about the signals they support, and the Linux driver will set the MMC host for the least common denominator bus width.  And because SD/MMC doesn't do in-band control and error checking of data, running in 4 bit mode when not all 4 bits are wired will silently corrupt the data passed between the ESP and the driver.

I added a parameter to the SDIO overlay that lets you specify the host bus width capability, overriding the default of 4 bits.  I pushed a patch to the Pi kernel github rpi-4.1.y branch last night.  Unfortunately that means you'll need to rebuild the kernel until this is approved and the firmware repo is updated with a new kernel binary.

  Are you sure? yes | no

jacksonliam wrote 01/11/2016 at 17:23 point

Ah that's fair enough, I can see why the Pi and ESP would both advertise 4 bit mode. Tying to CS is a neat solution!

That will be a useful patch, hopefully it will get accepted quickly! 

I thought you needed the 4.2y branch for this SDIO stuff? 

  Are you sure? yes | no

ajlitt wrote 01/11/2016 at 21:24 point

That was what I thought when I started on this.  pboettch's patch that added repetitive polling for devices in the SDIO overlay went into 4.2.y first, so that's the kernel I did my testing with.  I had assumed (incorrectly) that there were other changes to 4.2.y over 4.1.y that affected the SDIO driver (not true).  And at that time I was squashing kernel API mismatches in the driver so I didn't want a moving target.

By the time I had something working and posted here, the poll_once patch made it into 4.1.y and from there into the firmware repo.  Hopefully my patches will make it in time for the next churn of the firmware.

  Are you sure? yes | no

deʃhipu wrote 01/10/2016 at 10:44 point

This is awesome! So what was wrong with the previous attempt?

  Are you sure? yes | no

ajlitt wrote 01/11/2016 at 21:33 point

Turns out that 3 of the 4 SD data lines are also straps that are latched at POR by the ESP.  I'm not sure what they do, but these feed into the "y" in "boot mode: (x,y)" that the ESP ROM prints at power on.  The signal shared with the SPI flash /HOLD pin throws the ESP into an unhappy place if I just tie it low.  The pinlist spreadsheet alludes to existence of straps but doesn't elaborate.

So the fix as I mentioned above was to tie the SPI flash's /CS and /HOLD together since /HOLD is don't care when /CS is inactive high.  /CS is shared with SD_CMD which is driven by the Pi when polling for devices, so there's the possibility of a race between the ESP ROM starting  (by legit POR or by CHIP_EN going high) and the Pi issuing a polling cycle.  Since we have control over CHIP_EN, I'll eventually let the driver take control of that and jog power if the firmware download fails.  The Pi is much slower to start than the ESP, so I don't think we'll hit a race condition on POR.

Also the kernel MMC host driver REALLY wanted to use 4-bit bus mode despite being told to stick to 1 bit by the devicetree overlay.

  Are you sure? yes | no

duke wrote 01/13/2016 at 23:11 point

Nice work!
- Which pin on the 12E is /HOLD and is CSO (pin 9) CS of the spi flash?

Edit: Nevermind, just found a drawing of the ESP-12E, pin 9 is CS of flashchip and /HOLD seems to pin 14.

  Are you sure? yes | no