Project Logs

Collapse logs

• The PHY reset saga

  Patrick Van Oosterwijck • 03/11/2019 at 16:09 • 0 comments

  Finally, a long overdue update! The silence doesn’t mean we’ve been taking it easy, on the contrary: we’ve been hard at work to make the wESP32 even better!

  A yield problem

  After the initial build of 245 units, we were able to ship 205 units that were good right away. There were various issues with the remaining 40, but the vast majority of them had the same problem: the Ethernet PHY would not get initialized properly.

  The LAN8720A Ethernet PHY is kind of a difficult part. It has several pins that have double use: during reset their logic level indicates various configuration settings, while after reset they are used in the RMII interface to communicate with the ESP32. An extra difficulty is that the chip expects the 50 MHz clock to be present during reset, so the chip can clock in these configuration settings.

  Unfortunately, the ESP32 by default uses the IO0 pin for the Ethernet MAC clock. (More options have become available for dealing with the Ethernet MAC and PHY clocks, but because IO0 was the first option available in the SDK, it enjoys the broadest firmware support so that’s why I keep using it. Plus to keep as many other ESP32 pins as possible available to the user!) The ESP32 IO0 pin is kind of a difficult pin as well, since it is used on reset to determine whether the ESP32 should go into programming mode or run the current firmware. So we have a perfect storm of an IO0 pin that needs to be high at boot to run normally, but low at boot to go into programming mode, and needs to have the 50 MHz clock on it after boot to allow Ethernet communication, and that before the PHY comes out of reset.

  When I designed the wESP32, I just looked at how Olimex dealt with the IO0, ESP32 reset and PHY reset on their ESP32-GATEWAY and copied that for the wESP32, figuring they must already have found a good solution since this was a product in production. On my first hand-built prototypes, it all worked well, so I went ahead with it in production. Then I found that more than 10% of the boards had an issue. Interestingly, in their latest revisions, Olimex has changed their product to use IO17 as PHY clock output to bypass the whole problem! This might be a smart thing to do, but I am loath to make backwards incompatible changes and take a precious GPIO pin away from the user. So I started looking at other ways to deal with this.

  Looking for a solution

  I needed a solution that would act as a reset manager for the 50 MHz oscillator and Ethernet PHY and would be triggered by the EN signal to the ESP32. When EN is low on power-up and when triggered by a programmer, the 50 MHz oscillator needs to be off, so the programmer can drive the IO0 pin low to enter programming mode, or the IO0 can stay pulled high and the system boots. The PHY needs to be kept in reset as well, because its reset can only be released after the 50 MHz clock has started up. After the ESP32 has had the opportunity to determine the boot state, the 50 MHz clock needs to be turned on, and only after the clock signal is present can the PHY reset be released.

  I had assumed there would be plenty of low cost, SOT packaged voltage detectors and reset managers that could replace my existing solution in the space I had available, but to my great surprise, nothing I could find in the market really seemed to fit the bill. There are plenty of supervisors that monitor multiple voltages and generate a reset, but I couldn’t find any low cost options that monitor one input and generate two reset signals with separate delays.

  So I started thinking outside the box: did I really need to add a little microcontroller to do this? There are several microcontrollers that are cheaper than higher end supervisor chips, and it would give me full flexibility to generate the reset signals exactly how I wanted them. On the other hand, I didn’t really want to deal with programming this micro on every board. I would need to add programming pads and a manufacturing...

  Read more »

• Don't blow up your Ethernet phy!

  Patrick Van Oosterwijck • 01/16/2019 at 16:29 • 0 comments

  You may have noticed a warning at the bottom of the MongooseOS demo in the demo code repo, about accidentally blowing up the Ethernet phy while playing with MongooseOS demo code.

  I didn’t find any time to pursue this further, but I have to give the people at MongooseOS credit for getting to the bottom of this! I received this message from them:

  We did some research, and found the problem with ethernet. It happened to be quite stupid: the default config for the built-in LED used pin 22, which conflicts with the ethernet TX.
  So, this command should set the board correctly:
  mos flash esp32
  mos config-set eth.enable=true eth.mdc_gpio=16 eth.mdio_gpio=17 board.led1.pin=-1 board.btn1.pin=-1
  

  This highlights an issue that can happen with other firmwares as well: misconfigured pins doing damage to the phy. By default, the wESP32 ships with MicroPython installed and it is configured correctly out-of-the-box for the Ethernet phy as it is configured on the wESP32. If you load different firmware though, it is your responsibility to ensure the ESP32 pins are configured correctly and will for instance not configure an ESP32 pin as an output that connects to a phy output, and thus cause a short. Please refer to the schematic to make sure. The wESP32 website section on Software provides more information on wESP32 support in different environments.

  If you have ordered one or more wESP32's, I recommend you try it with the default MicroPython firmware when you first receive it, to confirm the hardware is sound and it receives an IP address (check with lan.ifconfig()) from DHCP over Ethernet, which only happens if the phy is working. All units are tested to do this before they are shipped, but this way you can prove to yourself the hardware is OK and nothing bad happened during shipping. After that you are free to use whatever firmware you want of course, but I will not replace defective units once they have been flashed with different firmware, as I cannot know what happened to them.  Thanks for understanding!
  

• We are shipping

  Patrick Van Oosterwijck • 01/16/2019 at 16:24 • 0 comments

  On Monday 1/7, we finally received completed panels from Colorado Tech Shop!

  So we spent most of this week doing final assembly (installation of two through-hole electrolytic capacitors and the Ethernet jacks) and testing for all 245 boards in the first batch. I am quite pleased with the result, I hope you backers will be happy as well!

  Some boards were used for internal testing, some had to be sent back to Colorado Tech Shop for rework after finding issues during test, but today we were able to ship the first 205 units to Crowd Supply for delivery to backers next week.
  

  One of the boards was used for an experiment. I wanted to see if the board worked with the AI-Thinker ESP32-S module instead of the ESP32-WROOM-32:

  I’m happy to report it seems to work just the same, so this module could be used to provide a future U.FL antenna connector option!

• Production update, and testers!

  Patrick Van Oosterwijck • 12/30/2018 at 06:39 • 0 comments

  Last week it dawned on me that time keeps passing and I hadn't given much thought yet to how I was going to test the wESP32 and wESP32-Prog boards before we ship them!  A very important issue to start cranking on, especially with holidays and vacation eating up a good amount of time.

  wESP32-Prog tester
  

  The wESP32-Prog board is quite simple and because of this, the tester can be simple as well.  Building the tester only took half a day and this is what I ended up with:

  The hardware consists of a little STM8S103F3 board that can be had for less than a dollar on AliExpress.  I had it laying around and all I needed for this was a simple UART capable chip, so this was perfect.  I used the Sduino project project to speed up firmware development.   It looks slightly different from normal Arduino code because the SDCC compiler doesn't support C++, only C.  It was simple enough to get it to work though, resulting in this code:
  

  // wESP32-Prog tester using STM8S103 board and sDuino
  
  #define IO0   PA1
  #define EN    PA2
  
  void setup() {
    // Initialize digital pin LED_BUILTIN as an output.
    pinMode(LED_BUILTIN, OUTPUT);
    digitalWrite(LED_BUILTIN, 1);
    // Initialize EN and IO0 test pins as input with pullup
    pinMode(IO0, INPUT_PULLUP);
    pinMode(EN, INPUT_PULLUP);
      // Init serial port to 115200 bps
    Serial_begin(115200);
  }
  
  void loop() {
    // Read a byte from serial port
    int c = Serial_read();
    // Did we get a byte?
    if (c >= 0) {
      // Set lowest bit to value of IO0
      c = (c & 0xFE) | (digitalRead(IO0) ? 0x01 : 0);
      // Set second lowest bit to value of EN
      c = (c & 0xFD) | (digitalRead(EN) ? 0x02 : 0);
      // Echo byte back with altered IO0 and EN bits
      Serial_write(c);
      // Set the green LED state according to the second highest bit
      digitalWrite(LED_BUILTIN, (c & 0x40) ? 0 : 1);
    }
  }

   All this does is when a byte comes in, echo it back with the bottom two bits replaced by the current state of the IO0 and EN signals.  The second to highest bit also sets the tester board's on-board LED state.

  This firmware works in conjunction with a little Python script on my Linux box:
  

  #!/usr/bin/env python3
  
  import os
  import glob
  import serial
  import time
  from termcolor import colored
  
  while True:
  
    print("Waiting for wESP32-Prog...")
      ports = []
    while not ports:
      ports = glob.glob('/dev/ttyUSB*')
      time.sleep(0.1)
  
    print("Serial port detected, waiting for access...")
      while (not os.access(ports[0], os.R_OK|os.W_OK)):
      time.sleep(0.1)
      print("Testing wESP32-Prog...")
      test_ok = True
    s = serial.Serial(port=ports[0], baudrate=115200, timeout=0.1)
  
    s.setRTS(False)
    s.setDTR(False)
    s.write(b'0')
    res = s.read(1)
    if (res not in [b'0', b'1', b'2', b'3']):
      print(colored("ERROR: Serial data did not echo correctly, is the tester connected?", 'red'))
      test_ok = False
    if (res != b'3'):
      print(colored("ERROR: IO0 and/or EN pulled low when neither is asserted!", 'red'))
      test_ok = False
  
    s.setRTS(True)
    s.write(b'0')
    if (s.read(1) != b'1'):
      print(colored("ERROR: EN not pulled low correctly!", 'red'))
      test_ok = False
  
    s.setDTR(True)
    s.write(b'0')
    if (s.read(1) != b'3'):
      print(colored("ERROR: IO0 and/or EN pulled low when both are asserted!", 'red'))
      test_ok = False
  
    s.setRTS(False)
    s.write(b'0')
    if (s.read(1) != b'2'):
      print(colored("ERROR: IO0 not pulled low correctly!", 'red'))
      test_ok = False
  
    if (test_ok):
      print(colored("OK! All tests passed.", 'green'))
      s.write(b'@')
  
    s.close()
    print("Please unplug wESP32-Prog.")
      while ports:
      ports = glob.glob('/dev/ttyUSB*')
      time.sleep(0.1)

   What this does is wait for the wESP32-Prog board to be plugged in, which will create a /dev/ttyUSB0 device.  For some reason, trying to immediately open the serial device after this threw an access error, so we wait until it becomes accessible and then open it.

  Testing ensures that when we send a byte, the STM8 micro echoes a (modified) byte back to us.  We do this with various states of the RTS and DTR signals which control...

  Read more »

• Crowd Supply campaign ending!

  Patrick Van Oosterwijck • 11/12/2018 at 17:51 • 0 comments

  The Crowd Supply campaign has been going very well and will be ending soon, so if you haven't put in an order yet and you want one, now is the time! :)

  I've been putting quite a bit of work into the software side of things during the campaign, to make it as easy as possible for anyone wanting to design something with this to get started.

  First of all, there's now a web site at wesp32.com.  It contains information on the hardware and on software development in different environments.  Most importantly, it contains a link to the Product Brief, which is pretty much the go-to for all detailed information.

  While the web site contains guidance on how to use the wESP32 with different development environments, there is now also a wESP32 demo code repository on Github, which contains full projects to get you started!  At the time of writing it has the iperf Ethernet IDF code I used to get the performance numbers mentioned in my project logs, an ESP32-Arduino example to control a display from a web server, two simple MicroPython examples to control an LED light, one from a web server and one using Dweet.io, and a Mongoose OS demo that allows you to control an LED light very simply from an automatically generated web app!

  If you take a look at these examples, you can see that most of them are quite simple and don't require much code at all.  For instance, here's the LED control from Dweet.io code:
  

  import machine
  import time
  import urequests
  
  # Initialize light PWM, flash for 1/4 sec on boot
  led = machine.Pin(23, machine.Pin.OUT)
  ledpwm = machine.PWM(led)
  ledpwm.freq(500)
  time.sleep(0.25)
  ledpwm.duty(0)
  
  # Get the ESP32 unique ID as a string
  uid = ''.join(['{:02x}'.format(x) for x in machine.unique_id()])
  
  # Check dweet.io every two seconds to get and update the light level
  while True:
    try:
      light_level = int(urequests.get(
          'https://dweet.io/get/latest/dweet/for/wesp32-light-demo-{}'
          .format(uid))
          .json()['with'][0]['content']['light'])
      ledpwm.duty(light_level)
      print ("Light level {}".format(light_level))
    except:
      pass
    time.sleep(2)

  So if you thought the wESP32 was interesting but would be too hard to get started with, think again!  There are still a couple of days left to change your mind and get your own wESP32 by backing my campaign! :)
  

• Crowd Supply campaign now live

  Patrick Van Oosterwijck • 10/05/2018 at 21:40 • 0 comments

  The wESP32 Crowd Supply campaign is now live!  I have parts for about 230 units in-house, so if you want one before the end of the year, at the time of writing there are less than 100 of those left!  If you order later, they will ship from a new batch which is scheduled for February 2019.

  As I mentioned in the first campaign update on CS, I received the production PCB panels for the first batch:

  I'm really pleased with how these turned out, my wasp is finally black and yellow as it should be. :)

• Crowd Supply pre-campaign page live

  Patrick Van Oosterwijck • 09/05/2018 at 00:17 • 0 comments

  The wESP32 Crowd Supply pre-campaign page is now live!

  Check it out and sign up if you want to be notified when the campaign goes live!  There will be some early bird specials so signing up may land you a special deal. :)
  

• Coming to Crowd Supply!

  Patrick Van Oosterwijck • 08/31/2018 at 18:43 • 0 comments

  The wESP32 is coming to Crowd Supply!  After seeing how successful our #LiFePO4wered/Pi+ campaign was, it made sense to repeat the same thing for the wESP32.  I hope to have the campaign up very soon now so you can start ordering!
  

  I've been very busy the past week with ordering boards, ordering components, and coming up with marketing materials.  A campaign video is in the works, and we need demos to show off some cool stuff you can do with the wESP32.  One simple idea was to make a "remote status display" you can send messages to:

  You'd probably want a bigger screen for the real deal, but is conveys the idea. :)

  @martinschki had a good question in the comments: "How much heat do they emit under a low power application?"  I've been doing tests and taking thermal pictures at high load, but I never thought of doing it for low loads!  And in applications such as his where things get built into a wall this is important.  So here's a thermal image under low load, running the demo shown above:

  So it looks like the hottest thing on the board is the Ethernet phy chip.  At 51°C I don't expect any problems for having it in a wall.  But if you're going to use this to build an environmental sensor, you definitely need to keep this heat in mind when you design your device or your readings will be way off!
  

• Final performance checks

  Patrick Van Oosterwijck • 08/21/2018 at 18:19 • 0 comments

  I had left off my data performance testing last time wondering why the iperf UDP data rate was so low.  Well, @aenertia on Twitter gave me a hint: you need to specify the amount of bandwidth you want to test on the client side, otherwise the UDP test defaults to 1 Mbit/s!  So, just an average case of operator error. :)  The iperf on ESP32 example code is too simple and doesn't provide / need this option, and I haven't played enough with this kind of stuff before to know things like that.
  

  Here are the results for UDP when the wESP32 is the client (outgoing traffic):

  Around 70 Mbit /s, nice!

  So to test server, I used the following command on the client:

  iperf -c 10.10.16.44 -i 3 -t 60 -u -b 100m

   This sets the target bendwidth to 100 MBit/s.
  

  Here are the results for UDP when the wESP32 is the server (incoming traffic):

  Wow, >90 Mbit/s!  That's close to the theoretical maximum!

  Now I'm pretty sure that from the moment the ESP32 would start to do anything with this data other than throw it away, the data rate would go down, but it's still nice to see that the Ethernet data system as implemented on the wESP32 performs very well.

  One last thing left on my list to validate the design was to test power performance when the 5V power option is selected by bridging the solder jumper.  As mentioned before, the whole power system is optimized for 12V power, and the 5V option was a little bit of an afterthought but it could be useful to many people.  So how does it do?

  In my first test I was really pushing it:

  Trying to push >2A through a power path that was designed for 1A: not a good idea.  That poor diode was frying, but it lived.  So, intermittent 10W may be possible at 5V, but don't run it like this continuously.

  Here's a more moderate test:

  That's better.  At 6W output (1.2A), the diode temperature isn't so bad.  Still, when running at 5V, your continuous average load should probably be kept below 5W.

  These results are fine though.  If you need high power, use 12V.  It keeps the currents much more reasonable.  If you really need high power at 5V, add your own buck converter.  The on-board option for 5V will likely still be very useful for the majority of people who need 5V in their system though.  5V sensors, keypads, displays, etc. usually don't need that much power.
  

• More data performance

  Patrick Van Oosterwijck • 08/19/2018 at 05:14 • 3 comments

  After I posted the previous log, I kept working on trying to figure out the data rate figures, the worrying one being iperf performance when the wESP32 was the server, which was really poor.
  

  Taking a step back, the reason I was doing these tests in the first place was to validate the hardware.  Before I go to production, I need to have confidence that the design is sound, and in the case of this test, that there aren't any circuit or layout issues that are making the signal path performance poor, such as interference from the PoE power system or impedance mismatches causing reflections and affecting the signal to noise ratio.  Such issues would result in high packet loss, making retransmission necessary, which in turn is observed by poor data rates.

  So from the previous tests, it looked like the outgoing signal path had been validated as working well (iperf client), but the incoming signal path was suspect (iperf server).  So I started looking at the layout to see if there was anything obviously different between the incoming and outgoing signal paths.  Not really, both the TXP/TXN and RXP/RXN were laid out as impedance matched differential pairs.  In the picture below, you can see them as the 4 fatter traces under the "Wired ESP32 with PoE" writing:

  While I was perusing the LAN8720A phy datasheet to see which pair was suspect, I looked at the "Architectural Overview" block diagram on page 5, and I noticed something I had forgotten: that this chip implements "Auto-MDIX", a feature that automatically switches the TXP/TXN and RXP/RXN pairs as required to work correctly with either straight or crossover Ethernet cables.

  This gave me an idea: if I used a crossover cable, the incoming and outgoing data would switch on which differential pairs they were travelling, and the phy would internally switch them back so things would still work.  If the problem moved from incoming to outgoing when I did this, then I'd know one of the data pairs was performing poorly.  If nothing changed, and the performance on the outgoing data test was still good, I'd know both data pairs were performing well.

  After locating a crossover cable, I ran the test, and the results were identical to the ones from the test with the straight cable.  This means both data pairs are performing equally well and the data path is sound!  Yay! :)

  So, the question remained: why was the iperf server test working so poorly?  Even though I had pretty much validated the hardware, this still bugged me so I kept trying to see if I could figure out the issue.  I decided to play with some of the settings offered in 'make menuconfig'.  In the process of doing this, at some point I got the client performance to >80 Mbit/s, but the server performance remained poor.  Eventually I did find something that made the TCP test work: the option "CONFIG_EMAC_L2_TO_L3_RX_BUF_MODE".

  The docs explain that "when Ethernet MAC doesn’t have any unused buffers left, it will drop incoming packets (flow control may help with this problem, to some extent)".  Still, the docs recommend to keep this turned off if unsure, because it involves copying data instead of passing pointers, which decreases performance.  Still, the iperf test was apparently running into the problem that the MAC didn't have buffers left, because turning it on made a big difference.  Here are the results of the iperf client test with this config:

  And here are the results of the server test, which now performs well:

  That's when testing with TCP.  Oddly, the UDP server test still performs very poorly: I only get around 1 Mbps.

  Obviously, this is some other configuration issue.  Since TCP was working well, I called it a day.

  Overall, I'm very happy with the performance of these first prototypes: the power output beats my expectations and the spec, the attainable data rate is very good and beats my expectations as well.  With these prototypes working as well...

  Read more »

View all 23 project logs