Project Logs

Collapse

• And the projects come and go, talking of Michelangelo...

  ziggurat29 • 05/17/2018 at 16:14 • 0 comments

  Summary

  Time to Rise and Resurrect work on this project after languishing a few months.  When I left off, I was having a Devil of a time getting eLua to work with absolute stability on my NetDuino/STM32Workbench/FreeRTOS platform.  I am taking a step back (for now at least), and trying a straight port of Lua 5.3.4 for now.

  Deets

  When I left off, I had eLua somewhat working, however it had a tendency to crash all the time.  I know it must be something dumb that I am doing, but I really have to say it should be simple to do a platform adaption, and I guess for some reason for me in this case it is not.  Again, 'it probably is just me', but I'm tired of reverse-engineering the code to try to guess what is going on.  So, I'm taking a step back to the beginning, and porting the Lua 5.3.4.

  Ultimately, I wanted 5.3.4, anyway, and eLua (as with other projects like LuaJIT) are stuck in the past with 5.1.x.  I'm not sure why, but it is pretty common.  I mainly want 5.3 because it has bitwise operators, and why not be on the latest if you're doing new work.  The upside is that I have already integrated 5.3 in other (desktop) projects, so I am familiar with it, and the downside is that it does not have the eLua optimizations for memory constrained devices -- the so-called "Lua Tiny RAM" patch.  'So-called' because I don't see a patch file anywhere.  Rather there is just the resulting patched source.  Anyway, I've decided for the moment that I'll save analysing what that is and make a proper 'patch' to the 5.3 source as a future exercise.  For now, I want an unconditionally stable Lua running in FreeRTOS with the STM32Workbench toolchain on the NetDuino board.

  So, I made a new branch, 'elua002' (named before I decided to abort elua for now), and transferred over my memory manager and wired in the Lua 5.3.4 source.  I modified the 'lua.c' which contains a 'main()' that needs some minor changes so as to be invoked from a FreeRTOS task, and built.  I didn't bother trying to run it since I know it is not going to work without more support.

  The STM32Workbench toolchain uses newlib-nano.  Newlib expects to have a bottom edge ('syscalls and 'stubs') realized that provides platform-specific implementation for things like open(), close(), fork(), kill(), etc.  This makes some sense, but to my great vexation, the STM32Workbench build of newlib-nano has some sort of default implementation of those lower-edge functions.  What do they do?  I can tell you what they don't do:  they don't interface with my board's serial ports or filesystem implementation.  So I gots some works to dos.  But what, precisely?

  My first attempt was to #define my self out of reality.  There is a master header 'lua.h' that all the lua modules include, and it has a section marked at the end stating 'here be your dragons', so I made a 'redirection' header which #defined all the stdio/stdlib stuff that I thought I would need into my own implementations, cleverly named with a prefix if 'my-'.  This was a fairly surgical change, and that aesthetic appealed to me.  However, studying the resulting map file showed that some libc things were still getting hoisted in, so I ditched this approach.  I probably could have continued analying it and got it working, but it gave me the heebie-jeebies as far as being unconditionally stable in the case of evolving more code.  So I gave this up, and went back to bit the bullet by implementing the newlib bottom-edge functions properly.  This should be unconditionally stable, though potentially more, or awkward, work.

  <soapbox>

  • I would have preferred that the vendor (STM?) have not built newlib with the default implementation.  This would have cause linker errors that would make it very obvious as to what I needed to implement.
  • I would have preferred that '-specs=nosys.specs' linker switch would...

  Read more »

• Reimplement Realloc Redux

  ziggurat29 • 01/18/2018 at 03:03 • 0 comments

  Summary:

  I have implemented 'realloc' in the FreeRTOS 'heap_4.c' implementation, along with some other debugging features.

  Deets:

  We left off with my mentioning a way to intercept and redirect function calls at link time (on gcc, using 'wrappers'), but that there was a functional gap in that the FreeRTOS heap implementation does not have a 'realloc' function.  I set down to implement 'realloc', and I can report that I have emerged triumphant.

  I started with the 'heap_4.c' implementation.  In this implementation, the heap is realized as a series of blocks aligned to a defined boundary size (in this case '8 bytes').  The blocks have a short header consisting of a pointer to the next block, and a size of the block itself including header.  The size field furthermore uses the most-significant bit to indicate 'allocated' or not.  If a block is /not/ allocated, it will be part of a list linked by the header's 'next block' pointer.  If it /is/ allocated, the header pointer will be considered invalid (it will actually be NULL-ed), and the size will have the most-significant bit set.  So, alloc()'ed blocks are not linked, but free blocks are, and in address order.  Free'ing a block will return it to the linked list in the proper place, and merging of adjacent blocks is performed.

  I implemented a 'pvPortRealloc' function that defers to the existing pvPortMalloc and pvPortFree for the degenerate cases, and otherwise tries to satisfy a resize request by nibbling away from the next block, if it is free, and if there is enough space.  If there isn't, it will try to do a malloc-copy-free sequence.  One implementation detail of 'realloc' is that if it fails, it does /not/ free the original block; callers must be aware of that.

  I created some unit tests and verified the code to my satisfaction.  In the process of satisfying myself, I discovered a couple things:

  1. the heap is created from a static chunk of RAM named 'ucHeap'.  FreeRTOS will defined for you, but there is an option 'configAPPLICATION_ALLOCATED_HEAP' that will suppress that, in which case you are meant to define it yourself.  This lets you place the memory in other regions if you like.  One thing I noticed is that the default definition is not suitably aligned (8 byte), and consequently four bytes were wasted at the beginning and end.  I want my 8 bytes back!
  2. heap memory is not initialized in malloc, of course.  This made visual debugging of the heap a bit tedious, since it was not readily obvious what parts were allocated/freed/written to.

  For the fist thing, I used the configAPPLICATION_ALLOCATED_HEAP option and a gcc-specific declarator '__attribute__((aligned(8)))' on my own ucHeap, and was able to get the whole heap aligned correctly.  Ostensibly, the linker would also stick some other variables in the space 'recovered' this way to be efficient, though I didn't verify.

  For the second thing, I added a new feature whereby the heap memory was filled with distinctive patterns under certain conditions:

  • for memory that can not be used (e.g. padding), I filled with 0xdd (for 'dead data')
  • for memory that has been freed, I filled with 0xfd (for 'free data')
  • for memory that has been allocated, I filled with 0xcd (for 'clear data')
  • for memory that has been just initialized, I filled with 0xed (for 'virgin data')

  These fill patterns are only done if a switch 'configMALLOC_FILL' is defined to 1, so it's a completely optional debugging feature.  Now I can visually see at a glance where blocks are bounded, what is allocated and cleared, and what has never been used at all.  It's kind of pretty!  It's so much easier to visual inspect the heap in the debugger than it is to manually go through the bytes with a calculator to see what is and isn't used.  This leads to the next thing...

  Armed with my newfound heap internals knowledge, I implemented another debugging tool:  a heap...

  Read more »

• Wrappers Delight, and Quantum Interpositioning

  ziggurat29 • 01/13/2018 at 17:46 • 0 comments

  Summary:

  I am trying to replace the malloc() implementation so as to get an idea of memory usage patterns.  Along the way, I discovered some interesting things about the internals of libc, eLua, FreeRTOS, and some features of the linker.

  Deets:

  When I last wrote, I was having more of the ever-increasingly-familiar hard faults.  I was able to improve that by fiddling with some heap and stack size parameters, but I really needed to use a little more rigor into understanding.  At that time, I had two major problems:

  1)  I had no real visibility into actual heap usage or patterns
  2)  I could not get an answer from (e)Lua as to it's perspective of memory usage

  The second wound up being simpler, so I'll explain it first.  When I finally got Lua stabilized enough to execute slightly less trivial statements, I am supposed to be able to get the memory usage by issuing something like:

  print ( collectgarbage ("count") )

  but all I got was a blank line (hey, this is an improvement of the hard fault I was previously getting).  I debugged into this a great deal already using my bogo-binary-search approach, but this seemed something else.  Then I remembered two things:  Newlib Nano specifically excludes floating point specifiers to printf() by default, and numbers in Lua are all double-precision floats.  (The newest Lua -- 5.3, to which I ultimately wish to use -- supports integers (and can be made to support 32-bit integers and floats, which I very much wish to do with this hardware), but I'm stuck on 5.1 for the time being.)

  Remembering this, I used a linker command line switch to include the float support in printf():

  -u _printf_float

  OK!  Now I can get some output:

  (Those numbers are in kilobytes.)  Interesting how it goes up and down.  Garbage collection in action.  But still not huge enough that I would have run out of my previous 64K heap when I was crashing, so there's more that needs to be understood.  What I really want to do is a 'heap walk', so I can see all the blocks allocated to understand better what's happening.

  For my first amazing feat, I did some research into replacing malloc() effectively in a newlib project.  You'd thing this sort of thing is done all the time, and actually it is.  There's a variety of techniques for it.

  • Avoidance
    By far, the most common technique is:  don't use malloc() in an embedded system, you are asking for trouble with deterministic runtime behaviour.
  • Replacement
    Many well-crafted libraries provide a mechanism to customize key features such a memory allocation.

  • 'Interpositioning'
    A variety of techniques to coax the compiler/linker into doing what you want to do instead of what the author wanted to do.

  I appreciate 'avoidance', and if I totally controlled the code, that's almost certainly what I would do, but this (e)Lua is simply beyond my control and I have to accept that there will be dynamic memory allocation requests.

  I did discover whilst tracing through the code that Lua provides a very tidy means of fully controlling the dynamic memory allocation scheme:  you simply implement a 'frealloc' function, and set that when you initialize your Lua state via lua_newstate() or lua_setallocf() if you already have a state on-hand.  Thanks, Lua!  I wonder why that was not used?  I will probably explore this later, but for now there's still more to the system than just Lua.  Amongst other things, malloc() is used in various internal implementations in libc.

  The 'interpositioning' techniques are used to cause the target binary to invoke your own code, instead of the originally intended code.  These techniques tend to be compiler and linker specific.  In this case, with gcc, there are at least a couple things that can be done:

  1. the linker will 'prefer' to link the first symbol it finds in the order of object modules and libraries it is specified to use.  Indeed,...

  Read more »

• Shell, Interrupted

  ziggurat29 • 01/10/2018 at 17:15 • 0 comments

  Summary:

  I managed to get the shell receiving data by crowbarring eLua to not install interrupt handlers.

  Deets:

  Where I left off, I found that my UART receive function was never called.  At the time it seemed that the eLua is expecting there to be a separate activity -- perhaps interrupt oriented -- that filled a receive buffer.  This turned out to be the case, and moreover eLua wants to install it's own interrupt handlers under conditions unknown to me.

  Since I'm wanting to use the STM32 HAL libraries (at this point in time, I may switch to the Low-Level ('LL') libs later), I don't want eLua doing anything interrupt oriented or otherwise down to the metal.

  At length, I found a couple defines that cause the interrupt handler to be installed

  CON_BUF_SIZE
  BUF_ENABLE_UART

  These get emitted into the generated board header (in this case 'netduinoplus2.h').  I tried changing the board def to specify 0 for the buffer length of the serial port, but this cause the build system to break.  So for the moment, I manually undef'ed them in netduinoplus2.h to remove the defines.  Now the getch() for the console does make it all the way down to my platform receive function, and I get characters!

  So, now I need to find if there is a more orthodox means of removing the buffer and interrupt management from eLua.  Along the way, I noticed a BUILD_C_INT_HANDLERS define.  This seems to come from a board config setting 'cints = true', so I turn that to 'false'.  This causes other problems in a BUILD_LUA_INT_HANDLERS, which in turn is derived from a 'lints = true' so I turn that off also.  Eventually, this did not fix my console buffer size hack, so I had to leave that in for the time being until I learn more as to what all this stuff is.

  At this point, I am able to run eLua, but I can't do much.  I can run things like:

  print ( "Hello World!" )

  for i = 1, 10 do print ( "Hello" ) end

   but more complex things, like:

  print ( collectgarbage ("count") );

  winds up in the HardFault handler.  *sigh*.  Surely I am not out of heap on these?  Well, I did reduce the heap artificially to 64k 'just cuz'.  I also changed FreeRTOS to 'static only' so that I could deterministically see it's use.  FreeRTOS also has a heap implementation that lets me see some statistics such as 'what was the least amount of heap available, ever'.  The trick is to get all this other code (eLua and libc) to use it, whereas they are coded (and in the case of libc, already compiled) to use 'malloc'.  So I'm going to look into some magikry to redirect those calls into my (er, FreeRTOS') malloc implementation.
  

  Next:

  Try to replace memory management.

• Shilly-shally with the Shell by the Seashore

  ziggurat29 • 01/08/2018 at 03:13 • 0 comments

  Summary:

  I first try to get a UART up to provide some I/O for the shell.  Half seems to be working.

  Deets:

  For my first step, I am going to simply get a serial port up with the System Workbench HAL drivers, and attempt to wire that into the eLua.  This should give me a serial console, if it all works out.  This will be an interim implementation, though, because the selection of those pins to that port will be hard coded int the firmware.  I believe the ultimate goal is that the pin configurations will be assigned at runtime, under Lua control.  Nevertheless, the experience should help guide me in how to do that, and anyway I really need some I/O now.

  As a simplification for now, I simply configure STM32CubeMX to designate the (user-visible) D0 and D1 pins to be UART -- specifically USART6 (because that's how the board is wired) and emit init code for that.  Then I will tweak the eLua board def to emit a header indicating that eLua's notion of 'UART 0' will be the serial console.

  This is not how it will ultimately work.  I'm expecting that in the end, all the pins will come up in tristate or analog mode, and then the Lua application code will 'open' the various devices, which will cause the pins to be configured at runtime in the associated manner.  I.e., D0 and D1 would be useable either as UART or digital IO as per application; not hard-coded to be serial as I'm doing now.  But I've got so much more to learn and do before I can implement that fanciness.

  Anyway, the STM32CubeMX part is trivial.  Now I have to wire it in.

  I took several quick trips to Hard Fault land, and after spending many, many, hours (days?) with my 'binary search' approach to finding the offending code, I decided to put forth a little effort towards getting more info in those cases.  Hard faults on the ARM cause context to be dump to the stack before vectoring to the handler, but Eclipse does not know how to present that information, so you don't see the usual stack trace if you set a breakpoint in the handler.  I found some code on FreeRTOS's site for some simple information gathering, but it didn't work as-is.  It involves inline-assembler, and I guess my compiler (gcc) is slightly different than whatever they were using (which I would have thought would be gcc, but whatever).  My problem was extremely simple:  I couldn't branch to a subroutine via a register -- the address loaded was always incorrect.  I did manually change the register to the correct address, and verified the other stuff worked as expected.  After yanking-and-twisting for a while, I did get the handler working, so I post it here for posterity:

  void prvGetRegistersFromStack( uint32_t *pulFaultStackAddress );
  
  __attribute__( ( naked ) ) void HardFault_Handler(void)
  {
      /* USER CODE BEGIN HardFault_IRQn 0 */
      //XXX there needs to be __attribute__( ( naked ) ) void HardFault_Handler(void)
      //XXX but the code generator will probably overwrite that.  Verify that has not
      //XXX been lopped-off before proceeding, lest your stack references be off.
  
      __asm volatile //'volatile' to prevent gcc from rearranging them
      (
          " tst lr, #4                        \n" //test EXC_RETURN number in LR b2
          " ite eq                            \n" //if zero then
          " mrseq r0, msp                     \n" //Main Stack, put MSP in R0
          " mrsne r0, psp                     \n"...

  Read more »

• The Crash of the Titans

  ziggurat29 • 01/06/2018 at 19:54 • 0 comments

  Summary:

  Before getting down to the business of gluing eLua to the System Workbench STM32 HAL libraries, I find some sources of crashes, and fix them.

  Deets:

  After finally getting a build to complete, I let it run and forthwith wind up in the Hard Fault handler.  Again, this is not great surprise (I'd me more surprised if I /didn't/, since I haven't wired any platform code to do IO).  Stopping in the Hard Fault handler does not admit to a stack trace in the Eclipse tools, alas, so I incrementally zone-in on the faulting line the old fashioned way by doing a coarse depth-first search with breakpoints.  It takes a while, but at length I found the fault to be stimulated by a call to getenv().  This call was being made while loading libraries -- naturally with my luck the last library to be loaded:  'package', where it was trying to get the LUA_PATH and LUA_CPATH env vars.  It has logic to handle the 'variable not found' case, but getenv() itself was crashing prior to that.

  getenv() doesn't make much sense in embedded, since there is no OS or shell, but the standard library implementation (newlib-nano in this case) exposes it and that does make porting simpler.  grepping shows that the (e)Lua code has plenty of calls to getenv() throughout, so I look deeper.  The System Workbench does not ship the libc source, alas, which is a real pity, so I look for documentation.

  Fun fact:  System Workbench does install (some) library documentation, but it does not link it in the Start Menu, or make it particularly visible.  In my case I found it at:

  C:\Ac6\SystemWorkbench\plugins\fr.ac6.mcu.externaltools.arm-none.win32_1.15.0.201708311556\tools\compiler\share\doc\gcc-arm-none-eabi\pdf\libc.pdf

  Obviously, right?  Anyway, the documentation mentioned that it requires a global variable 'environ' to work.  Since I successfully linked, I must have the variable somewhere, so that's not it.  I looked for the source via web search and found it:

  https://github.com/eblot/newlib/blob/master/newlib/libc/stdlib/getenv.c

  However that was not particularly interesting because it was just a wrapper around an internal function '_findenv_r': 

  char* _DEFUN (getenv, (name), _CONST char* name) {
    int offset;
    return _findenv_r (_REENT, name, &offset);
  }

  I may later download the source package, but it will be not as useful as I might like (interactive debugging), since it will not be the version used to make the shipped binary libs anyway.  But much better than nothing.  In the meantime, I was bored with this, and decided to interactively look at this 'enviorn' variable.  I declared it 'extern' and then was able to inspect it via the debugger and see that it exists, and that it points to a single entry of NULL.  I would not think that to be a problem, it's just an empty environment, but I decided to make a new environment list consisting of two entries:  an empty string, and a NULL pointer (to terminate the list).  This worked fine.

  I don't know what this means, there's no similar code in the eLua, but it is using a different libc, so maybe this is a bug.  If so, it could have easily have been missed, because who uses 'getenv()' in an embedded context except for ported desktop code (which Lua is).

  Happy that I had solved that crash, I let it fly again, and it crashed again.  This time in an fprintf ( stderr, ... ) call.  Again, I'm not too surprised because there has not been any standard objects created, though I would more expect that it would simply direct to the functional equivalent of /dev/null instead of crashing.

  Well, after many hours of stepping through assembler (because I don't have libc sources that match the binary libs), I popped out back into user C code!  The eLua code already had overridden the 'bottom edge' to redirect IO to peripheral devices.  Of course, I haven't implemented anything in that area, so...

  Read more »

• ROMfs and Remus, and the Founding of Rome

  ziggurat29 • 01/05/2018 at 23:29 • 0 comments

  Summary:

  Made a separate tool to generate 'romfs'.

  Deets:

  I set down to reverse-engineer the Lua-based build system to see where the 'romfs' is generated.  Happily, this wound up being a relatively easy exercise, performed by a deviously named routing 'make_romfs' and an auxilliary module named 'utils.mkfs'.  This step is executed later in the build process, and was being blocked by the toolchain detection stuff.  I eventually just decided to cruft together a bespoke tool that performs that single step, rather than fooling around with trying to hook in the System Workbench toolchain at this juncture.

  I copied build_elua.lua to a new file build_romfs.lua and then commented-out everything except for that top-level routine: make_romfs().  Then I ran it with Lua5.1 and observed how it croaked.  I then incrementally added things back until it stopped croaking (and took out a couple things that I know I didn't need that wired into the build system) and wound up with a minimal tool that generates the file.

  The build_romfs ostensibly can operate in three modes:
  

  1. 'verbatim', which copies all files as-is into the image
  2. 'compressed', which runs the files through a minizer first, before coping into the image
  3. 'compiled', which compiles them to bytecode first, before coping into the image.

  The first two are straightforward, and are supported.  The last one is useful, but not supported at this time because I have to build a special version of the compiler 'luac.exe' to make this work.  This feature is expected to be beneficial because the parser for the Lua interpreter apparently can be RAM-intensive, and so it would be beneficial to avoid the compilation step on-device.  However, this will be some work on my part to get running, and if I am to ever get the patches in eLua ported forward into 5.3.4 (as I want to), then I'd rather tackle mess with a special luac after that happens.  This device should have enough RAM to allow me to at least continue the evaluation without prematurely performing that optimization.

  OK, so the build_romfs tool emits a single file:  a header which defines a C array that is the filesystem image.  It is included in one module only (as it must since the definition is in the header), so I simply copy it over and start the build again.

  To my delight (and no small surprise), the build finished compiling.  There are a few warnings, but they are minor.  It does not link, however.  I need to pull some more source files.  These appear to be platform support source files, so I guess platform_int.c is not the sole interface.  (In retrospect, I now know that this is just for interrupts.  There are many others for the other peripherals.)

  This will probably be an incremental process, because I imagine that the board configuration will define what modules are actually needed.  I start pulling the currently required ones in, and stubbing the implementation out as I did for platform_int.c.

  I needed to add the uIP library, and several other modules, but most of the work was the usual 'stub out implementation connecting to the platform' that was done in platform.c.  There is going to be a lot of work to do here, and I'll have to figure out what is the actual public interface.  I also had to twiddle the linker script to emit a few symbols that the eLua relied upon to assess flash usage and the extent of the read-only area for something related to strings.

  In the end, I was able to once again finish linking, and I got these size results:
  -Og
   194872
  -Os
   180568
  -O0
   272336
  -O1
   194496
  -O2
   194696
  -O3
   239120

  So we still have plenty of room for more code.  I don't know how much more will be required, but I don't think it will be much.  I suspect the networking and filesystem library code was evicted at link time, so that could add a chunk.  Also, if I include SSL support, that will incur a big chunk...

  Read more »

• Mass o' Masochism

  ziggurat29 • 01/04/2018 at 16:11 • 0 comments

  Summary:

  Continuing eLua toolchain setup masochism.  I got it mostly compiling, but there is another build tool challenge with 'romfs'.

  Deets:

  Continuing the self-flagellation with the build system masochism yesterday, my freshly healed wounds are ready for more abuse.  But armed with my newfound experiences, I find I have grown stronger in the broken places.

  First I needed to get the Lua-based build system up, which entailed getting Lua 5.1.x up (the scripts as-is are not 5.2+ compatible), getting Microsoft Visual Studio installed (required for Lua Rocks when you have to build some modules from source, as we will here), and Lua Rocks.

  1)  Visual Studio.  I installed the free 'community edition' VS 2017, and made sure all the C/C++ stuff was selected to be installed.  This is a long, slow process.  Oh, you also have to register some account -- don't know what it is for, but I just bound it to my usual dev email.

  2)  Get Lua 5.1.x up.  This wound up being simpler than what I was doing yesterday, because it turns out that Lua Rocks comes packaged with a Lua 5.1 if you don't have a Lua of your own.  So that saved a step.

  3)  Get Lua Rocks installed.  This was a little klunky -- certainly not the drop-kick of, say, nodejs and npm, but I was armed for combat after yesterday's boot camp training.

  For the curious, setting up the Lua Rocks consisted of:

  1. start an administrative command prompt.  Yes, admin, you'll need it.
  2.  setup the Visual Studio variables, so the compiler can be used.  There is a batch file in the installation that will do this for you.  It's deep down, so I'll give you the path for my system, which is probably the same as yours, or only trivially different:
       "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars32.bat"
     Realize that these vars only last for the current command prompt session; you'll have to invoke that each time you create a new command prompt, so don't close it before the next step.
  3. install Lua Rocks.  You can download it from the Internet - it's just a zip file of stuff.  Amongst that stuff is an 'install.bat'.  OK, you're going to need a few options.  Feel free to run 'install.bat /?' to see all the options, but spoiler is you need '/L /MSVC' which will tell it to install the included Lua (conveniently for our needs version 5.1), and also tell it 'trust me, use the Visual Studio'.  The install system tries to be clever, but for some reason it is not clever enough to understand VS2017.
  4. add some variables to the environment:
     To PATH, append 
        ';C:\Program Files (x86)\LuaRocks'
     To LUA_PATH, make it equal, or include 
        'C:\Program Files (x86)\LuaRocks\lua\?.lua;C:\Program Files (x86)\LuaRocks\lua\?\init.lua;C:\Program Files (x86)\LuaRocks\systree\share\lua\5.1\?.lua;C:\Program Files (x86)\LuaRocks\systree\share\lua\5.1\?\init.lua;;'
     note the trailing double semicolons!
     To LUA_CPATH, make it equal, or include 
        'C:\Program Files (x86)\LuaRocks\systree\lib\lua\5.1\?.dll;;'
     again, not the trailing double-semicolons!
  5. Fun:  since you have altered the environment variables spec, those changes will not be reflected into your current command prompt session.  You can either manually set them there, or you can close-and-reopen the command prompt.  If you do close-and-reopen, make sure once again that it is 'admin', and also set the Visual Studio environment variables as mentioned earlier.  Whee!
  6. Install the required rocks:
       luarocks install luafilesystem
       luarocks install lpack
       luarocks install md5
      
     The first and last will require Visual Studio to compile the C-side code, so you should see soem of that happening.  Once you've got these three rocks installed, your pre-requisites are set up.
  7. Note:  the Lua packaged with Lua Rocks is named 'Lua5.1.exe' so that's the name you must use...

  Read more »

• eLua and eGads!

  ziggurat29 • 01/04/2018 at 01:29 • 0 comments

  Summary:

  I tried a simple and naive inclusion of the eLua source.  This was not successful.  More effort will be required.

  Deets:

  Encouraged with how trivially simple it was to integrate the canonical 5.3.4 source into my project, I tossed all that and attempted the same with the eLua source.  More precisely, I included the eLua's copy of the canonical Lua source in the same way.  This didn't work at all, there were other needed modules from elsewhere in the source tree.  That's not a complete surprise.

  I took a look at the diff of the eLua 5.1.4 source and the canonical 5.1.4 source, and there are extensive modifications (presumably mostly embodying the so-called Lua Tiny RAM ('LTR') patch)  So I decided to defer studying that patch to see how to mod 5.3 for now.  I'll just settle for 5.1.x to get that running.

  Remember how I was saying that I was hoping to not have to get into learning another build system, and just include the source directly?  Well, I can kiss that dream goodbye for several reasons:

  1. There's generated code.  So I've got to understand the build system to understand what is being generated and put where.  So far at least it seems only to be generating some headers out of... stuff!  But I have to find all that stuff, wherever it is, and get it into my build system (i.e. 'System Workbench for STM32').  This is time-consuming and quite boring.
     
  2. There's generated include and source paths.  It's a personal peev of mine when #include "xxx" means 'xxx is somewhere, you'll have to find it and add the path to your includes' as opposed to 'xxx includes an explicit relative path to this file' so I don't have to do that, and I don't have to reverse-engineer the build system to figure out where it is and where it is set.
     
     Anyway, maybe I'll bit the bullet and try to run the build system a little at least to make it emit the generated files, and carry on from that point.  The build system won't work to completion, of course, because the compiler toolchain is not visible to it.
     
  3. The build system is 'scons', as per project documentation, which is based on python.  This is yet another 'make' replacement, apparently in the cmake tradition of puking out toolchain native project files based on some configuration.  I really am not in the mood to install python just to drive a build system.  Also, the documentation refers to some configuration that is contained in 'SConstruct', but I cannot find this file.  After rummaging through the source tree, I think I figured out why...
     
  4. The build system is Lua, as per reality.  At some point, the developers may have come to a similar viewpoint regarding the python dependency, and they created their own (I think) build system based on Lua scripts.  I might have been nice to have updated the project documentation to that effect (e.g. on the page http://www.eluaproject.net/doc/v0.9/en_building.html[Building eLua]!)

  OK, I like to kvetch.  Here's another one:  so, I build my Lua.  Then I install the package manager, LuaRocks.  So, I have to rebuild my Lua, and move it to some apparently conventional tree structure, then I install the package manager, LuaRocks.  OK, so I have to install Visual Studio and rebuild my Lua, and install it to an apparently conventional tree structure, then I install the package manager, LuaRocks.  OK, so I have to put a bunch of command-line overrides on the install.bat to LuaRocks, because for some reason it wants to deploy it's 'rocks' into c:\, and it's necessary to have the Visual Studio build environment setup before invoking the installer, and you need to tell it 'don't try to figure out what Visual Studio is installed, you won't do it right, anyway'.  Sweet Jesus!  But at length I have a presumably usable Lua, and I got it's rocks off 'luafilesystem', 'lpack', and 'md5'.

  Now, from the cleverly named 'build_elua.lua'...

  Read more »

• Ye Olde Build System Setup

  ziggurat29 • 01/03/2018 at 00:32 • 0 comments

  Summary:

  Setting up the build environment, and debugger.

  Deets:

  For my first amazing feat, I get the debugging pod working, and verify that I can program the device.

  I have done this before, so for the moment I'll just link a reference to another project where I went into the details of such at greater length, q.v.:

  https://hackaday.io/project/25616/log/62106-setting-up-build-environment
  https://hackaday.io/project/25616/log/62150-torment-and-torture-by-tortuous-tools
  https://hackaday.io/project/25616/log/62581-tool-everything-tool-terrible-two-openocd-rift
  https://hackaday.io/project/25616/log/62656-serial-killer-the-sorrow-of-sad-sorry-serial-stuff

  ST Microelectronics ('ST') has a 'wizard' style tool called 'SM32CubeMX' which lets you select peripherals, pinouts, clocks, etc., and then it generates a skeleton project with all the relevant libraries referenced and initialized.  The code generated is oftentimes of dubious quality, but it certainly is convenient, and the device has a large flash, so I usually opt for the convenience for starters, then optimize if needed later with custom code.

  I created a boilerplate config file for ST32CubeMX describing the Netduino Plus 2 by meticulously combing through the schematic and reflecting those clocks, pin assignments, and peripheral choices into the description board.  I saved this off as 'NetduinoPlus2.ioc' which can be used as a basis for starting other projects for that board.  Then I made a copy for the NYETduinoPlusLua project and generated the skeleton.

  I build the project and verified that I could flash the firmware and step through the code.  It does nothing - not even a 'blinky', but that's good enough for me since I've worked with this toolchain before.

  Eager for an early 'go/no-go' on Lua, I decided to simply dump the Lua source into the project, and build it and see what happens.  This is the official Lua source, version 5.3.4 specifically, and it is structured simply (just a bunch of source and headers in one directory.  I copied that stuff over and included it all (except for luac.c, which is the stand-along compiler) and built it.

  To my great surprise, it compiled with not a single error/warning.  This is very rare, but Lua is known for it's ease of integration.

  When built as non-optimized for debug (-Og), arm-none-eabi-size "NYETDuinoPlusLua.elf" reports:

  (-Og):
    text     data     bss     dec     hex filename
  157756      560    2912  161228   275cc NYETDuinoPlusLua.elf

  For completeness, and for the curious, I built it with the other optimization levels, as well, and have included their final statistics:

  (-Os):
  147832      560    2912  151304   24f08 NYETDuinoPlusLua.elf

  (-O0):
  217364      560    2912  220836   35ea4 NYETDuinoPlusLua.elf

  (-O1):
  157660      560    2912  161132   2756c NYETDuinoPlusLua.elf

  (-O2):
  160292      560    2912  163764   27fb4 NYETDuinoPlusLua.elf

  (-O3):
  201324      560    2912  204796   31ffc NYETDuinoPlusLua.elf

  So that's pretty compact!  Lua is known for being a 'bare-bones' system out-of-the-box, so this doubtlessly includes nothing other than the interpreter, runtime, and a few standard libraries.  But it does look like there is breathing room to add stuff on this device, which has 1MB flash.  I have not idea about the RAM usage at this point, though.  There's a little bit of initialized static/global data (.data) and uninitialized (.bss), but who knows about runtime heap usage.

  On a lark, I wired in it's main, passing dummy parameters for the expected argc, argv, and stepped through it.  It did allocate and initialize the environment, so I let it go and it made a quick...

  Read more »

View all 11 project logs