-
Merging C and ASM...
07/23/2022 at 06:32 • 6 commentsNext-Next and Next-day brief Updates at the end...
...
I am by no means experienced nor knowledgeable in this realm...
I have done a bit of inline asm in my otherwise avr-gcc progects. But this is very different.
Here we have a booting system written in assembly, and I plan to make a bootable utility in C that makes use of the initial-setup and functions that assembly-code provides. In a sense, I'll just replace its "main loop" with my own.
Initial experiments were very promising. I wrote a small main()-like function in C, wrote a "void function(void);" *declaration* (not definition) for one of the assembly functions, called that function from mine...
From that alone (not even "including" the original assembly file in any way) sdcc gave me assembly-output that was *easily* modified in such a way that I could little more than copy/paste its output into the original assembly file and run my code as though I'd written a new function in assembly in the original.
Basically, sdcc's assembly output did "calls" to labels, and those labels just happened to not exist within its awareness... and it didn't seem to care.
Which was *great* because when I pasted that output into the original assembly file, those labels now were available, and my assembler replaced them with the appropriate 16bit addresses. Just like assembly does.
I guess I'd expected sdcc to croak on the missing labels long before actually outputting a usable assembly file.
So... Awesome!
...
So then I bit off more than I could chew.
Wrote the entire utility in C, tested it with gcc as best as possible all along the way. Finished it, finally, today...
Then... yeah.
OK, so the big thing that ultimately stopped me from proceeding with my original plan (of copy/pasting sdcc's output into the original assembly file, then hand-modifying it as-necessary) is the fact that the sdcc output uses the same label-names in all functions. e.g. "00104$" which resulted in my assembler's complaining of duplicate labels 69 times. I *almost* considered hand-changing them until I realized it looks like it only complains about the first duplicate of each... Heh.
So then, obviously, just throw the thing into sdcc's assembler, instead, right? But apparently *that* didn't like the original assembly in an entirely different and equally difficult to repair way: Apparently it's so low-level that it doesn't handle things like "equ"... which would be a tremendous feat to remove all instances of. Heh.
Again, I'm no expert, maybe it was just a matter of find/replace... but this came after quite some effort dealing with many other incompatibilities, e.g. "immediate values" are prefixed with # in one, but not the other...
So, finally after much "hand-jobbing" I decided it was time to throw up my hands and come up with another way.
Now, I should probably interject that obviously there is some "right" way, otherwise we wouldn't have many of the fine things we have... I imagine "the linker" is a big part of it. I have history with that beast that prevents me from preferring trying another go at it over, say, hand-editting 69 labels.
But, I think I came up with another solution, which actually should be easier... just modify the original to call some specific address, say near the end of the ROM. Assemble that in the usual way. Then modify sdcc's assembly-output with actual addresses instead of labels pointing to the original code. There's only three functions and one buffer to be loaded into RAM. Four addresses to hand-enter. Oh, and a .org at the beginning of its output to somewhere after the original's. And, finally, add a .org at the decided-upon jump-to-address, with a jump to my main(). Then, of course, use sdcc's tools to compile that, as though it was its own completely standalone thing. Merge the two ihex files, and we're done! Scripting that whole process should be easy-enough, too. And it allows for keeping the two codebases separate, which has many benefits.
I dunno what this all equates to in "the normal way". Probably a minimal linker-script, maybe a tiny C-Runtime...
I think it should work. I guess we'll see.
...
Oh, Almost forgot the days of trying to figure out sdcc's method for passing arguments to functions, getting back return values, keeping its registers from being mangled, etc... but that's another topic. Along with numerous others I'm forgetting.
...I think this could work... we'll see.
....
Next-Day Update:
It took a little learning of the linker, after all. But, actually, I looked into its documentation (again, some twenty years after my months-long failed ordeal with it) just a tad and saw a bulletted-list of what it's supposed to do (if you're not like me in your attempts to code it). On the list, nothing was there I hadn't already done by hand *except* converting it to ihex. Heh. I thought that was objcopy's job, so looked there, first. But, thankfully, I'd found it in a combined-manual for the whole process, which, scrolling caught my eye with "convert to ihex" and only later did I realize I was looking in the linker's section. Heh!
Handy doc I've not seen elsewhere:
http://www.ee.nmt.edu/~rison/ee308_spr97/asmlnk_doc.html
Anyhow. the short of it is: It's not a huge nor difficult process at all... A small bash script could surely take care of everything I did. BUT. I dunno why it doesn't work. It looks like it's doing some pretty gnarly stuff right after it calls main... randomly resetting, hanging with the address bus going wild, ignoring interrupts... It's like it jumped to some random address somehow. But I've looked that bit over quite thoroughly.
Did I mention I *hate* trying to debug huge code changes? This aint small potatoes, here. And the friggin irony is that the whole reason I'm doing this particular subsubsubproject is to enable testing of small code changes, so I won't wind up in exactly the situation I'm in. GAH THAT FRICKIN CHICKEN AND ITS EGG!
I should make a list of all the things this blasted utility contains that really should've been tested on real hardware, every step of the way, rather'n trying to simulate. Believe me, I wouldn't've gone this route if it hadn't basically been week(s?!) of a dangling-carrot just inches away from completion.
I've written yet another gnarly rant about debugger-culture I'll spare y'all from.
....
Next-Next day:
Well, I decided to settle in for the long haul to figure out where it was crashing, step-by-step starting from before calling main() to just after... Coded up wrappers for a couple led functions... Turned on some LEDs at different steps... What else? Nothing, really, far as I recall.
And It Just Works. (so far as I could test).
Heh!
At the same time I wrote-up a step-by-step hand-linking procedure. It's not at all difficult if you're just cross-calling a few functions, as long as you've got wrapper-functions for them, to interface C's call/arg/return convention with ours in assembly. After that it's just a matter of copy/pasting a few addresses and some other goofies like using just one section with absolute addressing via ".org". Oh, and I couldn't figure out how to typecast a uint8_t* to a void function(void) so as to execute the actual Flash-writing procedure from RAM. No matter what I tried, no matter where I put "const" sdcc insisted on using a "trampoline" function to call the absolute address. "ld hl, address ; call __sdcc_call_hl" which is goofy and I didn't include *any* external libraries, anyhow, so that resulted in a missing symbol. Anyhow, the solution is easy in assembly, so I just rewrote those two lines by hand in sdcc's assembly-output as simply "call address". Heh. (Hmm, I suppose inline-asm would've been far easier). Oh, right, and for some stupid reason its using jp instead of jr in the function that has to be copied-to/ran/from RAM... but, again, easy fix. Probably a few other things I'm forgetting... but not difficult nor unexpected.
So, now, I can use serial I/O and a few other things within C, while those functions, buffers, and interrupts were written [now seemingly long-ago] in assembly and loaded at boot...
Kinda like a BIOS, maybe? Kinda cool!
Still rather frustrating it seems like it just fixed itself... Non-repeatable bugs are the worst. My changes were minor. Not intended to fix anything, just to *find* it. Though I suppose it's possible I missed a step in the first go-round hand-linking... I don't think I'm bouts to disassemble the old broken ihex... unless... this keeps... nagging at me... like it is. ARGH.
As it stands, I couldn't test everything... The actual flash-writing fails because I've yet to modify the board/circuit to handle two 16k pages instead of just one. I expected that. Too dark to solder tonight. Though, seeing it actually working all the way through is also nagging at me.
....
Next Next Next day... And into the next thereafter:
I couldn't allow this bug to just magically fix itself...
14 hours later... amounting to little more than organizing files and diffing... It would seem there were two stupidities on my part and, yes, both were fixed... Properly. No magic involved.
There were three versions until one worked properly. The first acted quite weird, randomly resetting, disabling ints. I caught that I'd accidentally jumped to main before the global variable inits which were in the previous mainlike function. Changing that was what version2 was all about... And it *sorta* fixed some things, but not really much more in the way of functionality before it appeared to crash at basically the same point: right after jumping to main, but apparently before the "boot message." Version three was my hunker-down to begin debugging... The first steps amounted to nothing more than moving code from one big file into two smaller files. And... It Worked.
Now, that just don't make sense.
So today was all about investigating...
First, could the uninitialized globals *really* cause seeming crashing, random resetting, and Ints to be disabled? The only way I can perceive that possible is executing instructions outside of ROM, which took some time to figure out how that could possibly happen.
The number of factors involved are crazy... Basically, because the ReceiveData buffer is not initted, its "length" (aka number of bytes currently in the buffer) could be anything... IF its length is greater than the number of bytes the buffer can store, then it will never stop returning data when requested. Now, I forgot to check, but I think the result would be just repeating the same 128 bytes over and over(?). Regardless, what it would mean (only if length just happens to power-up > size) is that the ihex parser thinks it's receiving data. *immediately* Now, the ihexParser ignores *everything* /before/ a colon... that space is for newlines and comments.
(Did I mention that I do not enjoy this kind of sleuthing AT ALL? 14 friggin hours! But I DO care about the quality and reliability of my work. So, even though I could've actually *seen* my weeks of hard work Doing What It Was Designed For for the first time today, instead I spent 14 friggin hours doing something I darn-near despise doing. And keep in mind this was 14 hours with code I already know inside-and-out from weeks of writing half of it myself, and weeks prior of working with the other half.).
OK, so the ihexparser gets random garbage that just happens to have been in the RAM at power-up (our RAM is battery-backed, which might help to explain the slight pattern in the randomness of its resetting, etc.) But, again, it totally discards everything before a colon. Which means, somewhere, randomly, it found a colon. And, again, I'm not certain, but that random colon might've had to have been in a specific 128bytes in the RAM.
Now, I didn't do anything fancy, like stop processing on CRC errors, or when something other than 0-9, A-F is received. (It *will* spit out a CRC-Error message, which if you see then you can just send the entire file again, to flash it again... Big whoop, 30 seconds). But, what that means, here, is that after it finds that random colon, it tries to parse the actual ihex data... which, of course, is garbage. But I didn't do fancy-math/testing, so any two-byte binary value would return *some* 8bit number.... Now, part of that ihex stream is a number of bytes in the ihex line. 0-255... So then the parser grabs *only* that many bytes (in hex) and the CRC. After those, it discards everything, again, until the next colon. No data is actually written to the flash until *at least* the next command comes through. (the last batch of data is written when the ihex "EndOfFile" is received, which, yes, starts with a colon). This Means: No attempts to write data will occur unless *two* colons are received, *at least* as far-apart as whatever it interprets the two bytes following the first colon to equate to in a uint8_t... So, somewhere between 0 and 255 bytes, as "specified" *at least* have to separate these colons. (Note that the parser ignores 0-byters... So, actually, we have to have a somewhat-specific sequence of ":XY" for this to work... And that somewhat-specific sequence has to occur at least twice... And at least a somewhat-specific distance apart).
It's further-complicated by: it only writes once an entire flash-sector (128 bytes) is loaded *or* the second ":XYABCD" where ABCD (the write-to address) is outside the range of the currently-selected sector.
So: Let's say the circular buffer just kept sending out the same 128byte sequence repeatedly... There's say a 50/50 chance the "number of ihex bytes" will actually be interpretted as greater than one sector... If it wasn't, then we'd not see this glitch, because it would just keep reloading the same <128 bytes to the sector buffer, and never attempt to write them.
Again, mind-you, we're talking about whatever just randomly happens to be in the RAM at power-up. And I never *didn't* see this glitch. (though, again, our RAM is battery-backed. The fact is, I think the battery is weak, as previous experiments showed its holding data between resets, but seldom long between power-cycles).
The only logical explanation for its malfunction is executing code from somewhere *other* than the ROM. Because, even if it were somehow executing some random address *in* ROM, it would have to get there, first.
So, we're at only one reasonable explanation... (aside from maybe power-glitches, and they didn't put decoupling caps on any of the chips, and the power traces are surprisingly thin and snakey... but despite ground-bounce my logic-probe picks up, this thing *has* been surprisingly stable when the code's not buggy)
Somehow that seemingly ~50/50 chance, at best, of two bytes being interpretted as >128 (Greater-than *not* equal!), combined with whatever weird-low odds of there even being a colon to go before them, and even weirder odds of there being *another* colon (followed by a non-zero two-bytes) to trigger the sector-write procedure... I mean, again, the best chance seems to be if the circular buffer just happened to contain those three somewhat-specific bytes in sequence... in only 128 positions?
Oh, right, there's more...
If the random data in the RAM just happened to have a *zero* in the "transmitting" flag, then we'd see at least a few characters... But I never saw any transmission. Which Also implies that the random data in the transmit-buffer's "length" must *also* be outside the actual size of the buffer. Otherwise, it would've hung early, long before the ihexParser was called, due to waiting for the Tx Buffer to drain so it could finish sending the boot message!
Really, it seems the odds of all these things happening exactly right to cause this particular glitch, not once in a while, but *every time* seem downright infinitesimal.
*Hanging*, waiting for the TxBuff to empty, that'd surely be *far* more likely, but even in those cases that seemed to be the case (when it took a long time to reset?), RxInts weren't being handled. Which, again, shouldn't be the case unless the Wrong Code was executed somehow/where... (to disable the ints). Which, again, can only happen in one place in the code: When the sector is to be written... Because that particular function has to be loaded-into and executed-from RAM, whereas ALL other functions/jumps are within the ROM, half of which were auto-generated by C, the other half meticulously hand-written and looked-over countless time in Assembly. NO external libraries, compiler-provided runtimes, nor even use of the linker.
These odds seem incredible.
NOW I'm In No Way trying to say the code should work correctly with those globals uninitialized. Nor even that my code is robust in cases like random data errors. I AM saying that the Particular Way it's malfunctioning seems incredible. As in: lacking in credibility. Improbable.
So... Again, even after, now, 16 hours of looking into nitty-gritty and a couple more hours' pondering, here, I'm still not entirely convinced The Cause of The Glitch I've seen can be boiled down to "uninitialized globals". HAH!
...
Meanwhile, the bug in the second version *seems* almost crystal-clear... I'd forgotten to change the addresses, when hand-linking, after fixing the global-inits. The result is that all my assembly-function calls are three bytes too early... Which most-likely throws them into the return-area of the previous function.
I, frankly, figured something like that was the case somewhere in this... it didn't magically-fix-itself; I just used a different procedure in the third version (which was near-identical, and should've been functionally-identical, and later proved to be) when hand-linking, which reduced the chance for that sort of error.
And, frankly, I almost thought it'd be a complete waste of time to do any of this on that account...
But those globals... Five hours ago it seemed blatantly-obvious (again) they were the culprit, and again that it would be a waste of time to look into *after* their proper initializations seemed to fix it...
But now after writing this, I'm again not at all convinced the glitch I saw (and seemingly disappeared with the proper inits) could've possibly been caused, repeatedly, by such a low-odds string of just-right-conditions.
....
Day number three thousand four hundred and seventy six:
I've been strung-along on this voyage for what seems like an eternity. I don't recall anymore how long ago I could've just counted my blessings that it worked and moved forward, when instead I chose to analyze in depth what, truly, specifically, caused the particular failure I'd seen in the first place.
Today I contemplated how to determine whether the original "obvious" conclusion-- which I later determined nearly infinitesimally-possible--was truly the culprit. I eventually came up with the least-invasive modification I could think of: Find the call to that function in the original ihex, and block it out with NOPs. If, then, the problem stopped, then obviously that would indicate that either my infinitesimal-odds-estimate was wrong, or that I'd really, somehow, encountered infinitesimal-odds countless times back-to-back, during the original test.
I reprogrammed it with the original (not yet NOPed) ihex, after much debate as to whether re-running those original experiments was even worth the time---I surely remember what I saw, after all.
Turned it on...
And...
No more.
Frankly, I was a bit dumbfounded upon seeing it, since I'd seen the other SO MANY times, previously. And it took me a while to verify I'd uploaded the right version, then to piece-together (again) what *might've* changed to cause this "new" behavior.
Eventually I realized that this time (and after *many* manual resets) it acted exactly as I determined it would've if those infinitesimal-odds *weren't* met. I have those theories documented in my logs from the "deep investigation". Recalling that, eventually, I recalled how to *test* it... And sure-enough the darn thing now gets RxInts, but otherwise seems unresponsive... exactly as I'd predicted.
Hah.
This, mind-you, is *exactly* what I'd expected to see *after* NOPing-out that function-call, as well. So, I guess it was a good idea to run this experiment with the original version again before doing that, lest I'd've confused "infinitesimal-odds-theory" (aka iot) with "See? my test proves it."
Ooof.
So... I suppose it's plausible iot was correct... And, yet, somehow I "lucked out" in getting those odds the first time, because, again, it brought my attention to the uninitialized variable problem, which today's experiment seems to show rather clearly would've otherwise been one of those *very rare* glitches, had I continued-on from how it functions right now. Heh.
Now, how on earth did those infinitesimal-odds occur *repeatedly*? Theory goes something like "battery-backed RAM". And how on earth didn't they happen this time? Same theory; wherein yesterday I ran code that initialized them correctly. NOW, How On Earth did the infinitesimal-odds get there in the first place that first day, when all experiments prior would've *also* initted those variables? Theory goes? *shrug* Maybe the battery-theory combined with a few days of discharge?
Anyhow, these new findings make it exceptionally difficult to try to test these theories. I'm far too deep in this rabbit-hole... I could actually be *using* this thing, for what it was intended, this evening; could've /been/ using it for many days now... Instead of, apparently, hunting down some White Whale that actually worked-out in my favor... I dunno.
Was this worthwhile? I dunno. I dunno if I know anythin anymo
-
To Xon or to Xoff, that is the question.
07/16/2022 at 03:21 • 10 commentsApparently the answer is "Just don't".
I'm too tired of the whole scenario to go into it. But let's just say that some once-standards seem to have been brushed-aside in really awful ways that make me lose faith in far too many things.
I commented on it in a previous log... Basically having come across several linux mailing list archives of folk submitting patches to support Xon/Xoff for various USB-Serial dongles (whose chips actually support it at the hardware level!) and yet those patches being abandoned for essentially "most USB adapters don't support it" despite the fact the drivers claim to. AND, the fact that the stty-default, when you plug those dongles in, is to claim that it's enabled. Worse than that, there were even folk submitting patches to at least give a *warning* to the user that xon/xoff isn't supported, and even *those* patches were seemingly driven out of the kernel.
I also found this: https://hackaday.io/project/16097-eforth-for-cheap-stm8s-gadgets/log/49010-a-serial-terminal-for-linux-with-working-xonxoff
Wherein Thomas went into a lot of digging to figure out (and share) where the hurdle exists...
...
This is now the /third/ such thing I've run into that was basically once such a standard as to be in nearly all the great references that were de-facto reading for generations of folk using RS-232.
The first was long-enough ago that it's a bit muddled in my mind. As I recall it had to do with counting edges on handshaking lines. The great example I recall of its disappearance (and yet claiming to still exist at the driver level) was a GPS standard which is often used to synchronize systems' realtime clocks, e.g. data-logging systems which aren't able to connect to the internet... Like what? Think trailcams if you can't imagine scientific research. Isn't that pretty much *exactly* what linux was once so great for? It blew my mind how, frankly, rude folk were toward this guy, and indirectly toward the entire scientific community, for not using things "the one way" "everyone" does. Goes ENTIRELY against everything that I thought made linux great.
....
The second was custom baudrates.
The Standard, for generations, was to assign your custom baudrate to 38400baud. The Idea being that nearly every terminal application, or serial application, supports selecting that baudrate from a list, whether supplied by the OS, or in a list in the program itself. Thus nearly *every* serial program could make use of a custom baudrate, as long as you configured it before loading the program.
Yes, at the driver-level that might mean running a custom program to actually set the appropriate registers, but even that had become commonplace enough that linux has provided the appropriate and common tools to do-so, for decades; one for countless different serial chips. EXCEPT. USB-Serial dongles. Why? Searches of mailing lists result in pretty much the exact same sentiment, over and over... "Most USB serial chips don't support it" which, frankly, wasn't even true, in my experience, decades ago, and far less today. AND, again, the drivers seem to allude to the support being there, and configuration-programs give no warning it isn't.
Again, this isn't just about buying cheap hardware, we now live in an era where USB is darn near the only reasonable option. This is downright absurd. Their arrogance is affecting everyone from kids learning Arduinos to government-sponsored research endeavors. Nevermind the folk who put tremendous effort into making great software that stood the test of time.
And, nevermind the folk who wrote excellent resources/reference-manuals that we still are referred to as "The Best" now in an era where those things we learned are now just flat-out lied-about still existing in our configuration utilities and drivers.
...
The third is XON/XOFF, and again, frankly, I'm still seeing red that I spent a week or more implementing that in an embedded project *because* stty reports Xon/Xoff is enabled As Default when I plug in my USB-Serial dongle from a very well-respected chip manufacturer whose documentation is incredible for this era, AND, has apparently gone out of their way to support Xon/Xoff *at the hardware level* for exactly the reasons of today's huge buffers and USB packets meaning that a CPU might not receive the XOFF until plausibly hundreds of bytes (even at 9600) or more have already been queued. WHICH would be FAR too much to expect for most of the very hardware that these things exist/ed/ for. The resounding sentiment: "Most USB dongles don't support it". RIGHT. But: Linux has darn-near always been about supporting *many* types of equipment, terminals, even teletypes. Nevermind specialized stuff that was never cheap, nor abundant, and is hard to replace like plotters and test equipment... If your equipment needs something like that, *linux* is "The Guy!" Until the USB-era, it seems.
Sure, your $20,000 piece of test equipment *could* run on even those serial dongles, if someone bothered to write the software for it (send one byte, wait for a little bit, make sure there's no XOFF, heck, do it right and the driver could get a good measure of the device's buffer size, send a burst of 1/4 as many bytes, wait, look for xoff...) There ARE solutions.
But, worse than those solutions' not existing is the fact that the ability to make our own solutions is downright hobbled by that recurring "most don't support it" mentality. You can write a chip-specific /driver/ patch to enable these things, but connecting that to the kernel, in exactly the way the kernel's serial documentation explains, means going through the generic "USB-Serial" driver, which doesn't, though certainly could, support it. What The What?! Since When is the friggin' kernel source full of such laziness and inconsistencies (and frankly rude thoughtlessness) as to not try to make every option available?! I mean, seriously, I can connect some friggin' random I2C port-expander device to the I2C bus dedicated to my DIMMs, and access its pins through /dev/gpio, but I can't friggin' use the *default* options pretty much in every manual on RS-232?!
The worst part is the flat-out lying involved. The driver says it's supported. In fact, it's default. Can you imagine the number of man/woman-hours *wasted* world-wide trying to hunt something like this down, Over and Over again?!
Wait, is that the worst part, or is the worst part that SO MANY /have/ hunted it down as to there being NUMEROUS folk submitting patches, and their hard work, given freely to everyone, being snub-nosed?
And did I mention the horrendous backward-step in rendering-useless multi-generationally-tested/fine-tuned/bugfixed software and reference manuals?
I CAN'T BELIEVE what I've seen.
-
L O L
07/10/2022 at 12:01 • 6 commentsIf you read the last log, you'll understand why I'm LOLing about what I have for this one...
Alright, so, I had several epiphanies lately for long-running unsolveds. Though, weirdly (to the point of LOLing) they seem nearly completely opposed to my plans eventually-made in the last log (many of which I guess I'd forgotten).
One was that I recalled once having written a program in C that didn't have all the system-specific stuff included/configured... and yet, it compiled to assembly source-code, which left the unknown function-calls (e.g. printf()) as just i.e. 'call printf' despite there not being anything in that assembly-output called 'printf'.
That was long ago, and a mistake, and I think I chalked it up to the idea that one can compile several separate C files, individually, then I guess the linker must be responsible for linking things like "call printf" in one assembly-output file to the actual printf function in another assembly-output file.
I was intrigued, but I guess it seemed kinda obvious-enough at the time that I didn't really think about it again until just a few days ago, when that memory came to me out of the blue, and I realized that I could compile my code in C, and the assembly output could be merged with the assembly @ziggurat29 wrote for things like configuring devices, and... using the serial port!
Hah! So, in C, I can pretty much simply call the "serpush" and "serpull" he'd written in assembly, as easily as (and essentially the same as) calling stdio.h's getchar() and putchar()!
This is revolutionary.
(just now it occurred to me I could prb do the same for programs on the TI-86(!!?))
Just before writing this, another similar epiphany: I could probably somewhat-easily merge assembly code I'd written /on/ the TI-86, /for/ the TI-86, into our code for the SD-70...
So, somewhere in there I decided to try this mystical idea of compiling C source code that wasn't fully-defined... and... Holy Moly, it worked.
First test was making a front-panel LED blink, by calling a function writen in assembly... Hah!
Realy, it only takes a *tiny* bit of finagling to literally throw the assembly-output from C into the assembly code we wrote for the replacement firmware. I can even call C functions from the assembly's original "main"-like loop.
I know, I know, it's friggin' obvious, right? Well, it was obvious to me when I made that mistake forgetting a function-definition long ago... I knew from many times compiling the Linux kernel that there were a TON of object-files each compiled from a C file which was aware *of* the others' functions (via declarations) but completely unaware of *where* to find them (the actual *address*). I dunno, I guess I figured there was no way the individual C files' gcc-output could possibly be human-readable, nevermind straight-up-easily-understood assembly that could quite literally be pasted into one's own assembly code.
Really, friggin' amazing.
And it worked like a charm in sdcc, turning my C code into z80 assembly, just like that. Hah!
So, here's where the LOLs come in... I guess I completely forgot my AVR-programmer idea (and many others) from the last log just a few days ago, nevermind the countless thoughts and ramblings on it prior...
And the next thing I know, (well, OK, there was a LOT of "knowing of" things inbetween) I've got an ihex-parser, tested with printfs in linux, in its native ARM (rathern trying to figure out a z80 emulator and whatever hardware emulation it might also need) that could easily be reconfigured to use "serpull" in the (new) firmware-assembly by merely adding 'getnextbyte equ serpull'. HAH!
Then, I wrote and printf-tested the sector-grouper... Because, e.g. the first 128bytes (one flash sector) are very sparse due to the RST vectors, etc... Each is in its own ihex line. So, all those (and everything up to address 0x07ff) have to be combined into a full sector's data-buffer before that sector can be written to flash. Just finished testing that a bit ago... And no chip-pulling-jumper-changing necessary during debugging!
Alright, so, that leaves only two unfinished pieces I can think of: The flash-sector-writing function, and I plan to use hardware-handshaking for serial so I can simply type 'cat newFirmware.hex > /dev/ttyUSB0' to do the whole uploading and flashing procedure.
The former (flash-writing functions) I might just extract from my work last year(!) on #Vintage Z80 palmtop compy hackery (TI-86) . It wouldn't be difficult to rewrite in C, and I do much-prefer the portability of C (like being able to test it on an entirely different and more versatile system!) but it was quite an effort to thumbtype in (then new-to-me) assembly on the TI-86's keyboard and tiny screen... So, since I now know how to mix C and Assembly in this new way, I might decide to reuse that, here, and give it new life.
The hardware-handshaking... well... "That should be easy," but, frankly, aside from modems and desktops I recall *many* an obscure handshaking-scheme, e.g. tying inputs to outputs that have entirely different meanings... So, we'll see. OTOH, I suppose since this whole thing is based on ASCII, xon/xoff might be simpler than I used to imagine (how on earth can you send a *character* to handshake, when it'd look like any other binary data?!). So, I might take a bit to look into that first.
But, really... this whole epiphany was such a "wow!" that I completely forgot my other ideas. And, having just reread them, I see that this whole scenario could lend itself to many of those goals, anyhow. E.G. the AVR-based plug-in in-system flash-programmer idea is great because it could be used with basically any system that has a firmware-ROM I can replace with a flash... BUT, now it doesn't /have/ to be an AVR... C's portable. And sdcc does 8051's, and I have some 89C52's I'd probably otherwise never get around to using... This might just be a good use for them. And, then, the idea is, any such system in the future could be "brought-up" with the flash's being programmed by that external in-system programmer, *until* (and if necessitated) the very sourcecode running on it gets ported to it, directly. Whew!
...fading, quick... sun's rising.
-
Chicken Egg In-System-Programming
07/03/2022 at 07:31 • 0 commentsHeh, apparently the idea of in-system programming has me in a chicken-egg infinite-loop.
'Cause apparently (clicking-through old logs to "add a log") I'e already written quite a bit about it... https://hackaday.io/project/185919-z80-reverse-engineering-and-hacking-adventures/log/207723-in-circuit-programming And, here I was, 'bouts to write about it again as though something new.
...
Anyhow, I've finally swapped-out my test-firmware EPROM for a flash chip, which at the very least will prevent me from stupidly looking at the UV eraser.
The flash chip (An AT29C512 in a PLCC package) I made a DIP adapter for back during #Improbable AVR -> 8088 substitution for PC/XT ... Stupidly, I lost the documentation, so had to beep out the wiring. But, after having done-so, I realized just how forward-thinking I was way back when (Thank you, old me!).
This guy can replace any (jedec-pin-compatible) PROM from the 28pin 64kilobitters to the 32pin 512kilobitters, and probably more in either direction, by changing a few jumpers.
And, unlike plugging a 256kbitter in a 128kbitter's socket, when you program it in a regular external chip programmer, with the regular [AT29C512] chip-specific settings, you don't have to load it at an offset (e.g. 0x4000, when using a 256kb chip in place of a 128kb chip, due to the higher address bit's being tied high for a different purpose).
So, I shoulda been using this all along, but I had to figure out those jumpers and its pin-compatibility first, so had been using a 256kb chip with an offset at 0x4000, and noticeably-increasing UV-erasure-times, and...
Anyhow, after I reverse-engineered my own project, I made sure I won't lose that documentation by folding it up and sliding it in the space between the PCB and the pins... Or, I would-have, but a folded 8.5x11 just won't fit. So I spent friggin' hours, strung-along by "just one more step" trying to print it smaller, just to find out all my attempts were unnecessary because apparently all I had to do from the start was unplug the printer and plug it back in. (Yes, OBVIOUSLY, one of the first things I tried was turning it off and back on. Stupid "soft" power switches!)
Anyhow, printed at about 1/4 page it fits snugly and should be there for me in another 7 years, for another project.
Before that, though, it'll speed up this project, slightly, and "idiot-proof" it a bit, and reduce the dangers.
BUT
It's still quite a process to pull it, change jumpers, put it in the programmer, and actually the process is tremendous...
So, yes, immediately after the first successful test and the noticeable albeit slight improvement, I started thinking, again, about in-circuit programming.
Heh.
This is no longer the "Wild West Of Computing"... we have fewer of the sorts of hurdles they had to hack-around... (Though, we have hurdles of our own). I'm not afraid to use some of our newer tools. 5V-only Flash-ROMs exist, now. OK? Right? So, Hey, Me: save yerself some trouble, dummy!
Right. So *all* I need is a means to upload the new firmware image, then the unit can program that to itself. Presto!
So, the whole reason this interests me, again, is because large/untested code-changes are *extremely* daunting to me.
And... what would an in-system firmware-downloader/flash-programmer be...? A large/untested code-change.
I [kinda] need this thing to be able to make this thing. Chicken-Egg.
I think I ran into the same with #Vintage Z80 palmtop compy hackery (TI-86) ... Heh!
Anyhow, not a show-stopper, but a bit ironic.
...
The other idea, e.g. using an AVR, putting the Z80 in bus-release, well, it has a slew of other such chicken-egg issues, many similar (e.g. parsing ihex files)... But, long-run, I guess, stand-alone in-circuit-programming is the best option... So, I guess I just need to bite the bullet.
So I guess we're talking a jumper/switch to boot to a higher address in the flash, loading the flash-writing utility into RAM (almost forgot that!), listening to the serial port for an ihex stream, parsing that into a byte-stream to be written, telling the sender to pause (flow-control...), writing the data (does this chip require sector-writes?), repeat.
Sounds easy! Except, of course, few of these things are written yet, and they... EACH... should be tested along the way...
And boy-howdy, wouldn't it be nice to have an in-system programmer to develop/test those?
Heh!
Hah! Was bouts to say "well, some of that can be tested on a computer, e.g. the ihex-parser"... but, hah! We don't [yet] have C for this thing. What'm I gonna do, run a z80 emulator? I wouldn't even know where to start!
Oooh, maybe the TI-86?
*head-crash*
...
Anyhow, it's just funny...
Happy Fourth!
....
Third reread: DUH. Period.
Hey this Z80 stuff is fun, but I'm all about code-portability and tools that can be reused...
AVR, now, seems the best way to go. A system like that could be chip-clipped to *many* different processors... And I have a box full of various boards with 8bit CPUs I might like to hack similarly (e.g. boards from an old modem, a C64 floppy drive, some custom 8085 SBC, even an old CD-player).
OK, so... Chip-clipping the CPU is really only necessary for bus-release and reset. Address/Data/Rd/Wr, etc. could all be clipped directly to the ROM... Since the whole point, then, is to replace the original EPROM with flash, it makes a heck of a lot of sense to chip-clip (or otherwise intercept signals) from there, than the CPU...
At which point a small board, similar to the one I already have (jumpers and all) wouldn't be a bad idea, maybe a header for a ribbon-cable to the AVR... And a couple pins with wires and test-clips from there to the CPU.
The board, itself, then, drops into the ROM socket. The AVR can be disconnected once standalone ISP is running...
And THAT might be the best part of this idea... Having written the ihex parser, flash-programmer, etc. in C, already, for the AVR, it'd be portable to the other CPUs once we get C for them.
Hmmm....
And *that* brings me full-circle to my old idea of trying to find a universal/minimal set of assembly instructions and registers... Hmmmm....
-
Another workbench!
06/27/2022 at 20:32 • 0 commentsFlat surfaces are at a premium in my lab... But have more than doubled in the last few weeks!
Would you believe there's a dev-compy behind and to the left of the executive swivel-chair? It's a sleek little machine, top of its line only a few years ago, portable, self-contained, and highly-cusomized for my needs.
In fact, there's now a 270degree work surface to be swivelled-to... Office supplies, pens, scissors, etc. on my right. Universal chip-programmer, and its dedicated computer to the left of that. SD70, next, and a printer behind it. Next is the surface for the sorting-tray when setting pieces and tools aside. It serves double-duty for cable-management and power-distribution. And its addition to the lab brought along with it a nice cat-hangout with in-floor A/C for these hot days. To the left of that is a toolbox. And finally that slick little dev-computer I mentioned, along with space for marking-up assembly-listings, drawing-up blueprints and schematics, and such. All just a swivel-away!
Oh, and I almost completely forgot the workbench behind the compy is dedicated to projects that are "backburnered". Just a chair-slide away to try out whatever new idea may've popped in my head during a sip of coffee.
And behind that (not in photo) is a 10ft wide and 8ft tall opening view window! Oh, right, and a dedicated executive parking-spot merely ten feet away!
Oh, that's just the start! I was talking about work-surfaces and completely forgot to mention the storage! A whole wall of small parts and drawers for tools. Another for bigger things like project-boxes, power-supplies, and motherboards. A third for more of that, lesser-used test-equipment, reference-manuals, and other paperwork/documents.
It's no longer just a vision, it is a reality! It's actually pretty awesome, if you can just see it in there...
(Where'd that crazy operations-tech disappear to now? Right... assembly-listings...)
...
Oh no...
Another project I'd been pondering led me down a rabbithole which gave me a very strange idea that, actually, is even stranger in that I hadn't thought of it before....
Yahsee, my family's very first computer (which, really, was for me) was a "Macintosh Performa 640CD *DOS Compatible*" Yep, you read that right... it had a daughterboard with a 486 CPU. A "DOS-Compatibility Card." I'd thought long and hard about whether we should spend the same amount for a faster PowerPC, or whether it'd be more worthwhile for me to be exposed to both Apples and PCs... And, well, the thought was that PPCs were *so* new, at the time, that nothing would really *need* it for several years.
Anyhow... It just now, via that other completely-unrelated project's research rabbithole, occurred to me that I could use *that* computer to run my chip-programmer. Heh!
Let's talk about *that* rabbithole, eh? It doesn't have a parallel port, nor ISA slots... But, yahknow, I think I've learned enough about the 486 chip, ISA, and parallel ports to actually be able to wire one in. HAH!
But, really, that'd be quite a thing, 'cause that poor DOS card was the major selling-point, but I hadn't yet realized that my interest in computers was mostly about I/O. So, it barely got used, until it was old-enough that I got a real 486 hand-me-down that got all the love the family-compy should've. Hmmm...
Ah, right... How to transfer files? What'm I gonna hook a USB Floppy or CD-burner to my phone? Whew! Nipped that one in the bud!
GAH, but wait! A) Sure, why not? [FAT12, maybe). B) CF-cards are IDE. C) I'll've already tapped into ISA. D) Most the remaining ROM-changes aren't gonna come off the phone. At which point E) Ethernet is a plausibility... Heh!
Power. Seriously. That 486-lappy surely *sips* juice in comparison. OK, bud-nipped.
But, some hack along those lines might be quite the thing some day... It's sad seeing the machine I cut my teeth on (and my dad worked so hard to save up for) collecting dust!
-
Coding Abounds!
06/27/2022 at 08:19 • 0 comments@ziggurat29 has been spending much of his spare time this past week+ coding at the lowest-level for a machine he's never even seen, except in photos.
I had a moment a while-back feeling it a bit like the olden days I've caught-wind of recently from the datasheets of old mask-ROM microcontrollers I've found in my PCB-scrap-bin. I came across at least one that supplied an empty table, in the back pages, that purchasers could fill with the raw hex values of their program, then mail to the manufacturer to do the actual ROM-writing. Heh, sneakernet, eat your heart out. We don't need to deal with the multitude of different floppy formats you might send-in! And, No. We can't supply you with an assembler to run at your own workplace, but if you'd like to dial-in to time-share on our central computer for a fee only huge corporations could afford, you can use the assembler we use here. They've got that, but, no mention of uploading ROM images. And, again, an empty table on several pages of paper.
Heh. What an era.
Alright, well, he's not looking up raw hex values... But still... "write it out, send it in."
For a minute there I also thought it a bit like NASA uploading new firmware to a probe, in order to overcome whatever weird new situation it faces... But then I realized, heck, I bet even in that extremely "out there" situation, NASA's programmers almost certainly have a duplicate in a lab that they can debug on.
A couple days after my thinking this, he suggested pretty much the same idea:
Comparing it to e.g. the way-back when a university might have a single computer, and the research-professors would send their stacks of punchcards to be processed. Their having to write the entire program, with no debugger and no step-by-step testing. Then waiting, possibly a few days, while a queue of others' code was executed, before hearing the result... And, then, possibly finding out they forgot to punch a hole. Heh! Or the instruction-set documentation *seemed* crystal-clear, while in-fact being ambiguous... Or who knows what all.
...
Anyhow, it's quite something.
In the times when I await the next stack of punch-cards, I'm running-around trying to maintain the central systems, over here...
E.G. The "punch card reader" (my beloved 486 laptop and my also-beloved parallel-port chip-programmer) is in sore need of a recalibration or two. Also, there's word in the industry of an upgrade that might remove a step or two... (Did you know FAT32 didn't come out until Win95 OSR2? And apparently Android doesn't do FAT16, heh! Yeah, this whole thing can run off a Win98 boot floppy, but then I need PCMCIA support in DOS, which I've never done... So... choices, choices...).
Anyhow, Central Systems also offers its remote clients limited debugging-support; Our maintenance/operations-tech has become somewhat-familiar with The Central Computer's Instruction-Set, and can spot a few common mistakes, increasing deugging turnaround-times from days to, well... days. Heh!
("increasing" "de-ugging"... hah! If Freud were here...!)
...
The cat says it's time for bed.
Anyhow, in short, we've got quite a few of the peripherals going... Turn it on at the stroke of midnight and you've got a clock in a rackmount! (Actually, maybe it's more of an instrument-case?)
And the "DART" (back then they hadn't yet standardised on "UART") is coming "soon" (where'd that operations-tech run off to, anyhow?), so we can attach a terminal!
Discussion for the future has gone as far as developing a C-Runtime Library and porting BASIC. And the system has been likened to "Heck yeah! Who doesn't want a rackmount TRS-80 in their garage?!"
...
The deeper we get into the system itself, well... I joked the other day along the lines of:
"A physicist, a chemist, and an electrical engineer walk into a bar... OK, so what *really* happened was the EE walked into the bar while looking at his calculator. The physicist and chemist, well, they saw it coming and thought it'd be funny to watch, not realizing they'd have to take-on the EE's duties while he was in the hospital during the month before their deadline."
Yeah, I'd say "it's that bad" (almost as bad as that joke's delivery/punchline). But, it's actually quite interesting, from my end, to see how *well* it works, despite its implementation: "Good practices" that, frankly, I had never tried (nor had seen) a system like this go without. Maybe their importance is greatly-exaggerated in my field?
I might do another post on those details, later.
But, with every new "stack of punchcards," I have to remind myself of many previously-observed factors. Like: that big Reset button on the front has been necessary, on many occasions dating back to its unboxing, for a proper boot after power-up. Or that my logic-probe picks up beep-inducing electromagnetic radiation before even touching pins. (Never did *that* before!)
That I discovered, actually, when I sat it on the laptop keyboard, which was not only powered-off, but also switched-off at the power-strip feeding its isolated power-brick. WOW, it radiates measurably by a *logic* probe, nearly six inches away! I'm near certain that's due to my "Modified Sine" inverter... which is little more than a fancy way of saying a square-wave that goes positive and negative. Pretty sure most things don't really like that... those high harmonics, and likely overshoot and ringing at the edges. My oscilloscope absolutely hates it:
[Forgot to mention in the vid, the probes are shorted to the scope's GND!]
So, debugging involves a bit more than merely checking whether it does what was intended in the "punchcards."
But we've come a long way despite the hurdles!
[Hey, Tech, quit yer yappin' and get back to debuggin'!]
-
"We" have hacked it!
06/23/2022 at 00:28 • 3 comments@ziggurat29 gets all the credit. But I got to be the first to see it running, and burned/swapped the ROM, so that counts for something, right?
Awesome work, my friend!
-
"How do I assemble it?"
06/22/2022 at 10:06 • 2 commentsAdmittedly, my first thought is probably far more realistic than I thought upon thinking it...
But my second thought was something along the lines of "is there an open-source assembler? [Ziggurat29 had issues with sdcc [but for an 8080]... Am I gonna have to use a CP/M-based assembler?"
Heh... Then thoughts went to, "BAH! I can use ZAC!"
Yeah, we're talkin:
1) Write the new SD70 code on my TI-86.
2) Assemble it, under ZAC, on the TI-86
3) Transfer that to compy
4) Convert it somehow to an EPROM image
5) Burn EPROM
LOL.
It makes a heck of a lot of sense, no? Use the tools I know!
(As crazy as it may sound, I *did* actually think of some code I wrote on the TI-86 which could be handy here... what was it? Oh, the flash-programming code... hmmm)
OK, that's a bit ridiculous, assembling code for this on my calculator... But it really made sense at the time!
Heh, maybe that flash-programming bootloader-ROM I mentioned earlier should use TI-Graphlink instead of Zmodem?
*Sigh* brain...