• New router

    Stuart Longland05/19/2018 at 09:39 0 comments

    So before going on the trip, I noticed the router I was using would occasionally drop off the network.  The switch still reported the link as being up, but the router would not respond to pings from the internal network.  If I SSHed into it from outside the network, and tried pinging internal IPs, it failed to ping them.

    Something was up.  After much debugging (and some arguments about upgrades), it was decided that the hardware was flakey.  In that discussion, it was recommended that I have a look at PC Engines' APU2 single board computer.

    This is the only x86 computer I have seen with schematics and CoreBoot out-of-the-box, and it happens there's a local supplier of them.  For sure, this machine is overkill for the job, but it ticks nearly all the boxes.

    The only one it didn't tick was being able to run directly from the battery.  As it happens, the unit only draws about 1.5A, and so a LM1085-12 LDO which can be sourced locally did the trick.  I basically put 100µF capacitors on the input and output, bolted it to a small heatsink and threw it all into a salvaged case.

    After hooking it up to a bench supply (disconnected from the APU2) and winding the voltage right up to the PSUs maximum, and observing that the voltage stayed at 12V, I decided to hook it up and see how it went.  I plugged in my null modem cable, and sure enough, I was staring at CoreBoot.

    I PXE-booted OpenBSD 6.3 and installed that onto the SD card, this was fairly painless and before long, the machine was booting on its own.  I copied across the configuration settings from the old one, set up sniproxy, and I was in business, it was time to issue a `shutdown -p now` to both machines and for them to swap places.

    Of course, a nicety of this box is there's three Ethernet ports, so room for a move to another Internet connection, such as the HFC we're supposed to be getting in this part of Brisbane (sadly, no thin pieces of glass for us), so in theory, I can run both in parallel and migrate between them.

  • arm-unknown-linux-musleabi… saga part IV

    Stuart Longland05/04/2018 at 22:54 0 comments

    So, at long last, I finally saw this in my chroot's /var/log/emerge.log:

    1524887925: Started emerge on: Apr 28, 2018 03:58:45
    1524887926:  *** emerge --oneshot sys-devel/gcc::musl
    1524888211:  >>> emerge (1 of 1) sys-devel/gcc-7.3.0 to /
    1524888212:  === (1 of 1) Cleaning (sys-devel/gcc-7.3.0::/root/musl/sys-devel/gcc/gcc-7.3.0.ebuild)
    1524888307:  === (1 of 1) Compiling/Packaging (sys-devel/gcc-7.3.0::/root/musl/sys-devel/gcc/gcc-7.3.0.ebuild)
    1525472690:  === (1 of 1) Merging (sys-devel/gcc-7.3.0::/root/musl/sys-devel/gcc/gcc-7.3.0.ebuild)
    1525472838:  >>> AUTOCLEAN: sys-devel/gcc:7.3.0
    1525473358:  === (1 of 1) Post-Build Cleaning (sys-devel/gcc-7.3.0::/root/musl/sys-devel/gcc/gcc-7.3.0.ebuild)
    1525473358:  ::: completed emerge (1 of 1) sys-devel/gcc-7.3.0 to /
    1525473360:  *** Finished. Cleaning up...
    1525473373:  *** exiting successfully.

    That's 6 days, 18 hours and 32 minutes, of solid compiling. BUT WE GOT THERE!

    What's left? This:

    Calculating dependencies... done!
    [ebuild     U  ] sys-libs/musl-1.1.19 [1.1.18]
    [binary   R    ] sys-libs/zlib-1.2.11-r1
    [binary   R    ] app-arch/xz-utils-5.2.3
    [ebuild     U  ] sys-libs/ncurses-6.1-r2 [6.0-r1]
    [binary   R    ] sys-libs/readline-7.0_p3
    [binary   R    ] virtual/libintl-0-r2
    [binary   R    ] dev-lang/python-exec-2.4.5
    [binary   R    ] virtual/libiconv-0-r2
    [binary   R    ] sys-apps/gentoo-functions-0.12
    [binary   R    ] dev-libs/libpcre-8.41-r1
    [binary   R    ] sys-apps/sed-4.2.2
    [binary   R    ] app-arch/bzip2-1.0.6-r8
    [binary   R    ] dev-libs/gmp-6.1.2
    [binary   R    ] app-shells/bash-4.4_p12
    [binary   R    ] sys-apps/file-5.32
    [binary   R    ] sys-devel/gnuconfig-20170101
    [binary   R    ] dev-libs/mpfr-3.1.6
    [binary   R    ] app-misc/c_rehash-1.7-r1
    [binary   R    ] app-misc/mime-types-9
    [binary   R    ] app-arch/tar-1.29-r3
    [binary   R    ] app-arch/gzip-1.8
    [binary   R    ] dev-libs/mpc-1.0.3
    [binary   R    ] sys-devel/gcc-config-1.8-r1
    [binary   R    ] app-misc/editor-wrapper-4
    [binary   R    ] sys-apps/less-529
    [binary   R    ] sys-apps/debianutils-4.8.3
    [binary   R    ] net-libs/libmnl-1.0.4
    [binary   R    ] sys-libs/libseccomp-2.3.2
    [binary   R    ] dev-libs/popt-1.16-r2
    [binary   R    ] sys-libs/e2fsprogs-libs-1.43.6
    [binary   R    ] sys-devel/binutils-config-5-r4
    [binary   R    ] dev-libs/libffi-3.2.1
    [binary   R    ] virtual/libffi-3.0.13-r1
    [binary   R    ] sys-apps/sysvinit-2.88-r9
    [binary   R    ] sys-apps/opentmpfiles-0.1.3
    [binary   R    ] virtual/tmpfiles-0
    [binary   R    ] app-text/manpager-1
    [binary   R    ] sys-libs/cracklib-2.9.6-r1
    [binary   R    ] sys-apps/install-xattr-0.5
    [binary   R    ] app-editors/nano-2.8.7
    [binary   R    ] app-portage/elt-patches-20170815
    [binary   R    ] sys-devel/m4-1.4.17
    [binary   R    ] app-arch/unzip-6.0_p21-r2
    [binary   R    ] sys-devel/autoconf-wrapper-13
    [binary   R    ] sys-devel/bison-3.0.4-r1
    [binary   R    ] sys-devel/flex-2.6.4-r1
    [binary   R    ] dev-libs/libltdl-2.4.6
    [binary   R    ] sys-devel/automake-wrapper-10
    [binary   R    ] app-text/sgml-common-0.6.3-r6
    [binary   R    ] dev-libs/libgpg-error-1.27-r1
    [ebuild  N     ] dev-lang/perl-5.24.3-r1  USE="-berkdb -debug -doc -gdbm -ithreads"
    [ebuild  N     ] sys-kernel/linux-headers-4.13  USE="-headers-only"
    [ebuild  N     ] virtual/perl-Data-Dumper-2.160.0-r1
    [ebuild  N     ] virtual/perl-Test-Harness-3.360.100_rc-r3
    [ebuild  N     ] perl-core/File-Temp-0.230.400-r1
    [ebuild  N     ] virtual/perl-File-Temp-0.230.400-r5
    [ebuild  N     ] perl-core/File-Path-2.130.0
    [ebuild  N     ] virtual/perl-File-Path-2.130.0
    [binary   R    ] virtual/os-headers-0
    [ebuild  N     ] sys-devel/autoconf-2.69-r4  USE="-emacs"
    [ebuild  N     ] sys-apps/attr-2.4.47-r2  USE="-nls -static-libs"
    [ebuild   R    ] sys-apps/coreutils-8.28-r1
    [ebuild     U  ] app-admin/eselect-1.4.12 [1.4.8]
    [ebuild     U  ] app-eselect/eselect-python-20171204 [20160516]
    [ebuild     U  ] sys-devel/patch-2.7.6-r1 [2.7.5]
    [ebuild  N     ] sys-apps/shadow-4.5  USE="cracklib xattr -acl -audit -nls -pam (-selinux) -skey"
    [binary   R    ] virtual/shadow-0
    [ebuild  N     ] virtual/perl-ExtUtils-MakeMaker-7.100.200_rc-r4
    [ebuild  N     ] sys-libs/libcap-2.24-r2  USE="-pam -static-libs"
    [ebuild  N     ] dev-perl/Text-Unidecode-1.270.0
    [ebuild  N     ] dev-perl/libintl-perl-1.240.0-r2
    [ebuild  N     ] sys-apps/help2man-1.47.4  USE="-nls"
    [ebuild N ] sys-devel/automake-1.15.1-r2...
    Read more »

  • Next steps, better control of the charger

    Stuart Longland04/28/2018 at 05:30 0 comments

    So, a few weeks ago I installed a new battery charger, and tweaked it so that the solar did most of the leg work during the day, and the charger kept the batteries topped up at night.

    I also discussed the addition of a new industrial PC to perform routing and system monitoring functions… which was to run Gentoo Linux/musl.  For now, that little PC is still running Debian Stretch, but for 45 days, it was rock solid.  The addition of this box, and taking on the role of router to the management network meant I could finally achieve one of my long-term goals for the project: decommissioning the old server.

    The old server is still set up with all my data and software… but now the back-up cron job calls /sbin/poweroff when it's done, and the BIOS is set to wake the machine up in the evening ready to receive a back-up late at night.

    In its place, a virtual machine clone of the box, handles my email and all the old functions of that server.  This was all done just prior to my father and I leaving for a 3 week holiday in the Snowy Mountains.

    I did have a couple of hiccups with Ceph OSDs crashing … but basically re-starting the daemons (done remotely whilst travelling through Cowra) got everything back up.  A bit of placement group cleaning, and everything was back online again.  I had another similar hiccup coming out of Maitland, but once again, re-starting the daemons fixed it.  No idea why it crashed, that's something I'll have to investigate.

    Other than that, the cluster itself has run well.

    One thing that did momentarily kill the industrial PC though: I wandered down to the rack with a small bus-powered 2.5" HDD with the intent of re-starting my Gentoo builds.  This HDD had the same content as the 3.5" HDD I had plugged in before.  I figured being bus powered, I would not be dependent on mains, and it could just chug away to its heart's content.

    No such luck, the moment I plugged that drive in, the little machine took great umbrage to the spinning rust now vacuuming the electrons away from its core functions, and shut down abruptly.  I've now brought my 3.5" drive and dock down, plugged that into the wall, and have my builds resuming.  If power goes off, hopefully the machine either handles the loss of swap gracefully.  If it does crash, the watchdog will take care of it.

    Thus, I have the little TS-7670 first attempting a build of gcc, to see how we go.  Finger's crossed our power should remain up.  There was at least one outage in the time we were away, but hopefully we should get though this next build!

    The next step I think should be to add some control of the mains charger to allow the batteries to be boosted to full charge overnight.  The thinking is a simple diode-OR arrangement.  Many comparators such as the LM393 have an open-collector output, which gives us this for free.

    The theory is this.

    The battery bank powers a simple circuit which runs of a 5V regulator.  That regulator powers a dual comparator IC and provides a reference voltage.  The comparator draws bugger all power, so I'm happy to use a linear PSU here.  It's mainly there as a voltage reference.

    Precision isn't really the aim here, so adjustable pots will make life easier.

    The voltages from the battery bank and the solar panel are fed through voltage dividers to bring the voltages down to below 5V, then those voltages are individually fed into separate pots that control the hysteresis.  I can adjust all points of the system.

    The idea is that should the batteries get too low, or the sun go down, one or the other (or both) comparators will go low and pull down on R2.  If the batteries are high and the sun is up, nothing pulls on R2 so the REMOTE+ pin on the HEP-600C-12 is allowed to float to +5V, turning off the mains charger.

    The advantage of this is there's no programming of a microcontroller, it's just analogue electronics.  The LM393s are pretty hardy things, the datasheet...

    Read more »

  • Fine tuning

    Stuart Longland03/31/2018 at 23:31 0 comments

    So yesterday I wound back the mains charger so that the solar would take on the load during the day.  Seems I wound it back a bit far, and the mains charger did almost no work overnight, leaving the battery somewhere around 11.8V.

    That's a wee bit low for my comfort.  Yes, they are deep cycle AGMs, but I'd rather not get that low.

    Thus, I wound it up a bit, float at 12.8V, so Vboost at 13.6V.  That looks to be the sweet spot.  Now that the sun is up, I'm getting nice healthy amps of current down the wire from the roof:

    The cluster is drawing about 8A, so that's the cluster powered, and about 6A going to the batteries.  It intermittently peaks about 15A or so.

    I also found myself fine tuning the Ethernet settings on the border router.  For some reason, its Realtek RTL8139 was happy to talk to the Cisco SG-200-08 it was connected to before, but didn't quite get along with the Linksys LGS326-AU.  I've told the switch to force 100Mbps full-duplex MDIX (evidently, it's a cross-over cable), and so far, that seems to have settled things down.

  • Coaxing two supplies to get along

    Stuart Longland03/31/2018 at 02:56 0 comments

    So last post, I mentioned about the installation of the new battery charger, which is fed from 240V mains.  Over the last few days this charger has held the batteries at a rock-solid 14.4V.  Not once did the batteries drop below that voltage setpoint.

    So good in fact, the solar charger does no work at all.

    By the way, this is what the install looks like.  I promised pictures last post.

    That's the DC end … and the nasty AC end is all sealed up…

    I will eventually move this to a spot on the back of the rack, but it can sit here for now.

    Ultimately, the proper fix to this will be to have the mains-powered charger power off when the sun is up.  On the DC output connector, the two rightmost screw terminals go to an opto-isolator that, when powered, shuts off the charger, putting it into stand-by mode.  This was one of the reasons I bought this particular unit.  The other was the wide range of voltage adjustment.

    The question is when to turn on, and when to go to stand-by.  Basically if the following expression is true, then turn off the mains:

    We do not want solar if the battery is very low, as there's a possibility that the solar output will not be sufficient.  Likewise, if the sun's out, we need the mains to keep the battery topped up.

    The solar output is nearly always above 15V when the sun is up, so there's our first clue.  We can safely get to 12.8V before things start going pear shaped on the cluster, so we can use that as our low-voltage safety net.  If both of these conditions are met, then it's safe to turn off the mains power and rely on solar only.

    We need a +5V signal when both these conditions are met.  This very much sounds like the job of a dual-comparator with diode-OR outputs pulling on a 5V pull-up.  Maybe a wee bit of hysteresis on those to prevent flapping, and we should be good.

    Unfortunately, to do that, I need to unscrew terminals to feed some wires in.  I don't feel like doing that just now… we're packing up to go away for a while, and I think this sort of job can wait until we return.

    In the meantime, I've done something of a hack.  I mentioned the PSU is adjustable.  I wound Vfloat back to 12V… thus Vboost has gone to 12.8V.  Right now, the mains PSU is showing a green LED, meaning it is in floating mode.

    We have good sun right now, and the solar controller is currently boosting the battery.  When the battery gets low, the charging circuitry of the mains PSU should kick in, and bring the battery voltage up, holding it at 12.8V until the sun comes up.  I'll leave it for now and see how this hack goes.

    On other news… I might need to re-consider my NTP server arrangements.  I'm not sure if it's a quirk of OpenBSD, or of the network here, but it seems OpenNTPD struggles to keep good time.  Never tried using the Advantech PC as a NTP server until now, and I'm also experimenting with using my VPS at Vultr as a NTP server.

    http://www.pool.ntp.org/user/Redhatter

    Both are drifting like crazy.  I have a GPS module lying around that I might consider hooking up to the TS-7670… perhaps make it a Stratum 1 NTP server on the NTP server pool, then the Advantech can sync to that.

    This won't help the VPS though, and I'm at a loss to explain why a Geode LX800 running on an ADSL link in my laundry, outperforms a VPS in a nicely climate-controlled data centre with gigabit Internet.

    But at least now that's one less job for my aging server.  I've also moved mail server duties off the old box onto a VM, so I'll be looking at the BIOS settings there to see if I can get the box to wake up some time in the evening, let cron run the back-up jobs, then power the whole lot back down again, save some juice.

  • New power supply wired up and installed

    Stuart Longland03/28/2018 at 13:48 0 comments

    So tonight I finally got my shiny new power supply installed.

    Tuesday night I took it with me along with a cable gland to HSBNE with three items on my agenda:

    • Hooking up a mains power lead to the power input.
    • Getting the newly hooked up lead inspected for electrical safety.
    • Putting some sort of cover over the screw terminals to prevent accidental contact.

    I did some digging around in the HSBNE bone yard, and managed to come out with a 10A 240V mains lead, the chassis of a Sonoff TH-10, and a bit of off-cut perspex from the laser cutter to cover the gaping hole in the TH-10 casing.

    The 240V mains lead came from someone's long abandoned project.  Not sure what it was, but it basically was housed in take away food containers, so losing its mains lead is probably a good thing!  The rest of it is there if they want it … whatever it is.

    I terminated the 240V lead with fork lugs, ready to go into the screw terminals on the power supply.  A small square of perspex was cut out of the off-cut, and that was sliced into three parts to be glued to the TH-10 case.

    The back panel of the TH-10 case had an opening cut in it to allow the screw terminal block to pass through the back.  One of the pieces of perspex had a 14mm hole drilled through it and the cable gland was fitted.  All that was left to do was some hot glue to fix the perspex panels into place over the hole, attach the mains lead and get it checked.

    Sadly, I couldn't find anyone about with an electrical ticket to actually install the cable, so I did that bit myself in the end.  There also weren't any glue sticks for the glue gun around, and I still had to think about how I was going to secure the TH-10 case to the aluminium of the PSU.

    I brought it into my workplace this morning and got one of the people there to check it over (there's two at my work who have a current electrical ticket).  My cabling job was given the tick of approval, and as a bonus, we had some silicon glue which could fix the TH-10 case to the aluminium panel on the PSU.  Perfect, two birds with one stone.

    Once home, I set to work on the 12V end of it.  I needed to go from 4 smallish screw terminals to an Anderson SB50 connector which was intended for 8AWG cable.  In the end, the solution was to use two lengths of twin-12AWG.  One end was terminated with fork lugs, the other was twisted together and soldered into a SB50 connector.  I had to solder it because even doubled over, it was too thin to crimp into the pins securely.

    I used about 10cm of 12AWG.  To that SB-50 I made a patch lead with two SB-50s out of figure-8 8AWG cable, about 50cm long to reach the charger input on the battery bank.

    I'll put some pictures up later, but already the silence of this new charger is deafening.  It happily boosted the batteries up to 14.3V and is now letting them sit in constant voltage mode.

    We shall see what happens when the sun comes up tomorrow.  Hopefully it just backs right off and lets mother nature do all the work.

  • ADSL powered by the sun

    Stuart Longland03/24/2018 at 04:52 0 comments

    So, I've now moved the ADSL and router onto the battery supply.  This has added an extra amp of load, but really, the solar panel handles this easy.

    I dug up one of my spare switchmode PSU modules and then got to thinking about how I'd mount the thing.  In the end, double sided tape… to keep the terminals of the adjustment pot from shorting, to a piece of old copper clad PCB from the project graveyard, with some wires soldered on.

    The donor PCB already had regions cut out for terminals around the edge, so I could use those for drilling mounting holes.  I just made additional terminal pads for soldering the input and output supply rails.  Initially I tried putting a 1mF capacitor across the output, but evidently the one I grabbed was crook as it presented a 10ohm load.  I don't think the cause was due to it charging.  The PSU has a 220µF there already, so let's see how it fares.

    Fairly simple, +12V comes in via the orange wire into IN+, the "LM2596" steps that down to 5V, comes out the red wire.  Screw terminals allow me to swap input and output.

    Before hooking it up to the ADSL modem, I made sure to dial it in to 5V.

    Meh… who's going to care about 3mV. :-)

    As it happens, the original PSU puts out 5.3V.  I think I'm closer.  I can always dial it up if needed.

    I put the lid on the case and made up the rest of my wiring harness.  One 5A blade fuse, a bit of work around the back of the rack, and it was installed.

    In the meantime, I have my old server busy pushing its last daily back-up across to a newly provisioned virtual machine on the cluster.

    One problem this presents is that this one VM occupies about 70% of my usable storage cluster capacity.  The cases can take one 2.5" HDD, which unless you're willing to risk it with Seagate (I've had too many of them fail), top-out at 2TB.

    There are SSDs too, but I'm not made of money, and I've already spent the cost of a small car on this cluster as it is.  My thinking is I might look at modifying the cases with a new lid to accept a 3.5" HDD.  If I make the case a wee bit taller, a 3.5" HDD would fit in the lid, and I could add fans around it to cool it.

    The other option is to make external eSATA 3.5" DIN-rail mounted cases.  I did look online, but didn't see any for sale.  That said, space is getting squeezy on that DIN rail, and I do have to be mindful of cooling.

  • Re-locating the ADSL service

    Stuart Longland03/23/2018 at 11:30 0 comments

    So last week, I came home to no power, which of course meant no Internet because the ADSL service is still on mains power.

    This is something that's been on my TO-DO list for a while now, and I've been considering how to go about it.

    One way was to run 12V from the server rack to the study where the ADSL is.  I'd power the study switch (a Cisco SG-208), the ADSL modem/router (a TP-Link TD-8817) and the border router (an Advantech UNO-1150G).

    The border router, being a proper industrial PC is happy with any voltage between 9 and 32V, but will want up to 24W, so there's 2A.  The ADSL modem needs 5V 1A… easy enough, and the switch needs 12V, not sure what power rating.  I'm not sure if it'll take 15V, I'd be more comfortable putting it on an LDO like I did for the Linksys switch and the cluster nodes.  (Thanks to @K.C. Lee for the suggestion on those LDOs.)

    With all that, we're looking at 3-4A of current at 12V, over a distance of about 5 metres.  The 6 AWG cable I used to hook panels to solar controller is obviously massive overkill here, but CAT5e is not going to cut it… it needs to be something around the realm of 12 AWG… 20 at the smallest.

    I have some ~14AWG speaker cable that could do it, but that sounds nasty.

    The other approach is to move the ADSL.  After finding a CAT3 6P4C keystone insert, I dug out some CAT5e (from a box that literally fell off the back of a truck), slapped my headlamp onto my hard hat, plonked that on my head and got to work.

    It took me about an hour to install the new cable.  I started by leaving the network-end unterminated, but with enough loose cable to make the distance… worked my way back to the socket location, cut my cable to length, fitted the keystone insert, then went back to the ADSL splitter and terminated the new run.

    There was a momentary blip on the ADSL (or maybe that was co-incidence), then all was good.

    After confirming I still had ADSL on the old socket, I shut down the router and ADSL modem, and re-located those to sit on top of the rack.  Rather than cut new cables, I just grabbed a power board and plugged that in behind the rack, and plugged the router and modem into it.  I rummaged around and found a suitably long telephone cable (with 6P6C terminations), and plugged that in.  Lo and behold, after a minute or two, I had Internet.

    The ugly bit though is that the keystone insert didn't fit the panel I had, so for now, it's just dangling in the air.  No, not happy about that, but for now, it'll do.  At worst, it only has to last another 3 years before we'll be ripping it out for the NBN.

    The other 3 pairs on that CAT5e are spare.  If I want a 56kbps PSTN modem port, I can wire up one of those to the voice side of the ADSL splitter and terminate it here.

    I think tomorrow, I'll make up a lead that can power the border router directly from the battery.  I have two of these "LM2596HV" DC-DC converter modules.  I'm thinking put an assortment of capacitors (a few beefy electrolytics and some ceramics) to smooth out the DC output, and I can rummage around for a plug that fits the ADSL modem/router and adjust the supply for 5V.  I'll daisy-chain this off the supply for the border router.

    We're slated for Hybrid Fibre Coax for NBN, when that finally arrives.  I'll admit I am nowhere near as keen as I was on optic fibre.  Largely because the coax isn't anywhere near as future-proofed, plus in the event of a lightning strike hitting the ground, optic fibre does not conduct said lightning strike into your equipment; anything metallic, will.

    By moving the ADSL to here though, switching to the NBN in the next 12-24 months should be dead easy.  We just need to run it from the junction box outside, nailing it to the joists under the floor boards in our garage through to where the rack is.  No ceiling/wall cavities or confined spaces to worry about.  If the NBN modem needs a different...

    Read more »

  • Beware, I'm ARMed

    Stuart Longland03/17/2018 at 05:37 0 comments

    …but probably not dangerous.
    Last night, I got home, having made a detour on my way into work past Jaycar Wooloongabba to replace the faulty PSU.
    It was a pretty open-and-shut case, we took it out of the box, plugged it in, and sure enough, no fan.  After the saleswoman asked the advice of a co-worker, it was confirmed that the fan should be running.
    It took some digging, but they found a replacement, and so it was boxed up (in the box I supplied, they didn't have one), and I walked out the door with PSU No. 3.
    (Ohh, and note to self, don't EVER try to type a string matching the regex #\d\.$ on this site… it'll pop up a suggestion box of un-related projects and refuse to bugger off to let you start the next paragraph.)
    I had to go straight to work, so took the PSU with me, and that evening, I loaded it into the top box to transport home on the bicycle.
    I get home, and it's first thing on my mind.  I unlock the top box, get it out, and still decked out in my cycling gear, helmet and all (needed the headlight to see down the back of the rack anyway), I get to work.
    I put the ring lugs on, plug it into the wall socket and flick the switch.
    Nothing.
    Toggle the switch on the front, still nothing.
    Tried the other socket on the outlet, unplugging the load, still nothing.  Did the 10km trip from Milton to The Gap kill it?
    Frustrated, I figure I'll switch a light on.  Funny… no lights.
    I wander into the study… sure enough, the router, modem and switch are dead as doornails.  Wander out to the MDB outside, saw the main breaker was still on, and tried hitting the test button.  Nothing.
    I wander back inside, switching the bike helmet for my old hard hat, since it looks as if I'll need the headlight a bit longer, then take a sticky beak down the road to see if anyone else is facing the same issue.
    Sure enough, I look down the street, everyone's out.
    So there goes my second attempt at bootstrapping Gentoo, and my old server's uptime.
    The power did return about an hour or so later.  The PSU was fine, you don't think of the mains being out as the cause of your problems.
    I'll re-start my build, but I'm not going to lose another build to failing power.  Nope, had enough of that for a joke.
    I could have rigged up a UPS to the TS-7670, but I already have one, and it's in the very rack where it'll get installed anyway.  Thus, no time like the present to install it.
    I'll have to configure the switch to present the right VLANs to the TS-7670, but once I do that, it'll be able to take over the role of routing between the management VLAN and the main network.
    I didn't want to do this in a VM because that means exposing the hosts and the VMs to the management VLAN, meaning anyone who managed to compromise a host would have direct access to the BMCs on the other nodes.
    This is not a network with high bandwidth demands, and so the TS-7670 with its 100Mbps Ethernet (built into the SoC; not via USB) is an ideal machine for this task.
    Having done this, all that's left to do is to create a 2GB dual-core VM which will receive the contents of the old server, then that server can be shut down, after 8 years of good service.  I'll keep it around for storing the on-site backups, but now I can keep it asleep and just wake it up with Wake-on-LAN when I want to make a back-up.
    This should make a dint in our electricity bill!
    Other changes…

    • Looks like we'll be upgrading the solar with the addition of another 120W panel.
    • I will be hooking up my other network switches, the ADSL router and ADSL modem up to the battery bank on the cluster, just got to get some suitable cable for doing so.
    • I have no faith in this third PSU, so already, I have a MeanWell HEP-600C coming.  We'll wire up a suicide lead to it, and that can replace the Powertech MP-3089 + Redarc BCDC1225, as the MeanWell has a remote on/off feature I can use to control it.

  • History repeats: another PSU fan bites the dust

    Stuart Longland03/15/2018 at 10:54 0 comments

    Perhaps literally… it has bitten the dust.  Although I wouldn't call its installed location, dusty.  Once again, the fan in the mains power supply has carked it.

    Long-term followers of this project may remember that the last PSU failed the same way.

    The reason has me miffed.  All I did with the replacement, was take the PSU out of its box, loosen the two nuts for the terminals, slip the ring lugs for my power lead over the terminals, returned the nuts, plugged it in and turned it on.

    While it is running 24×7, there is nothing in the documentation to say this PSU can't run that way.  This is what the installation looks like.

    If it were dusty, I'd expect to be seeing hardware failures in my nodes.

    This PSU is barely 4 months old, and earlier this week, the fan started making noises, and requiring percussive maintenance to get started.  Tonight, it failed.  Completely, no taps on the case will convince it to go.

    Now, I need to keep things running until the weekend.  I need it to run without burning the house down.

    Many moons ago, my father bought a 12V fan for the caravan.  Cheap and nasty.  It has a slider switch to select between two speeds; "fast" and "slow", which would be better named "scream like a banshee" and "scream slightly less like a banshee".  The speed reduction is achieved by passing current through a 10W resistor, and achieves maybe a 2% reduction in motor RPM.  As you can gather, it proved to be a rather unwelcome room mate, and has seen its last day in the caravan.

    This fan, given it runs off 12V, has proven quite handy with the cluster.  I've got my SB-50 "load" socket hanging out the front of the cluster.  A little adaptor to bring that out to a cigarette lighter socket, and I can run it off the cluster batteries.  When a build job has gotten a node hot and bothered, sitting this down the bottom of the cluster and aiming it at a node has cooled things down well.

    Tonight, it has another task … to try and suck the hot air out of the PSU.

    That's the offending power supply.  A PowerTech MP-3089.  It powers the RedARC BCDC-1225 right above it.  And you can see my kludge around the cooling problem.  Not great, but it should hold for the next 24 hours.

    Tomorrow, I think we'll call past Aspley and pick up another replacement.  I'm leery of another now, but I literally have no choice … I need it now.  Sadly, >250W 12V switchmode PSUs are somewhat rare beasts here in Brisbane.  Altronics don't sell them that big.  The grinning glasses are no more, and I'm not risking it with the Xantrex charger again.

    Long term, I'm already looking at the MeanWell SP-480-12.  This is a PSU module, and will need its own case and mains wiring… but I have no faith in the MP-3089 to not fail and cremate my home of 34 years.

    The nice feature of the SP-480-12 is that it does have a remote +12V power-off feature.  Presumably I can drive this with a comparator/output MOSFET, so that when the battery voltage drops below some critical threshold, it kicks in, and when it rises above a high set-point, it drops out.  Simple control, with no MCU involved.  I don't see a reason to get more fancy than that on the control side, anything more is a liability.

    On other news, my gcc build on the TS-7670 failed … so much for the wait.  We'll try another version and see how we go.