The HYDRA multiheaded virtual computer

The HYDRA is a computing platform using a small footprint Hypervisor to run 1-6 (or more) hardware assisted virtual machines of any x86 OS

Related lists

2014 Hackaday Prize: Quarterfinalists

Projects that were officially submitted to the 2014 Hackaday Prize

Atmel® AVR® Projects

Atmel® AVR® 8- and 32-bit MCUs deliver a unique combination of performance, power efficiency and design flexibility.

2015 THP: Atmel Parts

Hackaday Prize Entries using Atmel Parts

2015 Hackaday Prize: Quarterfinalists

Projects that were officially submitted to the 2015 Hackaday Prize

Browse related lists

Description

New update in the project logs. Short version: I constructed a new case that addresses airflow & wire mgmt problems in the Chieftec case. The ports are now in the back and grouped to facilitate easy identification. I also wrote an Android App that manages the system completely from the console or other device. Out of chars
THANKS to all who Skulled & followed this project, I think the best way to thank you all is with some NEW STUFF to read and maybe try on your own....

VM's have many advantages over their bare metal counterparts, but the complexity makes these unavailable to most outside the datacenter. I aim to change that.

The VM accesses the video hardware using IOMMU, which allows a PCI device to be "given" to a VM. This Virtualization for Directed IO is what makes transparent virtualization possible.

The Hydra prototype contains 6 video cards, and gives each of these + USB to 6 different OS's. Runs Winx, Linux, OSx, Android, Stea

Details

The goal here is to make this hardware assisted virtualization platform as easy to use as it's bare metal counterpart. Or, the virtalization should be completely transparent to the end user. They should be "in the Matrix", in a virtual world unaware of the virtual nature of their constructed environment, yet endowed with "powers" that allow all kinds of amazing feats formerly thought impossible.

The Xbox One is completely "in the Matrix", with all management removed. Boots 2 VM's & that's it, if they break it's back to Day1 snapshot for you. If you're going to be in the "Matrix" you want to be Neo....

But datacenter level management comes with the expectation that you at least have your head wrapped around the whole virtualization thing.

The HYDRA attempts to strike a balance. ESXi boots from the primary gpu, which gets "stolen" for passthrough during boot at which time the screen freezes. The VM assigned to this GPU boots and takes the screen. Starting other VM's or switching the OS on the primary display needs some interface......

The Hydra was originally developed 1.5 years ago to be a digital menu board system for a sub shop. Since then I have used this system as my primary PC (s) , and have 2 deployed in others use. This allowed me to get feedback and explore alternative, user friendly methods for the tasks a typical end user may encounter, as well as discover all the capabilities.

First try: Installed a vm with only 1 usb passed through for a wifi card, & created a softAP. Wrote an Android app and got a cheapo Android tablet. This allowed power & snapshot operations, even showed the VMware vga interface during boot over VNC, so if you were stuck in "startup repair" you would see it at the tablet. However, wireless deemed unreliable.

2nd try: ESXi has python, so I wrote a menu system that is accessed from the console of the server, or by ssh, or over serial. Created installable VIB patch to easily install the menu. This works very well, power, snapshot, even networking settings are available. But, the end user is required to use a text menu at the console. Should be ok, but as it turns out it's NOT.

I once accidentally assigned the primary GPU for passthrough on my system, the ESXi screen froze halfway booting. This was an HP Riloe II card, but it wouldn't pass through. I tried an NVidia PCI card and could actually assign it to a vm.

3rd try: built mgmt VM including all the tools, simple & complex. Added the softAP. Set to autostart. Works very well, just wastes a GPU....

Redirecting the console to serial seems to be the answer. A microcontroller based serial console could handle all that is needed, as well as maybe perform other functions, such as HDMI KVM switching.

Current mgmt interface: I redirected TTY2 (techsupportmode) to com1. Now we have 6 GPU's for passthrough. Added bonus: modules loading are shown on the terminal during boot. The python menu will run on a laptop with putty, or even a palm3. I can't really require a laptop be attached for mgmt and there's only so many palm3's out there. Under development is a serial terminal menu device based on a ATMEGA328 with a 4x20 LCD. Very similar to:

http://hackaday.com/2008/05/29/how-to-super-simple-serial-terminal/

I am however using other libraries, to get the type of menu I want and support for a jog wheel rather than buttons. It will show 4 VM's with power status at a time, highlighting your selection, and have indicators for more above or below. Only power & snapshot operations would be available, due to limited space. The unchangeable management VM would head the list, and allow full management capabilities. But for daily operations, the 4 line lcd will be an interface end users can deal with.

Like this:

The HYDRA can be configured with 6 video cards, or with only one. Next prototype will be a mini-itx board with 2 GPUs. So a small...

Components

1 × Chieftec BL-01B Bravo Series Case w/VESA Monitor Mount http://www.xoxide.com/chieftec-bravo-case-vesa.html $78 + sh

1 × Asrock X58 Extreme Ebay $129

1 × i7 920 ebay $50

4 × ATI HD6450 HDMI, DVI, VGA Ebay $35

1 × ATI HD4650 Ebay $20

Project Logs

Collapse

10 gpu's passed to VM's in 1 box STUNT
eric • 01/19/2016 at 16:28 • 0 comments

Thought I'd try and jam as many GPU's in 1 box as possible, out of what I have laying around.
The hard part was finding 10 working GPU's that are passthrough capable. I had to move some fans from Geforce cards to Radeons that had stuck fans.
I am using the PLX PEX 8508 8-lane, 5-port PCI Express Switch RDK (PCIe 1x to 4 PCIe 16x) to "fan out" the slots to accommodate all the cards . I bought 2 of these for $25 on ebay (haven't seen that price again) and am using the working one. The other has a fried cap (at least). I might try to fix it....
Also, when you pass PCI cards (not PCIe), you have to pass ALL PCI devices to one VM. So I had to pass 2 FX5200's to one XP VM. (no win7 driver from Nvidia for these, and I'm not putting vista on it). For some reason, although both work individually, I can't get a display on both of them. Maybe it's because the cards are identical, and the Nvidia driver gets confused? I could try an HP lights out board with VGA, it's really a computer by itself.....
So I added some lines to the XP's vmx to hide it's "virtualness", and installed VMWare Workstation. With an xp VM this makes 10 VM's .
Also running Mac OSx & Android Kit Kat, plus 6 Win7's.
Built a case to address airflow & wire mgmt
eric • 12/01/2015 at 22:05 • 0 comments

I guess it's a case mod rather than "scratch built" since I started with this:
HP DC7700 SFF
All the parts
Cannibalized Dell monitor stand, sheet metal from Lowes & Harbor Freight brake
Case fits together
Ports facing up for better airflow & wire management
Modded Supermicro RM PSU
This connects.....
To this for the monitor power
Plexiglas back with grouped HDMI/USB & Ethernet
Directed airflow to GPU's
ALL wiring out the back
Of course, I would do it differently the next time....
Hydra MGMT by Android App
eric • 12/01/2015 at 18:23 • 1 comment

The goal of the Hydra project is to bring the benefits of Datacenter virtualization to the desktop. Without the Datacenter, and without the personnel and associated expertise required to manage it. VMWare's Vsphere Client of course allows for almost complete configuration (Vcenter is required for some things as of 6.0 ), but it's a little too complete to expect all but a few end-users to use. A little curiosity would likely result in a tech support incident. It also requires another system, running Win7 or later, to get an empty Hydra system up and running some useful VM's. So we need an alternate management system.
I tried a few different approaches, a serial connected Arduino device (too much programming) , a Python menu system running at the console (no ability to view a VM screen), and an Android App, The Android App worked pretty good, but I was writing it in MIT App Inventor since I'm lazy and it was easy. The limitations of App Inventor and Android ver 2.0 on the crappy tablet I was using ended development here.
Early last year the Guys at Androidx86 posted an ISO tuned to use Nvidia GPU's. I found it to work on PCIe GPU's as well as the FX5200 I use as the boot GPU. So the Hydra boots VMWare ESXi and the screen switches to Android with a touchscreen(with the ability to plug in a keyboard & mouse) I also took the time to learn a more comprehensive Android development platform - Basic4Android. Which is nearly as powerful as Eclipse, but very easy and fast to learn.
As a result I believe I have hit upon the best method of management.
The otherwise useless console is "stolen" from ESXi, which means no extra hardware.
Android is free, ESXi is free, and can be distributed with the free license (I checked)
B4A allowed me to cover all the management bases, edit VM's, assign PCIe devices, connect to other hosts, and build a new VM from scratch from an inserted CD. This is all from the touchscreen console, or with the app installed on a phone or tablet. Over IP, or serial.
A new Hydra would presumably come with Free ESXi, and an Android VM running the app. The end user inserts an OS cd, and builds a VM with whichever GPU is selected.
Discovering VM's (Flash animation with Hydra playing catch with the Droid)
Found the VM's, their OS by icon & power state by label color. A new Hydra would prompt for host IP, and it can be changed later. The IP button performs operations on the host, such as power etc. DCUI launches the ESXi console we stole, so it's not gone.
Single click to power on a VM (or power off according to vmware tools state) and it launches the boot screen of the VM. Long click allows power on to bios, launch screen independently, and other power operations.
Build a new VM - this is the current screen. I'm working on some more such as cores & sockets. Here I'm building Win8, 2GB, 20G HDD, and assigning an HD6450 and USB 1a1.
Android VM/App is the best management method
eric • 08/03/2015 at 19:40 • 0 comments

VMWare ESXi being a Datacenter Hypervisor, the management available from VMWare is the VSphere Client, which needs to run from another PC. Or from a VM on your host, but you still need another PC to get that up and running. You can do it from the console, but probably not without another PC to Google yourself through it. And the VSphere Client has many options an end-user does not need, and some an end-user should NOT have access to.
If this is ever going to be a end-user friendly system, I needed some other interface. A VM with the VSphere Client requires Windows, and a license.....So I can't "include" a mgmt VM without the whole OEM M$ etc....
So the VSphere Client is of course available from VMWare for those that are comfortable with it. But I needed a simpler management interface that would be always available without extra hardware, needs to communicate via SSH or Serial to send the vim-cmd commands to the Host, and needs to be able to VNC into a VM screen (for building or troubleshooting VM's).
First I was using an HP Lights Out card as the primary adapter. You could remote in and mount ISO's, and power the system on/off. The colors on the DCUI were all messed up, but it worked.
Then I wrote an Android App that would run on a cheapo tablet that I would include. It needed WiFi, so I ran a VM running a soft-AP using a Ralink USB. They don't last very long, not surprising considering the source & price ($4). And when the radio dies, you're stuck with VSphere. But it would VNC into a booted VM, so you could get out of "startup repair" if needed.
Then I wrote a Python Menu, as an installable VIB. This always worked for enumerating existing VM's and controlling their power & snapshots. No VNC from the console, even tried some frame-buffer VNC without success. This code is probably useful somewhere though....
Then I found I could "steal" the primary GPU from the Host, and give it to a VM. It would not work with the ILO card, so it took me a while to discover this. The ESXi screen freezes, and the VM using the GPU appears at the console when booted. No text mode, so you can't troubleshoot a non booting VM from here. (so whatever I run here needs to be RELIABLE) But, this is a PCI (not PCIe) card, and all PCI slots in the Host can only be assigned to 1 VM (so these slots are kind of useless anyway). I have these populated with a Geforce FX5200 & a USB card. I connected a touch screen monitor and am running Android x86 ICS with these 2 PCI cards. Now I can run the Android App from an autostarting VM on every boot. A nice touch screen interface requires no keyboard or mouse. The VM with the App can be "frozen" at a working state and return to the snapshot on every boot. The same App could also be on a phone or tablet the EU already has - or not. I could potentially modify the install ISO to include my App and run it live. I could also put the cheapo Ralink in the free USB and run an Android Tether soft AP in case there is no existing AP. The DCUI can be accessed over SSH, so that's not gone. No M$ license needed either. This seems to be the best EU friendly management interface and I am finally satisfied with it.
Now I'm re-writing the App.....
Oculus Rift working on the Hydra
eric • 07/17/2015 at 18:13 • 0 comments

So, I got an Oculus Rift DK2 cheap off CL. Tried for a solid day trying to get it running on the Hydra Game console, but no go due to the Quadro GPU. The six headed Hydra here uses HD6450's, and it lit right up. Just another peripheral, but 5 Oculus Rifts connected to 5 VM's seems to suggest some very cool 5 way gaming setups. Already have the Leap Motion working (in Ubuntu, with Compiz extensions you can spin the globe with your hand) and will be checking out some Leap + Rift games/content. Any suggestions?
Hydra Gaming Console
eric • 03/23/2015 at 20:03 • 0 comments

I'll add more later, here's the youtube vid:
https://www.youtube.com/watch?v=7cov4LHAduo&feature=em-upload_owner#action=share
Working on Vsphere 6.0 beta
eric • 11/09/2014 at 05:12 • 0 comments

First of all, before the Hydra was eliminated from the space trip contest, my attention was unfortunately entirely focused on income. This has been back-burnered since late July, with not much forward motion since. I didn't really expect to place or show, but entered for the publicity & collaborative interaction it promised. But at this point it's still just a hobby....
I downloaded 6.0 beta with the best of intentions, and it just sat in a folder, until Susie Felt from VMware called me and I asked her about PCI-Passthrough in 6.0, but she could not comment on it as it is still un-released. So I had to try for myself.....
I have been installing Digital Menu Boards for McDonald's lately, and the Hydra certainly could drive those 5 HDTV's. Maybe not for McD's, but there are MANY others who have not converted from cardboard menus yet. This would be a new market for VMware, and their interest would help to accelerate development.
I have been stuck on ESXi 5.0 build 768111 (this is 5.0 with some patches to fix VM auto starting) since ALL later builds break the ability to pass through onboard PCI devices. This made all the USB's unavailable, as well as any disk controllers (if I was inclined to try). 5.0 works, but how can I build a system that requires anything but the latest build?
In 5.1 they would revert back to not enabled for passthrough after the required reboot.
In 5.5 they would persist, and I could assign them to a VM, but the VM would not boot with these devices.
In 6.0 build 2159203 initial testing I have an XP VM using the AMD 4650 add in card, as well as 1 onboard USB. I have no reason to expect anything less than the functionality I had in 5.0 with the patches. VMware has a "bug hunt" contest, so I expect basic features like autostart to work as well.
I will be trying the PCIe switch next.....
Better Video
eric • 08/21/2014 at 07:05 • 0 comments

Mike makes it look so easy...
Overview video
eric • 08/21/2014 at 06:50 • 0 comments

I can make ESXi do tricks but videos still seem to be a challenge....
System Design
eric • 08/21/2014 at 03:30 • 0 comments

This diagram shows the basic components of the Hydra system design. There are many details which need to be addressed as these parts are assembled, but this overview shows all of the components apart from the obvious power supply, case, and RAM to meet your requirements.

View all 17 project logs

Build Instructions

Collapse

Step 1

Gather parts to build a system for your intended purpose:

Research your system board & processor & be double or more sure it is supported. Number of PCIe, PCI should reflect what you want to run. Internal serial HEADER is preferable over DB9 on the back. Avoid onboard video. DL the manual & check in the settings for VT-d, or "virtualization for directed IO" Check the Intel ARK to verify your CPU supports VT-d. Similar process for AMD, no first hand exp here sorry.

Use these links to select known working verified CPU/MB combos :

https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE#gid=0

http://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardware

Check the ESXi 5.0 HCL to make sure the nic on your board is supported, or that there is at least a patch that makes it work. If you need to apply a patch, additionally get a nic card that IS on the HCL so you have the connectivity necessary to apply the patch.

Lots of info here:

https://communities.vmware.com/thread/297072?tstart=0

GPU: ATI HD series are good for general purpose, pass through easy. You need above HD5XXX to run SteamOS, or for crypto currency mining.

High end NVidia cards work, K4000 tested.

Low end cards in the PCI slots work, it seems NVidia works better than ATI in this case.

I used a 2 GB SSD for the ESXi install. You can install ESXi to a USB stick, or even on the Datastore HDD. The SSD's are cheap, boot faster than USB, and best practice is to have your ESXi install separate.

8 GB ram is about the minimum useful amount.

Virtual disks can be pre-allocated (faster) or defined as some maximum size that they can grow to. So you can define 6 500GB virtual disks on a 1TB drive, and it's first come first serve as the disks grow. Which means you can over-provision your VM's initially, and move them to larger storage when needed.

Prior to virtualization, datacenters struggled with "server sprawl", and now we have "virtual server sprawl". Which means since it's so easy to create/copy VM's, they tend to multiply....

So size your storage accordingly.

You can use iSCSi, NFS, or ATA over Ethernet if you get the HBA card form Coraid.

Unless you build a bootable USB drive with the ESXi install, you will need a cd drive.

I use a Zalman ZM-VE300, which is a 2.5" drive enclosure that allows you to select an ISO file and emulates a USB CDROM drive. Haven't burned a cd (or coaster) since.

Keyboard, Video, and Mouse (KVM) matching the number of GPU's, or a KVM switch.

Step 2

At this point you have an assembled system, VT-d capable CPU & Board, 8+ GB RAM, 2+ ATI GPU's of at least HD5xxx, bootable storage for ESXi and storage for VM's, can boot the ESXi installer from cd/usb/iso, and a supported nic.

Get ESXi installation ISO from here:

Boot into the bios (or UEFI) and verify:

VT-d is enabled

Execute-disable is enabled

boot order facilitates installing ESXi from your chosen media

Discussions

AgatheBauer wrote 07/16/2016 at 04:14

Hey Eric,

nice work! I found this german article that mentions some difficulties and I wonder if you experienced the same issues and if you were able to resolve it:

https://struband.net/esxi-6-0-gpu-passthrough/

I translate:

The start (of the vm with gpu passtrough) only runs properly after a ESXi-Host restart. This means, the VM itself can't shutdown and start again afterwards, or simply being restarted.

Monitor Standby won't work either.

By the way, the ESXI Embedded host client might be of interest to you: https://labs.vmware.com/flings/esxi-embedded-host-client

Are you sure? yes | no

eric wrote 07/16/2016 at 18:12

Thanks for the links!

I had this with certain GPU's, I think it was with the Quadro 3800. It was also only with certain OS's. So I could run an Android VM at startup, but rebooting this VM needed the host to be re-booted. I found that I could "sanitize" the GPU again by running a Windows VM on the GPU, then the Android would run again.

I assumed the host left the card in some state, and the Win VM got it into a similar enough state. I did some experimenting to try and nail down what exactly was happening but never got a satisfactory answer.

I have since been running this on KVM/Qemu to resolve some of the limitations of ESXi. Not to fault VMware, since it's a Datacenter product and not meant for this.

But the impossibility of passing a Geforce card and the "no output to the passed screen until the driver loads" required so many convoluted workarounds for a not really gaming system. Weird stuff like the issue you asked about also pushed me in that direction.

I've gotten far enough with the KVM setup to probably post some logs here. Since they now allow file hosting I'm working on cleaning up my boot USB image & APK for others to test.

Are you sure? yes | no

greenaum wrote 02/06/2016 at 15:24

Pardon me asking a fairly simple question, I think I've got the gist of this thing... So you're controlling the whole thing from a GPU? It's in charge of the virtualisation? Or is it just arbitrating the hardware? Is this a new method of doing things, cos I've never heard of it before, usually all the hypervising is done from the x86 CPU. If I've understood it right, it's insane and genius!

Are you sure? yes | no

eric wrote 02/07/2016 at 19:40

What I am doing is I assign the PCI GPU that I boot from as a passthrough device, which makes the ESXi console "headless". This GPU is given to an Android x86 VM which runs the APP that manages the VM's. This is done by sending commands over SSH to the "headless" host. ESXi is still the hypervisor, I'm just managing it from a VM running at the console.

Are you sure? yes | no

Peter Walsh wrote 04/24/2015 at 22:23

I got an E-mail that appears to be meant for you/your project. Here's the full text:

Hello Hackers!

I'm contacting you because you have a project on Hackaday.io which you tagged
"2015HackadayPrize" but has not yet been officially submitted. Please take a
moment to do so now!

https://hackaday.io/project/1071-the-hydra-multiheaded-virtual-computer

Simply go to your project and use the "Submit project to..." button on the
left under the pictures. You need to do this because we
are starting to give away more and more prizes but only to Official
submissions.

Of course I hope you're hard at working turning your ideas into reality. But
either way you should do this to be eligible to win and to be counted as a
person who believes in the power of sharing ideas to make the world a better
place. Thanks for participating in the 2015 Hackaday Prize!

-Mike Szczys

Are you sure? yes | no

Joshua T wrote 04/24/2015 at 23:24

@Peter Walsh I got the same exact email. Someone at Hackaday must have made a mistake in setting up their automatic mailer.

Are you sure? yes | no

eric wrote 04/25/2015 at 02:33

They straightened it out, but thanks for the info

Are you sure? yes | no

eric wrote 03/20/2015 at 01:06

This is in the 2015 Hackaday Prize because virtualization is the future of computing. The goal is to bring the benefits of virtualization already enjoyed in the Datacenter to all. One of the benefits traditionally used to sell this to IT departments is the power savings. The nearly zero penetration of virtualization for consumers is an unexplored avenue for lessened environmental impact. My kids and I currently use the system pictured left simultaneously, so that's 1 PSU using power instead of 3. Not to mention the VM's that are available, I don't have a MAC sitting around (actually 2), or a Android x86 box, etc. All of those would be waiting for EOL and eventually headed hopefully to recycling, but maybe the landfill. 1 PSU, 1 MB, proc, Ram, 1 HDD to be recycled instead of many. Envision the power saved and mountains of servers no longer life cycled due to corporate virtualization, and the impact if that were universal.

Are you sure? yes | no

greenaum wrote 02/24/2016 at 20:07

Actually, I remember in the earlier days of PCs, maybe 20 or 30 years ago, an idea to share 1 PC with several users. It came as a card, ISA I think, with multiple graphics outputs. I think 2, plus the onboard one allowing for 3 users.

Dunno if it ran DOS or Windows 3.1. But the users were fairly independent, all seemingly having their own machine, although I'd imagine it was limited to Windows apps, no Doom or other games.

I was thinking something like this might be a good idea now. There's already piles of "thin clients" in the dustbins and back cupboards of IT departments around the world, they go for very little on Ebay. That'd be one avenue.

Another might be multiple video cards, or 1 video card with several independent outputs, which are popular nowadays, mine has VGA + HDMI and I think another one, DVI or something.

For stuff like web-browsing, the average multi-core PC is pretty over-specified. Maybe the presence of multiple cores could help too. Probably simplest, actually, would be using virtual machines.

Either running the extra machines over, say, an HDMI link, or a KVM, or over Ethernet on a thin client. Even games consoles are doing video over Ethernet (and Wifi!) now, you can play on a Playstation 4 through the PS Vita, a handheld, over Wifi. So the idea's in people's heads.

It might also have an angle of allowing parents to control their kids' computer usage.

Specifically, using multiple monitors on one PC would tie in with what you're doing. Is it possible to route multiple USB keyboards and mice to different virtual machines? It could be an issue as simple as buying another cheap mouse and keyboard, plus another monitor or even the TV (back to the '80s!).

Maybe even the option of using a tablet, they're popular in homes and cheap, as a client, together with a Bluetooth keyboard. There's lots of potential here! You could end up with a lot of people using your stuff. For a tablet version you could even charge money, enough to make up for the support headaches!

Are you sure? yes | no

meep wrote 10/12/2014 at 00:39

Loving this project. I got to a fairly similar place building a whole-house media system built on unraid and xen;

http://mediaserver8.blogspot.ie

I've just added a pcie expander which does a similar job to the switches you've tested though I really like your ideas on connecting multiple remote expansion boxes via fibre optic. Very neat.

Are you sure? yes | no

ArsenioDev wrote 10/04/2014 at 01:28

What are those cards with the RED DVI ports?

Are you sure? yes | no

eric wrote 11/09/2014 at 07:00

Sorry for the late reply, those are HD6450's

Are you sure? yes | no

ArsenioDev wrote 11/11/2014 at 17:34

Thank you!

Are you sure? yes | no

eric wrote 07/26/2014 at 03:32

Hi Rab,
I logged on to reply to some profanity laced comment regarding my Matrix analogy, but I guess someone at Hackaday is on top of things and it's gone.

I would classify my board as gamer class, it was popular for it's overclockability. You can hardly find them anymore since people burned them all up. But check out the board Milo is using below - Asus P9X79. I was able to individually assign the USB's for him.

When I first built this I used the CF adapter as well, but since stole it for my laptop carputer. I use Winimage to back up ESXi boot drives for me and customers and it works really well. Then you can name the vhd file whatever build you are backing up.

I have been MOVING, so I am somewhat distracted from this. I expect to have a re-write of the Android app for you to try soon. It's on App Inventor 2, so I will be uploading the whole project for anyone here to contribute to.

Are you sure? yes | no

rab wrote 08/11/2014 at 23:45

Yikes, not sure why your Matrix analogy would draw such ire. Ya, as I continue to learn about this stuff it appears to be more about the chipset then the actual board. Given the chance to do it over, I probably would have went with a x79 board and compatible CPU. I have too much $ in my current MB/CPU though so just have to fight thru it. I should still be able to get 4 "PCs" running on my config for a decent cost which fills the immediate need, although that PLX stuff looks really interesting.

I'm definitely up for giving your Android app a go once you get it done.

Are you sure? yes | no

rab wrote 07/16/2014 at 02:34

Hi Eric, super cool project. I have started building my own HYDRA and was curious how you did the USB passthru. Even with esxi 5.0U1 all I could see were 2 USB controllers available for passthru. I have sucessfully passed thru one, but the entire controller gets passed to the VM so I dont see how I can assign individual usb ports to other VMs for keyboard and mouse. Since I run esxi from usb the other controller is off limits for passthru. I was curious how you were able to passthru enough usb ports on your build to support 6 VMs. I have found a 4 port usb pcie card that claims to have a sepearate controller for each port, but since it is $100 would prefer to use the many onboard usb ports on my asrock z87 extreme4 MB if possible.

Are you sure? yes | no

eric wrote 07/24/2014 at 18:20

Sorry for replying sooner it's been one emergency after the next lately. I would suggest to first get some 2GB ssd which I think I paid ±$30 for 5x. This would at least give you one more. Your advanced config should look like the pic I am posting in a new project log. Refer to this and do a lspci at the console and you will see all the devices that could be passed through. As I am typing this got a phone call & have to leave.....but I will look up the z87 and see if I can figure out what you should expect to see. I am basically giving EVERYTHING to passthrough except for the nic & sata interface, so you would want to get the host down the minimum & pass ALL the USB's.

See if you can post a pic of your advanced config as well.

Thanks!

Are you sure? yes | no

rab wrote 07/25/2014 at 14:20

No worries Eric, I appreciate the reply and guidance. After brainstorming some more, I too came up with a path similar to what you suggested. I purchased a compact flash to sata card for $12 with the intent of booting esxi from it so I can use both onboard USB controllers for VMs, then just purchase 2 additional low cost USB 2.0 cards. It's been convenient booting esxi from a usb since it allows me to easily try different versions and get my host server up and running if I screw things up too bad, so being able to do the same with flash cards was appealing. I also dug a little deeper on the MB you are using and it looks like that board has the feature of being able to enable/disable the individual usb ports which I'm guessing also allows for individual passthrough. Since your MB seems to be server class and mine consumer level, I guess that is to be expected.

Are you sure? yes | no

John Kiniston wrote 07/03/2014 at 17:46

This is really cool. I host a weekly Lan Party of sorts where I setup 'Artemis Starship Bridge Simulator' and therefore have 6 machines to take care of, I had briefly looked at using KVM to build a rig similar to yours but I didn't think I could get enough Video cards and USB controllers into a single machine for it to work.

Are you sure? yes | no

numeror wrote 06/14/2014 at 19:52

Amazing project, thanks a lot for sharing!
I would love to build one Hydra. I don't want any physical machine in my home (or very small ones like NUCs, Raspberry Pi or cheap thin clients).

So should I make one VM per home machine + one VM per server then?

What do you think of this setup for example?
- 1 VM with XBMC and GPU connected to TV/Xbox controller via HDMI/USB over ethernet - launched when needed
- 1 VM with Windows connected to desktop monitor/keyboard/mouse via HDMI/USB over ethernet - launched when needed
- 1 VM with Windows and GPU + Steam (the "gaming server") - launched when needed
- 1 VM with a NAS (eg Xpenology) - always on
- 1 VM with a linux server - always on

Is it more energy-efficient and cost-efficient than a standard setup with multiple "physical" computers?
Can you passthrough a Nvidia GTX6xx/7xx? Because the "Limelight" software seems interesting to stream to Android devices.

In terms of case I was thinking of either in a rack cabinet or using a "HAF Stacker" stored in the basement. The Cooler Master HAF Stacker seems such a great case for a Hydra because you can extend it and pass cables from one case to another!
But a rack would be more convenient on the long-run, what do you think? Also do you know if HDMI/USB over Ethernet can go through a switch please? Or at least a wiring cabinet?

Sorry for all the questions but I'm very enthusiastic about your project! I'll follow the logs.

Are you sure? yes | no

eric wrote 06/14/2014 at 22:44

Hi and thanks for the interest.
Most of what you are asking is possible & straightforwardly so. The HDMI & USB over ethernet I have deployed in the past for video wall applications. You would need IP addressable extenders to have a switch in the mix, which are assuredly more expensive.

As far as energy, you are working 1 box HARD, with the obligatory cooling, as opposed to 5 boxes. I have not done any actual quantitative testing to compare the two, but I would have to assume it's less for VM's. The extenders could be POE, so you may not add much there. The savings would increase as you add VM's.

The Nvidia question is more complicated.
I've passed through a Nvidia K4000 (PCIe) and an FX5200 (PCI). There are ways to both soft mod as well as adding resistors to Nvidia cards so they behave like their more expensive counterparts. Whether Limelight works after modding remains to be seen. Modding allows the lower end cards to be passed through, but I would also guess that the big$ cards would support Limelight. I would gamble on a GTX465 and softmod it to a K2000 and see - the features of Limelight seem to be a worthy objective.

I have 2 Hydra's running now, one in the L shaped case for demo & testing, and the one I am using in a 8U rack. I'm sort of happy with it, had to do some chopping. I am working on an adapter to put a SFF PCIe card in a 1x slot, which will help with the cases. The expanding case looks to have plenty of cooling options, which may not be necessary in your basement.

The more questions the better....

Are you sure? yes | no

numeror wrote 06/16/2014 at 19:55

Thanks for your answers :)
I'll do my build this summer by trying to adapt my current desktop (which is limited by my mini-itx board)
- Case: Silverstone Sugo SG10 (micro-ATX case but 4 PCI slots)
- Gigabyte H87N-WiFi (I plan to upgrade to micro-ATX or maybe more if swap for a bigger case)
- i5 4670 with onboard graphics cooled by Cooler Master Hyper TX3 Evo
- 16Go RAM
- Sapphire 7970 Ghz edition 3Go

I plan on doing the following virtual machines:
- Gaming (Windows & Steam client - Sapphire passed through)
- Media center (Linux & XBMC - onboard graphics passed through)
- NAS (Xpenology - always on)
- Home automation server (Linux & Domoticz - always on)

How did you planned to do your mini-itx HYDRA exactly as it is limited in PCI ports?
Also you mentioned an Android app. Can you manage virtual machines by IP/HTTP commands? That would be useful for me because my home automation server can include HTTP commands in the same interface, which would be convenient to manage both my home and my virtual machines.

As for the name I'm still hesitating between "the Kraken", "World Tree/Yggdrasil" and "the Beanstalk" (tiny rig to the cloud...) :)

Cheers

Are you sure? yes | no

eric wrote 06/17/2014 at 00:38

I can't seem to reply to your latest, maybe the nesting would get too deep?

I'd interested to see if that board works even though it supports VTD. I'm reading conflicting reports about socket 1150. I had my eye on a similar one:
Gigabyte GA-Z77N-WIFI Mini-ITX

Check your CPU in Intel ARK..

I intended to use the PCIe slot for the "good" GPU, which would presumably disable the onboard one. They make adapters that go in mini-PCIe and give you a 1x wide PCIe slot. That would be the #2 GPU. I have had better success with Windows of some kind on such a bottlenecked GPU. This all needs to be tested, I am only making assumptions. The onboard GPU may work.

http://www.ebay.com/itm/Mini-PCI-E-to-PCI-E-Express-X1-USB-Riser-Card-with-High-Speed-FPC-Cable-/261446605806?pt=LH_DefaultDomain_0&hash=item3cdf6e9bee

My intent was SteamOS on the good gpu, and Win7 or maybe ReactOS on the mini-PCIe connected one. It remains to be seen whether the windows side could be a streaming host.

The other option would be PCIe switching, but I would have to design a card since none exist that would fit.

So if you have a spare HDD, you can install ESXi to a usb and see if a GPU in the only slot disables the onboard. And of course if passthrough works. If you still have onboard, maybe we can squeeze 3 interfaces out of it.

Yes, I wrote an Android app, which you could install on your phone or tablet. You can control power and snapshots, and I started working on passed through device management as well. I was using these Chinese ralink usb's, passing one through to a VM, and the Hydra would be an access point. The Ralink radios are CHEAP at $6 each, but they burn out in about a month. You could just use your existing wifi.

My goal was always to NOT require another computer for management, but you can manage it six ways to Sunday with another machine...

Are you sure? yes | no

Mike Szczys wrote 06/10/2014 at 22:36

May I also recommend uploading a profile picture? It doesn't have to be you (pssst... my picture isn't actual of me) but would be nice not to talk to the stock avatar.

Thanks!

Are you sure? yes | no

eric wrote 06/11/2014 at 23:36

Scanned my lic. Day off tomorrow, so look for updates. Adding raid controller & space to Milo's (below) Hydra, and he does the filming.

Are you sure? yes | no

Mike Szczys wrote 06/09/2014 at 18:38

Eric, you asked about posting videos directly in the project. Yes, that is possible. When in the editor window paste a YouTube link and then press enter. It should render the file inside of the editor. It's a bit wonky right now but I know it's on the list of stuff to be improved.

Are you sure? yes | no

eric wrote 06/10/2014 at 21:57

Hey that's just what I needed, I want the meat to be HERE instead of Google+.
I would have to do double documentation, which I am finding hard enough with all the work I have going......

Are you sure? yes | no

Mike Szczys wrote 06/05/2014 at 16:01

Thanks for entering this in The Hackaday Prize. Soooo many applications come to mind for a scalable system like this!

Are you sure? yes | no

eric wrote 06/05/2014 at 19:31

Thanks Mike,
Work is all of a sudden consuming my time, so I am quite behind in the documentation. I'm sporadically working on a big update offline in word that highlights the scalability. Lots of diagrams but should be up by the weekend.

Are you sure? yes | no

milo618 wrote 05/07/2014 at 21:54

I'm using one of his HYDRA computers for video editing and Photoshopping. I have Windows XP, Windows 7, Linux and Mac in one computer. I do all the editing with Windows 7, checking the e-mail, Facebook and surfing the web I use Linux, to manage all the computers' settings with vSphere Client I'm using Windows XP.

With the Windows 7 I'm using Nvidia k4000 video card and 28 Gig RAM on Asus P9X79 motherboard.

Are you sure? yes | no

The HYDRA multiheaded virtual computer

Description

Details

Components

Project Logs

Collapse

10 gpu's passed to VM's in 1 box STUNT

Built a case to address airflow & wire mgmt

Hydra MGMT by Android App

Android VM/App is the best management method

Oculus Rift working on the Hydra

Hydra Gaming Console

Working on Vsphere 6.0 beta

Better Video

Overview video

System Design

Build Instructions

Collapse

Discussions

Similar Projects

Restoring an Asus Chromebox CN60

Restoring the Palm Foleo

Hydra Multi OS Gaming Console & Controller

SATAtion

The HYDRA multiheaded virtual computer

Become a Hackaday.io member

Just one more thing

Description

Details

Components

Project Logs Collapse

Build Instructions Collapse

Enjoy this project?

Discussions

Become a Hackaday.io Member

Similar Projects

Does this project spark your interest?

Report project as inappropriate

Send message

Remove Member

Project Logs

Collapse

Build Instructions

Collapse