Janus Fleet

A project log for Metaverse Lab

Experiments with Decentralized VR/AR Infrastructure, Neural Networks, and 3D Internet.

alusionalusion 10/26/2015 at 23:311 Comment

Link to Part 1: Dockerized Multiserver

Link to Janus-Server

Part 2 of Microservice Architectures for a Scalable and Robust Metaverse

The Janus multi-server enables multiplayer and is allows one client to share information with another client. The server software talks to every connected JanusVR client to know who else is in the same virtual space and where other avatars are. It is what glues the Metaverse together.

It's time to give the Janus-Server the love and attention it needs to be more robust when receiving unexpected amounts of traffic. The solution I have been investigating is Kubernetes; a system for managing containerized applications across a cluster of nodes.

The idea is that you abstract away all your hardware from developers and the 'cluster management tool' sorts it all out for you. Then all you need to do is give a container to the cluster, give it some info (keep it running permanently, scale up if X happens etc) and the cluster manager will make it happen.

Greek for "Pilot" or "Helmsman of a Ship"

Kubernetes builds upon a decade and a half of experience at Google running production workloads at scale, combined with best-of-breed ideas and practices from the community.
Reference to Google paper [Lesson's Learned while developing Google's Infrastructure]
Production ready as of 1.01:

Key Points:

Introspection is vital. Kubernetes ships with [ ] cadvisor for resourcce monitoring and has a system for aggregating logs to troubleshoot problems quickly. Inspect and debug your application with command execution, port forwarding, log collection, and resource monitoring via CLI and UI

Kubernetes handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. Using the concepts of "labels" and "pods", it groups the containers which make up an application into logical units for easy management and discovery.

Best practice for containerized applications is to run one process per container. The node.js server should be linked to the mysql database container. Decoupling applications into multiple containers makes it much easier to scale horizontally and reuse containers. Let's start with the node.js portion. Note that this current implementation is nearly working. Make sure to have Docker installed, then docker pull node in a terminal.

**This current Dockerfile is still broken**

FROM node
RUN git clone
RUN cd avalon-server && npm install && npm install forever -g && \
    npm install forever-monitor
WORKDIR avalon-server/
RUN chmod +x
ENTRYPOINT ["/avalon-server/"]

forever is a simple CLI tool for ensuring that a given script runs continuously.

config.js is configured with hardlinks currently, this will be changed in the future.

I have some questions for the linux container community:

The script looks as follows:

# Generate the certificates
sh generate_key
# Start the server
node server.js
# Not sure yet where to add this but the container keeps exiting.
# tail the logs
# tail -f server.log

If I run tail -f server.log, the container will stay open but the container will fail to build because it hangs there.

In order to get the mysql container, type into a terminal docker pull mysql

** This Dockerfile is still broken **

FROM mysql
ADD janusvr.sql /tmp/
RUN adduser --home /home/janus --disabled-password --gecos '' janus
RUN MYSQL_USER="janusvr"
CMD mysql -u janusvr -p janusvr janusvr < /tmp/janusvr.sql

The latest MySQL image has environment variables passed in for configuring the database.

## ------ MySQL Container ------ ##
# mysql basics #
USE janus;
# Start a mysql server instance
docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag
# Connect to MySQL from an application
docker run --name some-app --link some-mysql:mysql -d application-that-uses-mysql
# View MySQL Server log through Docker's container log
docker logs some-mysql

Help Needed

The janusvr.sql tables and the other files can be found here:

In order to test if the multiserver works, you can clone and just have to change line 18 in to point to the mutliserver as indicated by the Docker container:

<Room server="DOCKER_IP_ADDRESS" port="5567" ... 

Docker for the Raspberry Pi: might be good for testing...

Lead engineer of kubernetes describe differences between Mesos and Kubernetes + Fleet / Swarm.

The Open Container Initiative is a lightweight, open governance structure, to be formed under the auspices of the Linux Foundation, for the express purpose of creating open industry standards around container formats and runtime.

Another solution recommended by a brilliant hacker:

To be continued...


underdoge.avi wrote 10/27/2015 at 00:33 point


  Are you sure? yes | no