Now that things were shaping up, I needed a way to access the language model from the outside of my local network. The idea is that you can just grab the box and take it wherever, or give it to whoever. However, I really don't want to just expose the model to the internet and cross my fingers, hoping that no one stumbles upon it by accident. While security by obscurity might work, it certainly is not a long-term solution.

While I have a certain understanding of how NGINX, dynamic DNSes, and port forwarding work together, setting it up this way still means that it's exposed to the open internet. 

But! There is another way to address this, and it's quite convenient. It's free, easy to set up, it's available on almost any platform (Windows, MacOS, iOS, Android, Linux), and did I mention that it's free? What I'm talking about here is NordVPN Meshnet. 

It allows you to create a mesh network of devices, effectively establishing direct connections between them. This enables a couple of things, but most importantly it grants access to the language model from anywhere in the world. The two additional benefits are that we can SSH into the Satellite at any given point, and all the connections through Meshnet are encrypted.

It comes with a convenient open-source Linux app, and an install script.

You can install it with the following command:

sh <(curl -sSf https://downloads.nordcdn.com/apps/linux/install.sh)

There are two options to log in: either through a web browser or in the typical Linux user fashion, using a token. Keep in mind that either way, you will need to open a browser at some point - to either click through the log in steps or generate a login token.

Web browser:

nordvpn login

Token:

To generate a token, log into your Nord account, head over to the Meshnet tab, set up NordVPN manually, and last but not least, click on "Generate new token".

Then you can choose to create a permanent or temporary (30-day) token.

nordvpn login --token <insert_your_token_here>

After you log in, make sure Meshnet is on with:

nordvpn set meshnet on

With Meshnet running, you can check your Satelite's IP/Nord name using:

nordvpn meshnet peer list

The same applies to the server hosting the LocalAI model. Once both devices have Meshnet running, everything is ready to go.

And that's literally it for the networking setup. As long as the satellite is connected to a WiFi network, it can access the language model through a POST. There are no hidden fees, no small print. It just works. 

Oh and by the way, if you want to run all of this from an actual virtual machine, that works too! Just make sure the guest has internet access.