Spicy Reality

A project log for Metaverse Lab

Experiments with Decentralized VR/AR Infrastructure, Neural Networks, and 3D Internet.

alusionalusion 10/09/2016 at 04:290 Comments


Many virtual art gallery applications seem uninspired and taste bland.

I wanted to create an artificial art gallery with neural networks that make their own art:

The ideas evolved from painting images in the WiFi to stepping inside the painting and connecting with others. A VR web browser can distinct each sphere from different file/web servers and represent them in playful ways. This site is even unlocked and editable, a functionality that most immersive digital content lacks. VR is as much as a creation tool as it is a consumer device and being able to edit the source from within the simulation is a key component for being able to bootstrap the Metaverse.


Lets enhance.

First we need to spice up these textures:

Using a combination of style-transfer and image super resolution you can transfer just textures plain without any seasonings:

Preview in VR:

But if you're like me, you like to take things up a notch. I styled in some Van Gogh and palette knife layers in the neural texture oven for 5 minutes.The scan looks much better when you import it into a game engine. Further optimizing this is very possible. What if it were possible track a persons movement or gaze on the 2D texture map and foveate rendering there to eliminate pixels and enhance? I think there are no limits with spicy reality.

Full album:


**UPDATE: New WebVR art Gallery here:

As long as your movement is being precisely tracked, you can render real-time effects into mixed reality by applying the shader origin to where a persons gaze rests and then switch states. You can kind of see where this can go with LSD for Hololens:

It'd be a cool sequence to tap into GPU when sitting idle and have it wake up entering into a dream state of where you last left off.

It's not enough to look around and just watch a 360 video anymore, in here you can be social and explore in an interactive 3D environment. Combine the power of the browser with a gaming engine written in javascript and the entire world becomes a massive multiplayer online video game anyone can be part of.