PacketVR

PacketVR is a terminal emulator that exists in a VR world. It supports multiple terminal sessions at once along with positioning of the terminals and interaction with objects in the world using the Leap Motion.

Log in to a machine and do most of the things you would normally be able to do from a text based terminal. List directory contents, change directories, run terminal based programs... even edit files in nano, vi/vim or other text based editors.

Sorry, but I haven't tested this with Emacs yet...

NOTE: This program works best with the Oculus DK2 and the Leap Motion controller. Only Windows is currently supported due to limitations in the Oculus SDK.

NOTE: This program only supports certain SSH server configurations due to limitations in the SharpSSH library. These limitations are explained in the README.md file of the PacketVR Apps repo: https://bitbucket.org/JamesHagerman/packetvr-apps

"But text is boring!!"

PacketVR has extended the "boring", old school text based terminal a bit...

After logging into a remote machine, run a program written to support PacketVR and the VR scene will be modified around you.

This "ANSI VR" functionality is, currently, relatively limited, but it will be greatly extended in the future. Currently, the system supports:

1. Dynamic loading of Cube primitives at specified x,y,z coordinates (these cube primitives respond to gravity and collisions)
2. Dynamic loading of OBJ files from URL addresses into the scene at specified x,y,z coordinates (statically placed in the scene, for now...)

Rotation, scaling, readable text objects, and other functionality will come in time. At some point, I would really like to add JSON, glTF, animation, and dynamic object support...

Why bother!?

PacketVR started off as the dream of displaying network traffic passing through my firewall in a VR environment. It ended up being a tool that lets people do real work all while staying inside a VR experience.

The very next development goal of PacketVR is to extend it's ANSI VR functionality in such a way that it will be useful for network traffic visualizations.

Piping the output of standard terminal applications such as tcpdump into simple PacketVR applications will allow visualization of almost any type of data.

Python Example Code

While it has not reached the point of building complex data visualizations quite yet, it should be enough to at least get started. Examples of the basic API have been written in Python and have been made available via the following Bitbucket repository: https://bitbucket.org/JamesHagerman/packetvr-apps

The API itself is easy to understand. If you can build a command line "Hello world" program, you can build an application for PacketVR.

I hope that other developers will enjoy playing with PacketVR as much as I have enjoyed building it!

Changelog

v2.0

• Skipping some versions for the final version I'll upload for the #3djam
• Added a real demo mode that logs you into a secured server with the demo scripts on it
• Scripts can now draw objects at wherever your right palm is pointing. Closing your hand into a fist means objects can be drawn wherever your hand is located.
• Added a cursor cube to show where objects will be drawn (if a script uses this positioning feature)
• Polishing some other things up for final submission

v1.6

• Getting scroll regions to work correctly. That means less and vim should be a LOT easier to use.
• Switched to the saxMono font because it looks better and has a better cursor char
• Updated the cursor char
• Speed up the cursor blink speed
• Handling tabs halfway decently now.
• Support for the csi-[-L and esc-M ANSI codes is complete and working correctly.

v1.5

• Adding Quick Switching gesture to help users see the real world sometimes
• Adding partial support for '^' control characters. This should help with some telnet text based Multi-User Domain/Dungeon games (MUDs)
• Updating the hit boxes again to make them easier to move.
• Removed a bunch of the debugging code I accidentally left turned on in v1.4

v1.4

• Breaking change to ANSI VR API: The codes now must start with \x1b! instead of \x1bV so that existing terminal applications do not send the "Start of Guarded Area" code (which is standardized to \x1bV) and confused PacketVR's ANSI code parser.
• Fixed up the hand hit boxes for the terminals so they should be easier to control
• Added a few more REALLY important ANSI control codes. This fix allows for the up arrows to search through Bash History correctly as well as allows terminal programs to operate with less display bugs
• Added support for the Home and End keys
• Added support for both Normal and Application cursor modes... though full support is still marginal
• Fixed line wrapping while editing a command

Credits

Credit goes to Bartek Drozdz (@bartekd) for writing the original OBJ loader for Unity:
http://www.everyday3d.com/blog/index.php/2010/05/24/loading-3d-models-runtime-unity3d/

And to Tamir Gal who wrote SharpSSH, a port of the JSch SSH2 library to C# that actually works with Unity's version of C#:
http://www.tamirgal.com/blog/page/sharpssh.aspx