Skip to content

Eye controlled music composition and performance environment.

License

Notifications You must be signed in to change notification settings

PaulBatchelor/HandsFreeSoundJam

 
 

Repository files navigation

Microsoft Hands-Free Sound Jam

Microsoft Hands-Free Sound Jam, or "MSHFSJ", is an eye-controlled music environment for electronic loop-based performance and composition. It is designed using familiar design paradigms found in commercial music production software, which have been adapted to work well with eye control. The core interface revolves around what is known as a clip launcher, inspired by the the one found in the popular digital audio workstation Ableton Live. Using the MSHFSJ clip launcher, one is able perform a piece of music by scheduling, or "launching", small musical fragments known as "clips". These clips are automatically aligned and quantized to the next downbeat of the song. Editing capabilities are provided through an editing window, where clips can be modified or rewritten entirely. Modified musical content will automatically be loaded and saved when the application is open and closed.

MSHFSJ uses the PCEye Mini via the Tobii Stream Engine API for eye tracking. Graphics are drawn using OpenGL via NanoVG, a lightweight 2d vector graphics library. Persistent data storage is provided via SQLite. Realtime audio is achieved using RtAudio, and digitial signal processing is built on top of the musical audio signal processing library Soundpipe.

Usage

Please refer to the Usage file.

Setup

Windows 10

The Windows build uses the MSYS2 environment.

Full instructions on building for on Windows can be found with the document build-on-windows.md.

Linux

On Linux, it's business as usual. You will need a few dependencies to be installed, including JACK, and GLFW3.

On Linux, run:

make linux

Before running the program, assets from the other parts of this repository need to be transferred. Do this with:

make transfer

Then run the program:

./MicrosoftHandsFreeSoundJam

Mac OSX

Mac OSX can be built from the commandline using the provided Makefile system:

make osx

Then the program can be run with:

./MicrosoftHandsFreeSoundJam

Screenshots

The Main Screen

The Clip Editor Screen

Internal Engine Documentation

About

Eye controlled music composition and performance environment.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C 95.0%
  • C++ 4.6%
  • Other 0.4%