Note: This project is still in an infant state of development. This is a public beta release, meaning it is not fit to be used in any safety critical applications yet. You can get in touch with us if you would like to use this project as a basis for further development.
Performant visualization is key to discovering context in large graphs.
Take screenshots and record videos
VRNetzer Architecture Overview
Tutorial 1: Using the Uploader to add your own network
Tutorial 2: Creating custom User Interfaces
Tutorial 3: Creating a route on the backend
We are running the platform on an MSI gaming laptop with the following specs:
- CPU: i7 - 7820HK
- RAM: 16 GB
- GPU: NVIDIA 1070 TI
- OS: WINDOWS 10
and a VR HEADSET - we tested the following:
- HTC VIVE VR Headset + controllers
- OCULUS QUEST + controllers
for a quick start, you can just download the VR Module executable and run VR_Module.exe on your windows computer with a SteamVR compatible headset. Please watch this video that will help to get you sterted. It comes preconfigured to connect to the other modules that we already installed on our server to make it easy for you to get a first impression. It has the ability to upload your own datasets, but we don't recommend it. Do NOT upload any sensitive data here, this is only for demonstration purposes and for the whole world to see. We don't guarantee your data's safety! If you want to work with your own data you should go with the Stand Alone version.
However, if you want to implement your own datasets and functionalities, you can also run everything localy on your machine (the SAFE option if you are dealing with sensitive data) or run the Analytics Module and UI Module on a networkserver or cloud service (if you need scalability and processing power).
Here is a step by step guide how to do a local installation on a windows computer. Note that the Analytics Module and the UI Module can also run on a (linux) server,
while the VR Module needs to run on a windows 10 machine as gaming hardware is required for Virtual Reality which is best supported under windows.
- download and install MySql Workbench
- download the .dump file of our database from here
- Create a new database, user and password and upload .dump file to it
you have now a clone of our database on your local machine.
2.) Clone the Analytics Module repository into a new folder somewhere on your computer called "VRNetzer"
-
install python >3.6, make sure its added to path variable
-
install pip for python
-
edit VRNetzer/Analytics_Module/db_config.py to match the database host and user you created before and save
-
right click on VRNetzer/Analytics_Module/runAnalytics_Module.ps1 -> "run with power shell"
-
if the output of the console window that just opened ends with "Running on http://127.0.0.1:1337/" you are good to go, if there are errors you will need to install dependencies.
3.) Clone the UI Module repository into "VRNetzer"
-
install python >3.6, make sure its added to path variable
-
install pip for python
-
edit VRNetzer/uiserver/static/js/UI_Module_Config.js and set "dbprefix" to the address of your dataServer , here: 'http://127.0.0.1:1337' and save.
-
edit VRNetzer/UI_Module/static/js/ UI_Module_Config.js and set "dbprefix" to the address of your Analytics Module, here: 'http://127.0.0.1:1337' and save.
-
right click on VRNetzer/UI_Module/run_UI_Module.ps1 -> "run with power shell"
-
if the output of the console window that just opened ends with "Running on http://127.0.0.1:5000/" you are good to go, if there are errors you will need to install dependencies.
4.) Download the VR Module executable and extract it to "VRNetzer"
- if you haven't already, make a Steam account and install SteamVR and test your headset
- edit VRNetzer/VRnet/viveNet/Content/data/Config.txt and change the address to the one where your UI Module is, here http://127.0.0.1:5000/
- run VR_Module.exe
It is also possible to run the VR Module in desktop mode which is practical for development tasks or if you do not own a headset.
To do so, simply change the value "vr":false in the config file of the VR Module to "vr":true and restart the VR Module.
If you want you can take screenshots and videos of the running application using nvidia geforce experience, which is a free download for nvidia graphics cards, find it here: geforce experience. After installation the default key combination for taking a screenshot is alt+f1 and to start or stop a video recording you press alt+f9. Depending on your setup you might need to enable the ingame overlay and enable desktop recording in the privacy settings in the geforce experience settings if the hotkeys do not work right away.
The VRNetzer platform consists of 5 Modules:
Unreal Engine is one of the industry leaders in the videogame world. We chose it as the base for our VR Module for it's astounding graphics performance, continuous support of upcoming VR hardware and because it's open source.
The UI Module is running in the browser of your local machine and can SEND GET and POST requests TO the Analytics Module.
Nearly all actions originate here (exept the ones in the VR Module, like if the user touches a node or moves the network with the VR controllers).
Think of the UI Module as the frontend of a website and the Analytics Module as it's backend.
A User clicks on a button on the frontend, this makes it to send a post request to a specific URL (route) on the backend (the Analytics Module), await it's response and finally display the result as text or as a graph on the user interface and/or call functions in the VR Module to alter the appearence of the network displayed in VR.
The Analytics Module can only RESPOND to those requests, meaning the Analytics Module can never send something to the frontend without being asked. Every communication is Initiated by the UI Module.
Now here is what's special about the UI Module:
- It can also SEND api function calls to the VR Module
AND
- It can RECEIVE calls from the VR Module
In contrast to the Analytics Module, the VR Module CAN initiate communication with the UI Module and call special functions set up in the UI Module. This picture illustrates the different routes of communication in the framework.
The Analytics Module is the backend that performs all the data science tasks. It has
- separate routes (URLs) defined for each task
When the UI Module sends a request to one of these, it parses the input parameters, makes database queries and performs calculations, and finally returns it's response to the UI Module. - it can run on the local machine or on a powerful cloud server if more power is needed
@poorCeline: Please add short discription about how tables are related and maybee point to some python code that illustrates the communication with the database
This is the data base schema:
- right click on VRNetzer/ Analytics_Module/runAnalyticsModule.ps1 -> "run with power shell" to start the Analytics Module
- open the web frontend of the Analytics Module in a browser http://127.0.0.1:1337/swimmer
- tick "Create Project" and choose a name that doesn't exist in the dropdown menu jet
- select .csv files to upload, they must be formatted after these guidelines
- restart the VR Module and load your project
The User Interface in the virtual reality module is a website made with jQuery. In this tutorial you will learn how to make UI elements like buttons or dropdown lists and how to communicate with the VR and Analytics Module.
Basic JavaScript and HTML skills required.
Prerequisites: install the VRNetzer framework on your local machine Installation: Stand Alone
-
in Visual Studio Code (or the text editor of your choice) File -> Open Folder -> navigate to your UI Module Folder
-
right click on VRNetzer/Analytics_Module/runAnalyticsModule.ps1 -> "run with power shell"
-
open a chrome browser at http://127.0.0.1:5000/ . This is the main UI to which we will add something in the right-most tab.
-
hit 'Ctrl + Shift + i' to open the developer tools, click on "Network" and tick the "Disable Cache" checkbox. Note the "Console" window, where debugging output is displayed.
-
open
VRNetzer/UI_Module/templates/main.htmlin your editor -
at the end of the file, after
<div id="tabs-7">add<button id="MyNewButton"> EXIT </button> -
save changes to main.html and refresh the browser by clicking in the red area and hit F5
-
if you now navigate to the right most tab again, the button appeared. It looks grey though, not like the other buttons.
The different elements of the page are set up in the html documents. All the logic happens in the JavaScript, in the UI_Module\static\js folder.
VRNetzer_API.js is the most important, it has all the functions to communicate between the Analytics Module and the VR Module.
Then there is a file for each of the html files (main_UI.js). These all have this initialization function $(document).ready(function () {....} and in there are functions that bind to the UI elements created in Html. For our button we do this:
- at the end of main_UI.js, but still inside the document.ready() function put this code:
$(function () {
$("#MyNewButton").button();
$("#MyNewButton").click(function (event) {
event.preventDefault();
logger("click!")
});
});
- save and refresh the browser
Now that jQuery is aware of the new button the css styling also works. Click it and the console outputs the message so we know it works.
- put this code in the MyNewButton.click() function
var args = {
"content": "somecoolName",
"route": "tutorial"
};
ue4("GetSelection", args);
Everything that looks like ue4("name","argument") are calls to the VR module, the so called VRnet API documented here.
The variable "args" is a JavaScript object and follows the JSON syntax.
The "GetSelection" call above returns the currently selected nodes in the VR module to this return function ue.interface.getSelection specified in VRNetzer_API.js around line 30
//// FUNCTIONS CALLED BY UE4
ue.interface.getSelection = function (data) {
logger(data);
switch (data.route) {
case "saveSelection":
SaveSelectionDB(data);
break;
case "reLayout":
ReLayoutSubSet(data);
break;
case "GSEA":
GSEASubSet(data);
break;
}
};
Because the GetSelection call is used for different things, it's arguments contain "route": "tutorial" that is used in the return function to determine what to do with the response.
- in the return function above, add a case "tutorial" to the switch statement
case "tutorial":
LogOnUIServer(data);
break;
LogOnUIServer(data); prints out the data we receive from the VR Module on our python console.
That's it! let's start the VR Module and check if everything works.
- start viveNet.exe
- open a layout
- load a selection
- click on MyNewButton
You should see some printout in the UI Module console window. We have now initiated communication from the UI module to the VR module (by pressing the button) and received a response. In the next step we will see how to do the same with the backend, the Analytics Module.
Basic Python knowledge required
Prerequisites:
-
local installation of the VRNetzer framework Installation: Stand Alone
-
Tutorial 2 finished
-
in a new instance of Visual Studio Code (or the text editor of your choice)
File -> Open Folder -> navigate to your ** Analytics_Module ** folder -
in
app.py, at the end of the document but beforeif __name__ == "__main__":insert
@app.route('/api/<string:db_namespace>/MyNewRoute', methods=['POST'])
@cross_origin()
def my_new_route(db_namespace):
data =request.get_json()
node_ids = [int(x) for x in data['node_ids']]
print(node_ids)
return jsonify(node_ids)
- save
app.pyand bring up the Analytics Module console, it should detect the change and restart automatically
That’s it! Now we will go back to the UI Module and forward the output of the VR module that we created previously in tutorial 2.
For this we will use a POST request.
- go to the UI Module project and open it in a new Visual Studio Instance.
- paste this code at the end of
VRNetzer_API.js
function MyNewPostRequest(data) {
payload = JSON.stringify(data);
//logger(payload);
path = dbprefix + "/api/" + thisNamespace.namespace + "/MyNewRoute";
$.ajax({
type: "POST",
url: path,
contentType: "application/json",
data: payload,
dataType: "json",
headers: {
"Authorization": "Basic " + btoa(dbuser + ":" + dbpw)
},
success: function (response) {
// DO SOMETHING WITH RESPONSE HERE!
logger(response);
},
error: function (err) {
logger(err);
logger(data);
}
});
}
This is a blank POST request that calls the route we created before.
path = dbprefix + "/api/" + thisNamespace.namespace + "/MyNewRoute" is the target URL, dbprefix comes from UI_Module_Config.js and thisNamespace.namespace is the name of the project.
path = dbprefix + "/api/" + thisNamespace.namespace + "/MyNewRoute" is the target URL, dbprefix comes from Analytics_ModuleConfig.js and thisNamespace.namespace is the name of the project.
- also in VRNetzer_API.js, in the function
ue.interface.getSelectionputMyNewPostRequest(data);instead ofLogOnUIServer(data);
You can find examples of all used .csv formats HERE to to use as templates for the formatting.
Note: You need to assign unique node and attribute ID's if you upload your own data to a new project.
Don't use commas in string fields, as they are reserved delimiters.
Node Lists look like this:
8473,0.4993,0.4544,0.640,188,20,26,100,3dportrait
...
where each line is a node with the following data:
| 8473 | 0.4993 | 0.4544 | 0.640 | 188 | 20 | 26 | 100 | 3dportrait |
|---|---|---|---|---|---|---|---|---|
| NodeID | X-Pos | Y-Pos | Z-Pos | R | G | B | A | Name |
Note: For now, you have to provide XYZ coordinates for the layout. The positions need to be normalised between 0 - 1
RGBA colours range from 0 - 255 Note: A value's should be 100. Bigger values makes nodes glow, smaller values make them darker.
Link Lists
1267,2945
...
where
| 1267 | 2945 |
|---|---|
| Start | End |
for now, only one link list per project is supported
Selection Lists
1849
2455
4029
...
are a list of ID's separated by line breaks
Labels
| x_loc | y_loc | z_loc | text | namespace |
|---|---|---|---|---|
| 0.5000000 | 0.5000000 | 0.5000000 | " C E L L U L A R C O M P O N E N T S" | 5_cell |
A simple way to add textlabels at certain positions to a specific layout (namespace)
Attributes
| node_id | attribute_id | namespace | name | description |
|---|---|---|---|---|
| 16048 | 4416 | DISEASE | "Down syndrome" | NULL |
Every node id can be associated with several attributes. Note
The following function calls are sent from jQuery to the VR module.
The syntax looks like this:
ue4("rw_result", response);
where "rw_results" is the function name and "response" the parameters.
This list will grow in the future.
| function | parameters | type | description |
|---|---|---|---|
| LoadDbNodeList | node list object | json | loads a node list into channel A of the layout |
| LoadDbNodeListB | node list object | json | loads a node list into channel B of the layout (only xyz, no colors) |
| LoadDbLinkList | link list object | json | loads a link list from db |
| LoadDbLabelList | label list object | json | loads a label list for channel A of the layout |
| LoadDbLabelListB | label list object | json | loads a label list for channel B of the layout |
| Init | --- | --- | start initialization routine, OnDocumentReady() |
| Morph | "A" or "B" | string | Morph between layout channel A and B |
| GetSelection | { "content": "Name","route": "saveSelection"} | json | Get active selection and do different things depending on route (save it on db in this case) |
| SetScale | value | float | set Network Scale |
| SetNodeSize | value | float | set Network SetNodeSize (linksize is also affected by this) |
| SetLinkSize | value | float | set Network SetLinkSize ATTENTION: big diameters cause lag |
| SetLinkAlpha | value | float | set Network Link transperancy |
| SetLight | value | float | set scene light intensity |
| ShortestPathPoint | "p1" or "p2" | string | get selected node ID from VR and define it either as start or endpoint |
| ExitIsolate | --- | --- | exit Isolate Selection Mode (show all links) |
| Rw_Result | rw object | json | display the random walk results in VR |
| Julia | {"seeds":[{"node_id":123}...],"variants":[...],"linker":[...]} | json | Gene Priorization example on side panel |
| ActivateNode | id | int | select and highlight single node in network |
| ReLayout | node list object | json | show newly created layout of subset in VR (this resorts the link list) |
| VRkeyboard | route | string | opens a keyboard in VR - after user presses ENTER, typed string is returned to a .js function by the same name as route - so you need to create this |
| loadSelection | name | string | Deprecated Load selection from csv file |