Skip to content

Commit

Permalink
edit app install and main page, fix fustrum in playground
Browse files Browse the repository at this point in the history
  • Loading branch information
SpookyCorgi committed May 4, 2023
1 parent 02fe1aa commit a65b765
Show file tree
Hide file tree
Showing 9 changed files with 97 additions and 219 deletions.
1 change: 1 addition & 0 deletions site/src/routes/+layout.svelte
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<script lang="ts">
import '../theme.postcss';
//import '@skeletonlabs/skeleton/themes/theme-skeleton.css';
import '@skeletonlabs/skeleton/styles/all.css';
import '../app.postcss';
import { AppShell, AppBar, Toast, Drawer, drawerStore } from '@skeletonlabs/skeleton';
Expand Down
52 changes: 52 additions & 0 deletions site/src/routes/docs/+page.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
## What is Phiz?
Phiz is an open-source tool that gives you the power to do facial motion capture right in your browser, using the webcam on any mobile device or computer. You can then send the blendshape data, either locally or remotely, to another browser or popular game engines; you even have the option to broadcast the blendshape data through OSC protocol using Phiz's cross-platform app.

**Phiz is under development with frequent updates. If you encounter issues, ensure you're using the latest version.**

## How does it work?

- **Facial Motion Capture... in Your Browser**

Phiz's motion capture system was built using [Mocap4face](https://github.com/facemoji/mocap4face) and [Google mediapipe](https://github.com/google/mediapipe)'s machine learning model. **No iPhone LiDAR or RGBD camera needed!** The underlying ML code is not fixed and will be updated whenever there is a better model.

- **Streaming Data: Unreal and Unity Support**

Phiz includes custom plugins that fire up WebSocket servers within [Unreal Engine](https://www.phizmocap.dev/docs/unreal/basic-usage) and [Unity](https://www.phizmocap.dev/docs/unity/basic-usage), which are used to receive live data from your browser. The plugins also include convenient presets that you can easily use with popular 3D avatars, such as Metahumans and ReadyPlayerMe characters.

- **Streaming Data: Browser to Browser & Browser to OSC App**

Phiz uses WebRTC (the protocol used in video calling) to send data from one browser to another or from a browser to the app. However, no image or video is sent at all. WebRTC also allows for peer-to-peer secure connection and low latency.


- **ARKit Support**

Phiz uses the same blendshapes as ARKit. Check out the [Blendshapes page](https://www.phizmocap.dev/docs/specs/blendshapes) for more information about the blendshape names and orders.


## How do I use it?
**Disclaimer:** Ensure the browser page remains active during recording.

1. Visit the [capture page](https://www.phizmocap.dev/capture) and allow the website to access your camera.
2. Select the correct camera, choose the desired frame smoothing, and decide whether to use sensitive motion capture or not.
3. Select one of the two methods to send the captured data:

- **Local:**
- Use Phiz's custom plugin to send data directly to [Unreal engine](https://www.phizmocap.dev/docs/unreal/basic-usage) or [Unity](https://www.phizmocap.dev/docs/unity/basic-usage) using your defined URL and port.
- Check the [documentation](https://www.phizmocap.dev/docs) for more information.

- **Remote:**
- Click on the 8-character code starting with "p-" (e.g., p-AwCodE) to copy it.
- To send data to another device (e.g., recording on phone and controlling avatar in Unreal on PC):
1. Open the [receive page](https://www.phizmocap.dev/receive) on another device.
2. Paste the copied 8-character code.
3. You should now see live streaming data on your device and be able to share it locally.
- To send data to the Phiz App for broadcasting OSC data:
1. Download and install the [Phiz App](https://www.phizmocap.dev/downloads) for your platform.
2. Open the app and paste the 8-character code.
3. The data will now be available in OSC format on your chosen port.

If everything is correctly connected but no data is showing up, try refreshing the page to generate a new code.

## What's next?
Check out other documentation pages. If you are on mobile, press the menu button on the top left.
182 changes: 0 additions & 182 deletions site/src/routes/docs/+page.svelte

This file was deleted.

1 change: 0 additions & 1 deletion site/src/routes/docs/+page.ts

This file was deleted.

17 changes: 17 additions & 0 deletions site/src/routes/docs/layout.svelte
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<script lang="ts">
import Footer from '$lib/footer/footer.svelte';
</script>

<article class="overflow-y-auto overflow-x-hidden w-full h-full flex flex-col items-center">
<div class="w-full pt-4 px-4 lg:px-8 lg:pt-8">
<div
class="prose dark:prose-invert prose-headings:font-normal prose-h2:text-4xl prose-p:text-base"
>
<slot class="" />
</div>
</div>
<hr class="w-full mt-12" />
<div class="w-full px-4 lg:px-8">
<Footer className="w-full" />
</div>
</article>
25 changes: 25 additions & 0 deletions site/src/routes/docs/osc-app/app-installation/+page.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Phiz includes an app that enables you to receive data from the Phiz website.
Review the [documentation](https://www.phizmocap.dev/docs) to learn how to use the website.

## Download the App
Download and install the app for your platform [here](https://www.phizmocap.dev/downloads).

Phiz is a free open-source project without funding.
**Due to the lack of a paid yearly developer license, the app will not be signed, and your OS will display a warning.**
You can still run the app by following the instructions below.

## Windows
After downloading the app, your browser may display a warning. Simply keep the file.
![windows-download](https://github.com/SpookyCorgi/phiz/blob/main/assets/docs/osc-app/app-installation/windows-download.png?raw=true)

When you click on the app installer, Windows will show another warning. Click on `More info` and `Run anyway`.
![windows-install](https://github.com/SpookyCorgi/phiz/blob/main/assets/docs/osc-app/app-installation/windows-install.png?raw=true)

## Mac
During the initial installation attempt on a Mac, you might encounter a warning without an option to proceed.

![mac-install](https://github.com/SpookyCorgi/phiz/blob/main/assets/docs/osc-app/app-installation/mac-install.png?raw=true)

To resolve this, go to `System Preferences` -> `Security & Privacy` -> `General` and click on `Open Anyway`. Then, install the app.

![mac-settings](https://github.com/SpookyCorgi/phiz/blob/main/assets/docs/osc-app/app-installation/mac-settings.png?raw=true)
35 changes: 0 additions & 35 deletions site/src/routes/docs/osc-app/app-installation/+page.svelte

This file was deleted.

1 change: 0 additions & 1 deletion site/src/routes/docs/osc-app/app-installation/+page.ts

This file was deleted.

2 changes: 2 additions & 0 deletions site/src/routes/playground/+page.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,8 @@
model = gltf.scene.children[0];
//make mesh global for mocap
model.traverse(function (obj: any) {
//prevent obj disappear at zoom
obj.frustumCulled = false;
switch (obj.name) {
case 'Wolf3D_Head':
head = obj;
Expand Down

0 comments on commit a65b765

Please sign in to comment.