Skip to content
/ FaceEMR Public

Facial Expression Recognition in android where the predictive model built in tensorflow using convolutional neural network

Notifications You must be signed in to change notification settings

robi56/FaceEMR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

FaceEMR

Emotion Recognition from Facial Expression

The project aims to train a model using tensorflow for facial emotion detection and used the trained model as predictor in android facial expression recongnition app.

The model is trained using tensorflow python framework and used in android application where the basic langauge is java.

Basically tensorflow provides a c++ api, that can be used in android application. The trained model by python langauge can be integrated with android project after inclduing tensorflow c++ framework dependencies and using native interface the model can be loaded and called in java class. This is the whole thing.

The total work of this project is divided into two parts

  1. Devlop a model in tensoflow using python langauge

    KaggleFaceEmotionRecognition folder contains the work

  2. Devlop an android appication for facial expression recongtion

    FaceEMR folder contains the work

Part 1. Facial Expression Recongition Model developed in Tensorflow

In this work , I have used a simple Convolutional Neural Network Architecture to train a facial expression dataset.

1. DataSet: The dataset is collected from Facial xpression recognition challenge in kaggle The challenge link https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/

The data consists of 48x48 pixel grayscale images of faces.The dataset contains facial expression of seven categories (0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral

2. Model:

In this work I have used the below CNN model 

  input_image->conv2d->pooling->conv2d->pooling->conv2d->pooling->dropout->softmax

The code fragment

```
x_image = tf.reshape(x, [-1, 48, 48, 1])
    #48*48*1
    conv1 = tf.layers.conv2d(x_image, 64, 3, 1, 'same', activation=tf.nn.relu)
    #48*48*64
    pool1 = tf.layers.max_pooling2d(conv1, 2, 2, 'same')
    #24*24*64
    conv2 = tf.layers.conv2d(pool1, 128, 3, 1, 'same', activation=tf.nn.relu)
    #24*24*128
    pool2 = tf.layers.max_pooling2d(conv2, 2, 2, 'same')
    #12*12*128
    conv3 = tf.layers.conv2d(pool2, 256, 3, 1, 'same', activation=tf.nn.relu)
    #12*12*256
    pool3 = tf.layers.max_pooling2d(conv3, 2, 2, 'same')
    #6*6*256
    flatten = tf.reshape(pool3, [-1, 6*6*256])
    fc = tf.layers.dense(flatten, 1536, activation=tf.nn.relu)
    dropout = tf.nn.dropout(fc, keep_prob)
    logits = tf.layers.dense(dropout, 7)
    outputs = tf.nn.softmax(logits, name=output_node_name)
    ```

3. Result: I have used 5000 iterations with batch size 100 and restore the model in protocal buffer file

Part 2. Facial Expression Recongition Application in Android

I have used Android Studio for this application.

Integrating tensorflow dependency in android is really a tedious thing. the good news is that the latest news that android studio manages all dependencis related to tensorflow after adding the dependencies in build.gradle(Module:app) file

dependencies {
    compile 'org.tensorflow:tensorflow-android:+' 
}

The final dependency part looks like

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation 'com.android.support:appcompat-v7:26.1.0'
    implementation 'com.android.support.constraint:constraint-layout:1.1.0'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'com.android.support.test:runner:1.0.1'
    androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'
    compile 'org.tensorflow:tensorflow-android:+'
}

1. Designing the UI Components

Home Screen look like this Home Screen

After taking a picture Home Screen

And the final result Home Screen

2. Interacting with the Tensorflow Native Api

The org.tensorflow.contrib.android.TensorFlowInferenceInterface handles all necessary operation to interact with native api. See more details in https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/android

3. Finalizing the work

There are lots of options for improvement.

  1. Adding some cool features and increasing model performance using another models
  2. Tuning the hyperparameters of used cnn model

About

Facial Expression Recognition in android where the predictive model built in tensorflow using convolutional neural network

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published