puzzlr

A computer vision puzzle solving app

Made during HackMidwest 2018 24 hour hackathon

User takes picture of puzzle and a piece. These pictures are uploaded to Cloudinary to resize, rotate, trim, and remove the background of the pictures. App then sends request to Flask server deployed on Heroku with address of pictures. Flask server breaks the puzzle into N pieces and breaks each of those pieces into 4x4 grids and averages the colors in each element of the grid. The server converts the uploaded puzzle piece into the same 4x4 grid and averages the colors the same way. The server then compares each orientation of uploaded piece to each of the N pieces in the puzzle using MSE and returns the app a similarity heat map of where the piece matches.

Won 'Most Creative Use of Cloudinary' at HackMidwest 2018

Cameron Jump - Android, UI, Cloudinary APIs

Visaal Ambalam - Backend Image Processing, Setup Flask Server on Heroku

John Bisognano - Backend Image Processing, Cloudinary APIs

Step 1: Upload the image of the entire puzzle and enter the number of puzzle pieces

Step 2: Upload or take an image of an individual puzzle piece after pressing the piece match button

For best performance, use a white background and avoid shadows when taking the image.

Step 3: Our algorithm will show on the heat map where your puzzle piece is most likely to fit

Brighter indicates better match