When it comes to building solutions with AI you want to be smart. Don't just run in there and start using Tensorflow. There are many good solutions such as Clarifai out there that don't require a lot of work, but still produce superior results.
I learned that again and again when building various experiments for customers. One of the experiments I'm working on right now is a celebrity scanner. This is a fun demo that demonstrates the power of AI when it comes to recognizing faces in pictures.
The application allows you to take a picture with your webcam. It takes this picture and scans for similarities with famous celebrities.
It's quite easy to make this app. In this post I will show you how.
Ready made AI with Clarifai
The goal for the celebrity scanner 3000 is to demonstrate how much fun AI can be. We combine the app with a poster that explains how a computer thinks.
It turns out its a pretty complicated topic, especially since image recognition requires a lot of code and training time to get right.
If you want to build something like a face recognition model yourself you will have to be prepared to train the model for hundreds of hours. Also it requires a deep knowledge of how a neural network works.
Now I can imagine that you don't have the time for that. And frankly for most applications you don't need to spend a lot of time to apply AI effectively.
With a tool like Clarifai you have face recognition within minutes rather than hundreds of hours. Clarifai is an online service that provides various image related AI models. You can invoke these models to do various things like:
- Recognize objects
- Tag photos
- Describe what is on the photo
The people over at Clarifai keep extending their service with models every week. It's quite amazing if you ask me.
For the celebrity scan 3000 I used the celebrity model that is currently in Beta.
To invoke the model I've written a small piece of Javascript code on my app:
let app = new Clarifai.App({
apiKey: 'some-api-key'
});
app.models.predict(modelIdentifier, {base64: pictureData}).then(function(response) {
let data = response.outputs[0].data;
let face = data
&& data.regions
&& data.regions[0].data
&& data.regions[0].data.face;
// Ah, we have a familiar face, let's grab the concepts
if(face) {
let possibleMatches = face.identity && face.identity.concepts;
let people = [];
for(let i = 0; i < Math.min(possibleMatches.length, 10); i++) {
let person = possibleMatches[i];
people.pushObject({
name: person.name,
score: person.value
});
}
return people;
}
return [];
}
Clarifai has a great Javascript programming model in the shape of a NPM package that you can install using npm i --save clarifai. You can use this in your web project using browserify, webpack or some other method.
Clarifai is a very generic image processing API, so the response is somewhat klunky. You have to travel down a few properties to get to the meat, but once you're there you get some pretty interesting results.
Getting the data for the picture
One problem though, I don't want people to upload a picture to my app I want them to take a picture with the webcam. For this I needed to build another piece of logic.
HTML5 has the media API that allows you to work with sound and video. It is however quite hard to use and not supported in all browsers. But as usual in Javascript there's a trick for that.
Some friendly people over at pitchly made a neat NPM package called pitchly-webcamjs that makes access to the webcam really simple.
So I wrote the following piece of code to access the webcam and grab a picture:
let webcam = window.webcam;
let videoElement = document.getElementById("my-picture");
webcam.init().then(function() {
videoElement.src = webcam.videoStream;
videoElement.play();
});
function captureImage() {
let canvas = document.createElement("canvas");
canvas.width = 640;
canvas.height = 480;
canvas.getContext("2d").drawImage(
videoElement, 0,0, canvas.width, canvas.height);
let imageData = canvas.toDataURL().replace(
/^data:image\/(png|jpg);base64,/, '');
return imageData;
}
The first few lines gain access to the video element to which I want to bind the live feed of the webcam.
To capture an image you need to execute the code within the captureImage function. This grabs the content of the video element and draws it on an invisible canvas.
The Clarifai API requires image data without the additional headers, so we strip them from the image that is generated through the hidden canvas element.
Combine the two elements, the clarifai API and the webcam integration and you have yourself a celebrity scanner.
Get the code
Interested in the code? Go grab it on Github: https://github.com/infosupport/celebscan