Beginner’s Guide to Using Microsoft Cognitive Services Face API: Part 3

In Part 2, I analyzed a single image that was stored on my local laptop. Great! I had the program working for a single image, but I needed to analyze a large number of these images. So here we go!

In this tutorial, we’ll work through some planning, new imports from the Python standard library, write some python functions and store the results from the MCS Face API.

Hopefully you’ve played with the api in part 2 more thoroughly than I did! By the end of that tutorial, I passed a binary of a jpeg file to the MCS Face API for analysis and received a JSON response from the Microsoft servers. I was successful with my sample picture and received a response that looks like this:

<Response [200]>Response:[{'faceId': '0b830588-049b-4399-85e1-b1b53bc78c0a', 'faceRectangle': {'top': 187, 'left': 313, 'width': 266, 'height': 266}, 'faceAttributes': {'smile': 0.997}}]

I got a [200] response code which means I successfully passed my data sent via the post request to Microsoft’s servers, yay! And the Microsoft servers kindly returned some information about the image. Let’s decode the info:

“faceId”: ‘long-string-of-numbers-and-letters’ = yes, we believe a face is in image
‘faceRectangle’: {‘top’: 187, ‘left’: 313, ‘width’: 266, ‘height’: 266} = we predict the face is within these pixel coordinates of the image
‘faceAttributes’: {‘smile’: 0.997}= we predict the person is smiling with 99.7% confidence

The initial code to analyze one locally stored image works! Now I’ll need to plan a way to automate the process over many images in a local directory. We’ll be moving into advanced beginner territory from here. I’ve made several assumptions about how much knowledge you’ll need to follow along, but hopefully we’ll both learn a thing or two if we stick together. Let’s consider batch processing as its own programming project, and we’ll begin at the beginning.

Step 1: Planning

Professor Terence Parr taught me that making a work plan will go really far in helping move a project forward from wanting data to coding a program that will give you data. I’m happy to share his gospel.

Professor Parr offers a work plan template that should work for most programming projects. He recommends filling out the all the parts of the work plan that you know how to do and then figure out any unknown steps by breaking a step into its own set of tasks. Thanks to the sample starter code, we’ve done most of the work plan steps in part 2.

What’s missing is Step 2 — how to access all the files in the folder that has all my images instead of one image at time. Let’s break down what we can currently do and what we need to get step 2 of the work plan done:

1. we already can send image data information one at a time and receive a response from Microsoft
2. we need to gather all the file names that we want to process into one data structure, likely a Python list or array
3. we need to access every file name in the data structure, likely using a for-loop technique
4. we need to pass every file name we accessed to the process we wrote in part 2

We can do it!

Step 2: Mise-en-Place for data and fetching tools

Mise-en-place in the cooking world means “put in place”. You’re gathering what you need and putting the components of your recipe / program together before you execute or build the dish. The parts will be ready exactly when you need them. These following bits of code snippets, and double checking with print statements is our way checking that everything works is ready before we build our program.

Once we break the tasks down, we can start Googling to learn about the tools that we need to get the job done. For the sake of testing, this demo will be using the Faces 1999 dataset from CalTech’s Computer Vision lab. I’ll be writing the code in Sublime Text 2 and saving the file as batch_face_detection.py . The output of the file can be seen by running
$ python batch_face_detection.py

These are the imports we’ll need now. We imported requests in part 2. ‘glob’ is for gathering all the filenames in a local directory to list data structure. ‘time’ will help us avoid the hitting the MCS rate limit error by slowing our program down.

Let’s deal with glob. It’s a weird name for a library, and whoever named it must have been tired and ran out of words that day.

I’m using the glob() function from the glob library. Here, I pass in the name of the folder and the file type that I want “globbed” together into a python list. *.jpg means that I want every jpeg file, regardless of file’s name, as part of the list. These lines of code produce this output:

[‘cal_tech_faces/image_0051.jpg’, ‘cal_tech_faces/image_0052.jpg’, ‘cal_tech_faces/image_0053.jpg’, ‘cal_tech_faces/image_0054.jpg’, ‘cal_tech_faces/image_0055.jpg’, ‘cal_tech_faces/image_0056.jpg’, ‘cal_tech_faces/image_0057.jpg’, ‘cal_tech_faces/image_0058.jpg’, ‘cal_tech_faces/image_0059.jpg’, ‘cal_tech_faces/image_0060.jpg’]

Now with all our filenames in a list data structure, we can check off 2.2 of our work plan.

I only want 10 files for this demo, so I take a 10-item slice of the list produced by glob().

Python has a for-each loop method built into its syntax, which lets us individually touch or see all the image filenames in our list. Checking off 2.3 of our work plan, done!

cal_tech_faces/image_0051.jpg
cal_tech_faces/image_0052.jpg
cal_tech_faces/image_0053.jpg
cal_tech_faces/image_0054.jpg
cal_tech_faces/image_0055.jpg
cal_tech_faces/image_0056.jpg
cal_tech_faces/image_0057.jpg
cal_tech_faces/image_0058.jpg
cal_tech_faces/image_0059.jpg
cal_tech_faces/image_0060.jpg

Step 2.4 of our work plan requires some prep or refactoring of the code we wrote previously. The first iteration is good for analyzing a single image at a time, processing the image only works if you run the program. We want our image processing and analysis steps to run over and over as we pass in each filename instead of each time we run the whole program.

The solution is to refactor or edit our previous code into a function named call_ms_face_api() to complete our coding mise-en-place — we want this process to be packaged and ready-to-go.

This tutorial assumes that you understand the python function definition pattern. We’ve taken the same code (minus most of the comments) from part 2 and wrapped it in a function definition, so we can use it over and over again without having to run the whole program. Function wrappers lets us make the processes we struggled to write the first time reusable and portable.

We’re keeping the same parameters and headers for now (you may be changing these depending on what features you want to detect or if you’re passing an image via a url), but we can slip in a new local filename for processing when needed. We’re done with step 2.4! Now we have all the pieces of the missing step 2 of the work plan.

Step 3: Coding it all together

You can see in the screenshot below that we’re combining everything from our work plan in the function batch_call_images()

There’s a few things I want to keep track of in addition to analyzing the images. I have a list where I save all the responses from the call_ms_face_api() function because it only returns a single analysis at a time. I also want to count and to keep track of how many faces were not detected in my list of images because I want to check if there’s a problem with my image or the api.

You’ll also see where I used the time library; the number you pass into the time.sleep() function is seconds. So for this example, I’m telling python to wait 5 seconds before processing the next image.

If you don’t use time.sleep() to slow down how fast your for-loop runs through each example, you’ll start to see the error below. According to the api terms, you can only use it at rate of 20 images per minute, and Python will process items at a much faster rate.

[{'faceId': 'ce069d3c-fe7b-461f-bc94-5a70afeaf167', 'faceRectangle': {'top': 213, 'left': 325, 'width': 258, 'height': 258}, 'faceAttributes': {'smile': 0.026}}]<Response [429]>Response:{'error': {'code': 'RateLimitExceeded', 'message': 'Rate limit is exceeded. Try again later.'}}

You can also see i’m calling the call_ms_face_api() within the batch_call_images() function. Organizing your scripts into functions is about keeping the tasks you want to perform separate and discrete. Smaller function task means each one is more portable and reusable. I’ll defer to Terence Parr again about good function practices and goals.

If you’re a beginner, I hope you learned how to tackle a programming task and about a few interesting libraries. If you’re more advanced, I hope you found working with a computer vision api informative and fun.

Below is the full code that we just walked through together. Cheers!