We Launched Book Lending Based on Face Recognition (And How You Can Build One as Well)

We opened a biometric library for our employees, where the borrower and the book to be borrowed are identified automatically.

According to our machine learning specialist Oliver Stimmer, the borrower must look into the camera while showing the book, and the system registers the event in his or her account. “If the solution were connected with the national database of document photos, it would be technically possible to start using face recognition-based lending in public libraries as well. We could do away with the library card and literally hand out books judging by the borrower’s face,” Stimmer said.

“At Helmes, book lending is connected with the development profile of every employee and the mentors can see what people are reading and suggest additional reading to further their knowledge,” he said.

The machine-learning expert explained there was technical readiness to use face recognition for tens of thousands of new services to make life more convenient and secure for the people. “For example, Helmes staff can access the office based on face recognition. However, among the really global possibilities are searching for missing persons or checking whether a person is of legal age. Amazon already has a store where the IT system recognizes both the buyer and the goods taken from shelves and automatically charges the person’s bank account for the purchases,” Stimmer said.

A Step-by-Step guide – How to Build a Book Lending System Based on Face Recognition

Tag your books with unique identifier stickers and add those to the database.

It is suggested to add text to the identifier sticker that is unlikely to appear anywhere else on the book cover, so it would be easy to detect and parse later. For example, at Helmes we use this format: “Helmes Book – [unique number of a book]”.

Train the facial recognition machine learning model.

Helmes uses Microsoft Facial recognition REST APIs to send the person’s photo (face clearly visible) and info on who the person is (name, username, ID, etc.) to the API. One photo is enough for training recognition.

  1. Create Person group. In the person group, all the photos of the persons which the algorithm can recognize are kept. It is a folder for keeping photos of people that the algorithm can identify. (A)
  2. Create a person (B)
  3. Add a face to Person (C)
  4. Train person group (D)

Fetch the photo from the web camera.

HTML 5 allows accessing any web camera with a few lines of code. This photo will be used both for facial recognition and optical character recognition. (E)

Send the photo to trained machine learning models (facial recognition and optical character recognition).

Send the photo containing the person’s face (clearly visible) and book tag to both recognition APIs.

  1. Detect face (F)
  2. Identify the person (G)
  3. Identify characters on the book label from the photo (H)

Receive information on who’s on the photo and the text from the label from recognition models and register the book to the user.

Use Regular Expression to parse the Book ID from all the text recognized from the picture. Save the book borrowing /return event to the database.

What is What?

Helmes is an international software house headquartered in Tallinn, with clients all around the world. Read more about Helmes.

A book is both a usually portable physical object and the body of immaterial representations or intellectual object whose material signs — written or drawn lines or other two-dimensional media — the physical object contains or houses. ?

Additional resources

(A) (https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f30395244

(B) https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f3039523c

(C) https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f3039523b

(D) https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f30395249

(E) https://www.html5rocks.com/en/tutorials/getusermedia/intro/

(F) https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f30395236

(G) https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f30395239

(H) https://westus.dev.cognitive.microsoft.com/docs/services/56f91f2d778daf23d8ec6739/operations/56f91f2e778daf14a499e1fc