The promise of AI has been mostly hype over the last few decades. Since the 1980s, I have recalled promised of magical problem solving and equally strong warnings of doom and gloom. However, today the theory is hitting the pavement with a new wave of systems and services that commoditize AI-based pattern recognition and help integrate it into todays networked applications.
Clarifai (https://www.clarifai.com/) is one of these service companies that brings cutting edge pattern recognition and AI to the traditionally manual process of tagging images with relevant text terms. With their easy API, you can submit an image, and have one of their pre-defined AI “models” quickly send back terms that match your image.
When I found out about the service recently, I signed up for a developer API and started reading up on the product. In trying to grok what it was doing under the hood, I came across a very good presentation by Cassidy Williams explaining how the system works. Once I understood what I was working with I was able to quickly put together the test app you see above.
Although can you train your own neural network model, I used one of the pre-defined ones for my test. In my case, I used the “General” model. There are models that have been trained for more specific things, like recognizing food, attire, celebrities, etc. If you try to send a landscape picture for analysis by the food model, you will definitely get strange results. So, watch what you are analyzing with.
When making a request, you send the URL of a publicly accessible image for analysis, and the model to test against. The system will then return to you a set of terms, and a percentage for each term. This percentage indicates the accuracy the model things the term applies to your image. In my case, I simply displayed this value to the right of the image in the UI. To reduce the number of displayed values, I set a percentage threshold for terms I displayed.
I core of my [very] simple test app is this function that calls the Clarifai API:
You can find the rest of the code at https://github.com/davehoran/clarifaitest.