C is for Cognitive Services

It’s a frequent and time-honored tradition; a child goes to their parent after dinner with a homework assignment they need help with. “Can you help me write a story due tomorrow?” That is often quite an ask of a parent with little preparation. Constructing a story is a difficult task for many people. One of the first questions that comes to mind is ‘Where do you start?‘ What is the story supposed to be about, and how would you gather the information to link the concepts together? Wouldn’t it be nice if there were an application that could assist and get them to bed on time! Just like that, an app idea has been found.

Question–what capabilities would you need to build an application like that? First, perhaps remove the writing portion and tell the story. Also, if the application could search broadly on general topics and drive deeper when needed, that would be ideal. Finally, if real-time feedback on searches could be found through reading an individual’s facial cues signifying emotions, that might speed the process along, as well. Microsoft Azure’s Cognitive Services provides API’s and has the capability to uplift the applications possible. You can now shift from purpose-built solutions to those that not just meet the goal of an application, but reach out to your audience and allow them to interact at a human level.

How to tell the story would first begin with LUIS. The Language Understanding Intelligent Service, or LUIS, has the purpose of understanding what a person wants, and to find the pieces of information that are relevant. Additionally, through the use of the Custom Speech Service, which eliminates speech-recognition barriers, such as speaking style, background noise, vocabulary and Translator API’s, you have the ability to make the application global and viable in almost every setting. Now, to take the concept of your story broadly, the Knowledge Exploration Service (KES) provides interactive search and refinement to applications. Combined with Entity Linking Intelligence API’s, which can apply context to two separately distinguishable entities such as Pluto the Disney character and Pluto the exo-planet, searches would be more targeted in the context of the story.

Through all of this, how does the application “know” it’s on the right track? Cognitive Vision services such as the Face and Emotion API’s have the capacity to gauge the author of the story’s reaction to searches and the content. From happy to sad and a wide variety of other emotions, the Emotion API would allow the application to interact at a more personal level, providing insights into new directions.

Now, that time -honored tradition of the surprise creative writing assignment could resemble a classroom of children – excited to be working on an assignment because it’s fun. Regardless of “studies have shown in classrooms…” statements, common sense and human nature tell us that if we enjoy an activity that promotes learning, we will retain the information and incorporate it in our lives. Take a look at the children in the image. If an application could teach them more about the world and incorporate what they learn with their imagination, would they use it? Would you?

Microsoft Azure’s Cognitive Services has the potential to uplift applications and ideas from how we use and implement them today. The creative writing example has touched on just a few of the capabilities that are at your disposal. There are many applications for Cognitive Services use within organizations today. So, as a final thought, what is the answer if you took yourself out of the situation of “this is how we do it” and asked “can we take our applications to the next level with Cognitive Services ?”.