Visualising Conversational AI
Python . DJANGO . REST API . Graphviz
Consider the ways in which to best communicate a conversational design between a user and a virtual assistant.
Once clear find the best method of transferring this information from supplier to customer.
The Problem
The supplier sells an AI virtual assistant solution to the hospitality industry designed as a front desk operative. The bot you could say is another employee of the customer, trained to answer questions on domain-related topics and designed to meet brand standards.
What I found through stakeholder research is that the hospitality industry runs on a bloodthirsty drive to ensure good customer satisfaction as there was a clear correlation that the better the customer experience the higher revenue. This meant the information the bot provides must be accurate, for example, if the bot gave information on an event the guest paid for such as ‘event x starts at 2 pm’ and the event starts at 1 pm the guest would miss it. This would result in the hotel losing revenue by refunding the guest or offering chargeable services for free. Here lies the problem.
This need for customer service to be perfect has resulted in military-like staff who will die for the customer’s satisfaction routinely needing to know how the bot is programmed. this triggers frequent requests to the supplier to provide something that could show the bot’s behaviour.
However, the supplier did not have a solution, and each request would take on average 3 hours to produce a spreadsheet significantly draining the resources and resulting in a lack of visibility and ownership of the solution for the customer.
This program resolves this problem.
The Analysis
My research starts by meeting with stakeholders including the customers who frequently requested this information. It didn’t take long before I learned the aim of the game is customer satisfaction, customer satisfaction, customer satisfaction. So enthusiastic about their guest’s experiences that it was inspiring. You can see the need for access to up-to-date and accurate data as incorrect information relayed to the guest can have a negative impact and result in refunds, services free of charge, or the dreaded negative online review. In a world so connected and so informed Trustpilot has become a microscope into a hotel’s experience at the fingertips of any future customer. Have you ever decided on booking a stay based on a Trustpilot review? It’s imperative the agent is up-to-date and its behaviour reviewed frequently.
The need was clear but also made clear in these interviews was the format of data it should take. This is key and a huge part of research, as you can imagine writing out an AI bot’s configuration into a text-based file like a spreadsheet for humans to understand can be complex. The data had to be graphical, but this introduces so many more questions, what type of graphic? What is the relevance of shapes? How do you show the direction of the conversation? Does colour give extra insight or cause further confusion? and finally, a list of core features requested across stakeholders was identified and added to the development.
Armed with an insight into what the customer wanted I took to academia and industry as AI is a hot topic and I won’t be the first to hit this problem. This research gave me two core themes guided by a question for each and sub-themes within. The first was access to the data, what is the best way to get this data from A to B? Based on findings, it is evident the best way to give data access is via a web portal. The supplier can centralise data which eases the management of access. Furthermore, this introduces a platform for collaboration streamlining projects, and increasing customer-supplier relations. The second theme as mentioned was data presentation guided by ‘how to present data to best understand and utilise it’. Here colour, shapes, and graphics were explored, and looking at IBM’s Watson and Google’s Dialog Flow a common theme emerged, presenting a conversational design is best achieved through flow charts. Although the true nature of the bot cannot be completely captured by these graphics, they do give the observer a strong understanding of the data.
Node key detailing action and purpose of each node in conversation design.
Conversation example between human and AI bot.
Example flow containing nodes which make up the conversation design in this instance on the topic of checking into a hotel.
Frequently asked questions page 1.
Frequently asked questions page 2.
The Code
Technical talk incoming.
Now I had the answer of how the data should be presented the goal is to design and implement a program that can extract the bot’s configuration, build a graphical output, and serve a web portal.
First, due to project constraints as other projects in the company were ongoing and that overlapped so strategically it made sense to pivot into achieving the data presentation theme first.
I implemented a program that pulled the bots configuration from an the AI suppler API and then parsed the underlying Python files that made up the AI behaviour and decision making.
Once parsed a solution called Graphviz was used to generate flow charts for each domain topic the bot was is designed to handle.
For example let’s say the guest asks, ‘What time is your check-in?’ the bot could respond with ‘Our check-in time is 1 pm, do you need to check in earlier than this?’.
For this design around check-in, the program would generate a flow chart showing exactly how the bot would respond depending on the guests’ questions and answers which you can see at bottom of this page.
The result of the program is a zip file of many graphics showing how the agent behaves on all topics it’s designed to handle.
The next theme to satisfy is the data access, the program needed to replace the customer’s request for this data, and the suppliers’ manual efforts to gather this and return it.
A web portal had been built for customers to access to manage their agents so naturally, this solution belonged here. A DJANGO API was built that fed the web portal the new output when requested. This allowed customers to log in and generate the output in minutes whenever they like.
Did It Work?
Before I tackled this problem it would take the supplier a few days to achieve this request from customers. It would take several hours and a skilled member of the team to produce a spreadsheet wasting resources.
A new solution designed specifically for this problem and satisfying both data access and data presentation has now been implemented which achieves this request in minutes but being an engineer, I love statistics, so here are a few of them.
At the start of the project, the below evaluation criteria were designed to indicate if the project was a success.
Increase Customer Confidence & Ownership of Solution.
Currently, lack of visibility does not give the customer a sense of ownership of the solution.
Measurement: 50% of customers who take the survey feel they have more visibility, control, and ownership.
Result: Out of all surveys completed 100% of stakeholders felt they had more visibility and control of their assistant.
Stronger Customer Relations.
The strain on customer relations is forecasted without a solution to the core problem. This can negatively impact the brand’s reputation, in turn affecting business opportunities.
Measurement: Positive customer feedback on the use of the new solution.
Result: When the first release was published further stakeholder interviews were held 4 weeks later to get feedback. All stakeholders expressed they were extremely happy with the new solution giving positive feedback such as “easier to use, less fatiguing, higher information density”. Furthermore, core stakeholders were asked if they felt that the features stakeholders requested were implemented and the results are shown in the graph below
Reduction of Wasted Resources
Supplier spends several hours producing the report for the customer draining internal resources. A solution will free up employees to focus on revenue operations.
Measurement: Reducing time to generate output by 95-99%, currently takes on average 3 hours to produce.
Result: The program generates an output in under three minutes a reduction of approximately 97.5%.