R&D: Tackling natural language search for travel companies

Earlier this year we were invited to take part in Microsoft’s AI hackathon, giving us access to expert Microsoft experience and insight in tackling a client problem with AI.

We’ve been working with AI technologies for a while, being one of the first companies in the UK to develop for voice assistants, ahead of the release of the Amazon Echo and Google Home in the UK. While working within frameworks like Alexa and Google Assistant has provided us with great experience in assistive technologies, we’ve been keen to explore beyond these, and build solutions over which we have more granular control of the natural language processing (NLP) within our solutions.

Recently we’ve been working on solutions for clients which introduce smarter features into their projects using Microsoft’s Azure Cognitive Services. Azure offers a suite of ‘building blocks’ which allow solutions designers and developers to develop smarter systems for their businesses. Our challenge going into the hackathon was to see how we could use LUIS to make site search smarter. LUIS is ‘a machine learning-based service to build natural language into apps, bots, and IoT devices’, and one of Microsoft’s key language understanding components.

Our objective: Create a prototype application using natural language querying that would allow users to search for travel routes within a website search box.

Technologies used: Azure Cognitive Services (LUIS and Cognitive Search), Azure Search, Azure Function App, Azure Maps, Azure SQL, Azure Storage

A well defined mission

We arrived at the hackathon with a pretty well-defined mission: to build a solution which would allow web users to search for travel routes using natural language, rather than having to set parameters using a search widget. We’d also set out a list of the Microsoft technologies we wanted to investigate to achieve this: particularly LUIS and Azure Search. While we had a high level plan of what a solution could look like, it was quite broad, and its elements ill-defined due to the lack of expertise in some of the technologies.

This drove our second goal; to gain a working knowledge of the AI-oriented services that we might use for this solution or in future projects, by taking full advantage of the expert Microsoft staff present at the event. The main questions we aimed to answer were ‘how’ and ‘if’ we could use our chosen technologies to power our solution.

A crash course in cognition

By the end of the event, pretty much all Microsoft staff present had been to our table, chatting, sharing knowledge, helping and contributing. With so much expertise in the room, it was a challenge to focus on our task while trying to assimilate everything they had to offer, and at the same time understand if the knowledge they had to share suited the problem we were tackling, or if we were trying to accommodate it because we didn’t want to pass up the opportunity to learn.

After three days full of crash courses on all things Microsoft AI, some coding and much harnessing of Azure's smarts, we’d achieved our goal and built a working prototype that could power a search-based application, and a system architecture to support it. More importantly, we learned a lot about technologies we hadn’t previously had the chance to work with, and left with a better understanding of what cloud services we can use to enhance, or power, future solutions.

Now, about those Azure smarts we used:

LUIS

LUIS is a Microsoft cloud API-based service that allows natural language interpretation to be integrated into a solution to retrieve information using a pre-defined and trained model.

The portal UI, and the creation and customisation of the language model through it, is similar to Amazon’s Alexa portal, but much more flexible and detailed, providing more information when testing intents, and more capabilities when creating and enhancing utterances and entities. There’s more to be discovered about LUIS than we were able to get to in our 3 days in London, but having experts on hand helping us model our solution was a big booster to our velocity, especially after running into early issues where LUIS failed to recognise any of the utterances we served it (although it turned out this was our fault).

Azure Search

Again a Microsoft cloud API based service, Azure Search is really easy to set up. It’s aimed at search-based solutions and allows the creation of a queryable index from a data source that can be modelled to serve any context needed, providing great throughput and performance. We had some data stored in an Azure SQL DB, at which we pointed the indexing service, and were happy with the job it achieved out-the-box, without any extra fiddling required. We then used this as one of the main data providers for our PoC solution.

In terms of configuration and capabilities, we were only able to scratch the surface during our projects. We didn’t have the time to try it all; there’s a lot of fine tuning and configuration capabilities, not to mention quite a few features that can be added (like result ranking and term weighing).

One thing we did add into our solution was the Cognitive Search feature, which uses artificial intelligence to extract insights and structured information from your documents. I found this interesting, as I’d pointed it at an Azure Blob Storage container containing a few PDFs with data, and it created an index of it. Unfortunately, as this was not a core requirement requirement of our time-limited project, I wasn’t able to spend too much time with it or customise the index structure, so the output of it was of limited use. Still, even with only a brief experience, I can see this being a very powerful feature, if used accordingly.

“Glasgow to Aberdeen 2pm”

From a user’s perspective, they ask a question on how to get from A to B (at time C), and are served the relevant timetable and route information (for now, as the future holds more detailed responses). From the system’s perspective, it processes a natural language query into keywords. These are then used to form and parametrise queries against various data sources, from indexes to relational databases, to geographical systems, retrieving relevant data and summarising it into the information requested.

The system we‘ve built uses Azure Functions App as an entry point API, that sends LUIS the user query. LUIS provides the intent, and the entities, as modelled. With the key information that LUIS extracts, the request for the Azure Search Index is formed. It’s response is a collection of information for each of those terms, in the shape of the indexed data. This information then allows for a target SQL Query to be formed, against an Azure SQL DB table, that provides the final piece of information needed to respond to the user query.

Azure Maps

I’ve mentioned Azure Maps as well, and while we never had the time within the hackathon to integrate it into the end solution, we explored it enough to hack together a prototype future feature. Users can provide a non-specific destination, such as the island of ‘Arran’, and be offered up travel options to several of the ports on the island. Under the covers, this maps the point of interest specified by the user’s query (in this case, Arran) to a geographical polygon representing it. Once the limits of the polygon are known, we can query our data to find any ports that fall within those limits and serve the appropriate result.

Now, there’s a bit more to that, as some points of interest will return multiple results (the main islands of both Orkney and Shetland are called ‘Mainland’), and some places that have ports on them (also known as islands) are formed of more than one body of land. Again this is a service we weren’t able to fully explore within our timeframe or challenge context, but one we’re keen to explore more to bring additional functionality to our solution.

While we can’t reveal too much about our current work with clients, check out our other work in this space, and get in touch if you're looking for an experience partner to explore how emerging technologies can boost your business.