Resource Center



BA Insight Meets Microsoft Azure Cognitive Search

Liam Cavanagh, Massood Zarrabian

BA Insight announced the immediate availability of BA Insight Search for Workplace powered by Microsoft Azure Cognitive Search and Services. This solution is a first of its kind, reducing store-by-store search and bringing knowledge to users, regardless of where the information is. But, what does that mean for you? Liam Cavanagh, Principal Program Manager for Azure Cognitive Search at Microsoft, and Massood Zarrabian, our very own CEO, are here today to walk us all through the vision in the underlying Microsoft technology, why organizations should consider it, and how BA Insight’s solution creates an incredible opportunity for your users.

Links & Notes

Massood Zarrabian
Massood Zarrabian
Liam Cavanagh
Liam Cavanagh


Pete Wright: Welcome to Shared Insights: The Podcast from BA Insight. My name is Pete Wright and I am joined today by our very own chairman and CEO, Masood Zarrabian. Welcome to the show again, Massood, number two.

Massood Zarrabian: Hello, everybody.

Pete Wright: Great to have you back. And with us for the very first time, welcome Liam Cavanagh, Principal Program Manager for Azure Cognitive Search at Microsoft. Liam, it is great to have you here.

Liam Cavanagh: I really appreciate you having me here. It’s great, thanks Massood, Pete. Looking forward to it.

Pete Wright: Last week, BA Insight announced immediate availability of BA Insight search for Workplace powered by Microsoft Azure Cognitive Services. This solution is the first of its kind, reduces store-by-store search, bringing knowledge to users regardless of where the information is, but what does that mean? Liam and Massood are here today to walk us all through the vision in the underlying Microsoft technology, why organizations should consider it, and how the BA Insight Solution creates incredible opportunity for your users.

Pete Wright: Liam, let’s start with you. Tell us a little bit about Azure Cognitive Search. It’s built on the foundation that is Microsoft natural language stack, Azure, there are a lot of brand names around this thing, but it is fundamentally something we know and love, right? It’s like Bing and Office and we’re living the legacy of this wonderful history. What is today’s vision of Azure Cognitive Search and how has it evolved?

Liam Cavanagh: Yeah, so Azure Cognitive Search, we started this, it’s actually been in production over five years now, we call it generally available. But the goal of this service has always been to make it easy for developers to build a great search experience over their data. So unlike your example of Bing or Google, it’s a great solution for searching the web, searching the general content. But what we see is that there’s a huge amount of people that have their own data, their own documents, their own data and data sources that they need to have that type of the capability to search over.

Liam Cavanagh: And that’s really what we’ve tried to do through our platform. Our platform as a service. Enable it so that developers can build that type of an experience.

Pete Wright: Okay. So then let’s talk a little bit about why organizations should be considering Azure Cognitive Search. Who’s your prime customer use case and walk us through some of the benefits.

Liam Cavanagh: Yeah, so that really goes back to the reason why we started it. A lot of people look at a lot of different technologies on the market. Things like Elasticsearch, Solar, really great open source technologies that have a lot of capabilities. And what we started seeing in our Azure cloud environment is that so many people were using this, but what we heard from them is that it’s very challenging to manage them. It’s challenging to make sure that I can make it available in all kinds of different regions. And that’s where we started. We started to think, okay, what can we do to make that simpler for people? Simplify the interaction with it, simplify the architecture, make it easy to scale up and scale down.

Liam Cavanagh: And that’s where we saw a lot of value in the technology that we offered. But then we also saw that, Elasticsearch, great core technology, which is actually at our core as well. But we started thinking, okay, Microsoft, as you mentioned through Bing and Microsoft research and all of the AI that we’re doing, we started saying, okay, how can we add in all of these things that we’re building in Microsoft and add it on top to make even better solution. And that’s what we’ve been trying to do. And that’s the things that we’ve tried to enable through this solution that we’re talking about today, as well as for other customers, developers building solutions.

Pete Wright: Well, and that sounds like exactly where Massood gets very excited.

Massood Zarrabian: I do, Pete, because I’ve been in the enterprise, I’ve been in the high tech market for over 30 years and I’ve been in enterprise search market for seven. And one of the things that worries me about enterprise search is how solutions are getting more and more closed. And as they get closed, the people who run it lose control over what they need to do to help their users. And then user complaints end up with pressure, right? They complained, but there’s nothing you can do. And as they get close from a user interface perspective, you also lose flexibility. So imagine that you use the internet and then you search for something, you got the wrong results over and over and over again, and there’s nothing you can do about it. Would anybody use internet? I doubt it.

Massood Zarrabian: So my view and the Insight strategies open the environment, give the customer, the user, the ability to control their destiny, make search internally better and make it so that people can accomplish their jobs. And to me, when you look at Azure Cognitive Services, it actually brings that opportunity to IT organizations, to users and to us. So we embraced it because of that, because of the openness. It does have a lot of good features, but I think the openness is the thing that is the kingpin of this strategy. And then you’ve got good technology, you have AI, you have natural language processing, you have a lot of other good things.

Pete Wright: You set up my next question, why did BA Insight partner with Microsoft to launch this new service powered by Azure Cognitive Search? It sounds like the benefits fit hand to glove, what you already with BA Insight aspire to do.

Massood Zarrabian: They do, and when you compare Azure Cognitive Search at the search index level, as Liam said, open source based on Elastic core scalable, so the search engine by itself is very competent. And it’s arguable if Azure’s is a little bit better or a little worse, and it can be biased. But with Azure comes a whole bunch of AI. You don’t get that with Elastic, right? You can go with Elastic and then you still need Microsoft, Azure, or Google, or somebody. You still need to do something about all these other technologies we need to make eternal search be as good as it is external. With solar Lucidworks, you get some, but it’s still you’re limited. You’re not spending the billions of dollars that Microsoft is spending on AI and trying to keep up with it is just impossible. So my view is we are going to focus on what we are really good at, which is enhance search engines and AI, and make it available to consumers inside the company and not try to compete on the AI front or search index front, because that is a losing battle long term.

Massood Zarrabian: It’s actually bad for the market, and my opinion is bad for the customers Just look at what happened with Google search appliance, then Google changed its mind, Oh, I’m over, it’s done. Thousands of companies had to migrate and move at an enormous cost and that’s just not good for what we’re trying to do for employees and team members.

Pete Wright: What are your intentions in that regard around meeting end user expectations, not just deployment cases, but how well does this end up meeting the expectations of those who are sitting down every day to actually do their jobs? What do they see and how does the technology further enable them to succeed?

Massood Zarrabian: Yeah. I think that Liam will have seen some of the stuff we have done, but our focus for the users is to give control to them. Historically, when you look at the past decade, when you look at how enterprise search has evolved, a lot of it has been either closed platforms, and there are a bunch of them out there, where you have an enterprise search platform that you use like an ERP system, and it costs you as much time and effort as an NERP system. I mean, anybody who has worked with SAP will tell you how painful it is to roll the system out and support it. So that’s one view of enterprise search. The other view of enterprise search is, here is a complete system, but your user doesn’t have control. I will tell you what you need to know. And with all respect to my partners at Microsoft, Cortana does that to me every morning. Every morning, Cortana comes and tells me the things I missed, and I didn’t. Right? I don’t have those action items.

Massood Zarrabian: So I appreciate they are helping me, but the reality is leave me alone. Right? So what we’re trying to do is give the users the kind of controls they need to have to be able to control their search, their destiny, the way they want to be presented. And the more we think about it in those terms, the more we move, on one side, making administrators have more control, on the other side, having users have more control. We have this concept of zero search, which is if you know Massood, and you know he works at BA Insight and you know he’s an exec and he comes in and you know he should know a bunch of things, why have him search? Why not just push it to him, right? Now to do that, Masood needs to decide what he wants to be pushed to him on one side, and the administrators need to also figure out what they want to push to you, and that combination eventually will make a perfect match.

Pete Wright: Insofar as I would love to further litigate the Cortana and the value of AI’s reminding us of things we already know, I will change gears because I have another surprise. You sent me notes, and in these notes there is a bullet containing a service, a product that I did not expect to show up in this conversation. Somebody tell me how it is that we’re talking about WordPress right now in this conversation.

Massood Zarrabian: So the way we think about the future of enterprise search is that people won’t go to a destination which is the place to go and find information. That what they do is they will search for information within the system they’re in. And we learn from somebody else to call it a workup. A workup is, if I’m in sales, I’m in Salesforce all the time. If I’m in support, I’m in ServiceNow, or Salesforce or Dynamics. So if I want to find something, you really want me to go to Office 365, UI, search for something? Probably not. The reason for WordPress and others going to follow up is extending that idea to external people is valuable.

Massood Zarrabian: Why shouldn’t WordPress people have the ability to use Azure Cognitive Services to get better results? Because if your website stinks, people are going to leave it, right? So the idea of the WordPress and other systems like that is embed what we do now as a workup in ServiceNow, in Teams, in Outlook, also externally to websites, so people can have same access and abilities externally than they can have internal. And that will make a search resource on websites a lot better than it would otherwise be.

Pete Wright: Liam, I find what Masood is saying fascinating. How does all of this fit into, again, your vision of Cognitive Services?

Liam Cavanagh: Yeah, the thing that I really like about the two products together is that we focus on the developer capabilities. We can enable certain so that a developer can build it and use it. As Massood was talking, there are so many different AI based capabilities that we’re adding through Cognitive Services that allow us to do more with the data, understand that data more effectively. We’ve talked about some of the natural language things so that when somebody provides us with some terms and words, we can understand what they’re saying and trying to present that back to the user and more of a machine learned fashion.

Liam Cavanagh: What we don’t offer is the solution, the capabilities on top of that, that allows that user to build and actually really use that. And that’s the part that I really like about this product that we’re talking about today, is that there are so many ways of taking these underlying search capabilities and exposing it, bringing it to the users where they need it. So for example, if I happen to be in Microsoft Teams, the great part about this is that I can then search for that and find my content.

Liam Cavanagh: If I want to leverage all the ranking and understanding of the information or bring in chatbots, that is part of the BA Insight solution that really enables us to shine. So from a perspective in the future, what we’re trying to do is continue to enhance and add in more, really AI-based capabilities so that we can then allow the BA Insight solution to better leverage that and just make that solution better.

Pete Wright: I know you both have some example use cases that we want to riff on a little bit. So in the spirit of the jazz format of our podcast, where would you like to start there? I know we want to talk a little bit about connectors and to your point, how we can continue to embrace various stores and make them work together, we’ve got the importance of AI. How would you like to start?

Liam Cavanagh: So I’ll start with a couple of examples. I’ve been able to work with Masood quite a bit over the last many months with numerous different customer opportunities, and what I’ve seen consistently is that these organizations that have this content, it’s very rare that it’s coming from one place. I might have data sitting in file systems and SharePoint in Oracle, or Documentum, all these various different places where data can be coming from. And so it’s very consistent healthcare manufacturing, finance, insurance, that there’s this vast amount of data coming from different places.

Liam Cavanagh: And so one of the scenarios that we’re seeing very common is that people want to be able to unify all this information so that I can look for it and find it really effectively. And that’s just been such a great link between our two technologies, because we as search, if we have that data, we can make it available and searchable and use all the things you need to do. But through BA Insights, and I think we’re talking about some of the connectors, being able to just tie all these different systems together to get that information and unify it really opens up a lot of opportunities for us to help organizations.

Massood Zarrabian: Yeah, same alignment. When I think about the success of the web, internet became successful because it can provide information to you independent of where it resides. If I go on Google and I say, I don’t know, 50 inch TV, Amazon is there, Best Buy is there. If I say trip someplace, right, Expedia is there, TripAdvisor is there. I don’t have to think about which site has the best information, right? It comes to me, it’s natural, it’s relevant, I click on it, I move on. That is the experience you want to have inside your business. That’s the experience you want your employees to have. You want them to stop thinking about, okay, where is this information? It’s funny, actually, Liam was saying this, one of the opportunities we worked together, the customer told us that once a week, they have a two hour meeting with a bunch of experts, but employees come to the meeting asking you what system has this information in it. Think about it. Think about what you’re doing, right?

Massood Zarrabian: And then they follow up by saying, “By the way, we lose 10% of our people every year.” We are not going to ever find it and then if you keep this up, eventually nobody’s going to be around to tell you what that information is, so why do it that way? Why not just connect everything to each other, make it so you have a UI, but people can find information and make it easy to have had an internet for your enterprise. Why not?

Liam Cavanagh: Yeah, it’s true. I definitely reiterate what Massood was saying. So many organizations I talk to, I ask them, “What do you do today?” And they say, “Oh, well, when I need to find this, I find the person that’s most senior in the organization and he may know where it is, but it’s not necessarily true or accurate or easy to find.” So the vast majority of their time is just spent finding a potential item to potentially get what they’re looking for. And I think that one of the challenges that until recently that’s been hard for people is that, because this content is in so many different places, it’s in so many different formats, for example, sure we have PDF and Office documents, which is one type, but I go to another organization and they have their own custom file types or own custom data stores.

Liam Cavanagh: And it’s just been really challenging until today to really get access and really not only get access to that underlying information, do OCR over the images if we need to, but then really leverage AI to not only get that data, but extract meaning from it. Now, all of a sudden, because we have these capabilities to actually do that, now we can actually get to the point where we can actually make it accessible and make it usable by those users. So I think when you ask why has it been challenging, it’s just been so hard to really unify and actually fundamentally get access to that information has been a big part of that.

Massood Zarrabian: Too often when you talk about enterprise search, people think everybody in the enterprise is going to use this. And they probably know about internal people having the ability to find information that could be one of our customers has it only for the sales organization, because that was the critical business issue. When I speak with people who think about enterprise search, we always start talking about the search engine. Honestly, the search engine is the least important part of this. The search engine is invisible to people, the difference in the search engines don’t matter. What’s really important is that in enterprises you have documents. On websites you have pages. You know how much money the team spends on web pages so when you search for something it shows up? Because if he didn’t, SEO will not find it, right? But in the enterprise, you have to find a page and a document. That’s more complicated than having a page that’s already been tagged by some marketer.

Massood Zarrabian: So coming back to what Liam said, the ability to take advantage of text analytics, to understand what’s in the document, understand what’s in the pages, extract it so that when you ask the question, I show you the page, not some 150 page document that you have to download, open up and look, is critical. You can’t do that without connectivity to other systems, we cannot do that without having AI kinds of technologies. And as scary AI looks, honestly, it’s getting commoditized. It’s becoming available, that low cost, to the user, to the customer, right?

Massood Zarrabian: Just like it’s on the web. On the web, you’re going to ask a natural language question, you never worry about costs or anything else. You go inside a company, you have to ask a text-based question, which chances are, is not going to give you the right answer. So AI making all these wonderful things is maturing what used to be enterprise search way away from the need for a search engine to the need for a platform. To me, AI Cognitive Services is like an operating system for future of enterprise. That’s what it is. It’s an operating system, it’s not anymore a search index, it’s not anymore AI, it’s an operating system and we are building on it.

Liam Cavanagh: And Massood if I could just add on too, I just want to reiterate one thing that you said, which is so important, which is that question and answerability. Because sure, it’s great if I can put in some search terms and it says, “Hey, here’s a document that’s available,” but that’s not what all users want. There are all kinds of different cases. Sure, in some cases it’s great, I’m doing research, give me all the documents that are relevant. But you mentioned the idea of find me the section in the document that’s most relevant. But probably more common that we’re seeing more often is, I have a question, just tell me the answer to it. You have all this content, you have all this knowledge, why can’t you just tell me what the answer is rather than making me open up a document or find the right page? And that is really critical, because when people think of search, there are all kinds of different tasks that users are trying to perform and solve, and by allowing us to enable all those types of things, it just provides a much better user application or implementation.

Massood Zarrabian: That’s actually a really good point. I completely forgot about that earlier. Because we had one customer who said, “Here is my FAQ document.” It was 27 pages. Would you open it twice, the FAQ document and go through it? Why can’t you just give me the one FAQ I need, right? But that’s the kind of a thing that’s going to change, right? We’ll go away from creating these FAQ documents that you sent in life sciences to people so people can support their users and replace it with exactly what he said. You ask the question, you get an answer. Does it matter what document it was in? Probably from tracking purposes it does.

Pete Wright: You have potential customers listening to this. What is the one thing you would like them to take away in terms of envisioning their future state after implementing this technology in their organization?

Liam Cavanagh: I think it really comes down to if your organization has a vast amount of information and it’s not necessarily just in files like PDF and Office documents, it could be in databases, it could be in all the different places, and you want to let your users be able to find answers within it, if I want to be able to understand that content and be able to extract out the knowledge from it so that I can ask questions and answers, I want it to let people search and be able to find the most relevant information really, really quickly. I think they should really take a look at this solution. I think it really offers a lot of capabilities to enable not only this, but actually bring additional insights that you may not have even realized was even there in there. And so that’s what I would really like people to think about in their organization. And if so, I really hope that they’ll take a close look at this.

Massood Zarrabian: Yeah, so I would love to have companies have their own internet. Not Intranet. Intranets are not good. Internet is good. In my vision, and what I would love to see, is people to actually have internet inside the company where employees and the contractors and consultants can find information, not worry about where they are, not worry about where the information is, get answers to their question, become more productive, being more successful, be less frustrated and turn around and say, “You know, I don’t have to go outside my company to find information. I can find it here.” I had a customer who told me, “My employees telling me that it’s easier to find information outside the company than it is inside a company.” What a shame, right? What a shame to have that situation. So a lot of things that Liam talks about are enabling technologies, but I think that should be the vision for companies. Why should your employees not be able to have the respect they deserve to find information they need when they need it independent of where are.

Pete Wright: Fight information hostility. There we go.

Massood Zarrabian: Or vote for information democracy.

Pete Wright: That’s right. Vote information democracy. I know my ticket. I stand with Massood. Gentlemen, I would love to give you an opportunity to share some links. I’ll tell you, I’ll be putting them in the show notes because some of them are kind of long. Start with Liam. Where would you want people to go to learn more about Azure Cognitive Search?

Liam Cavanagh: Yeah. So we have a documentation, which I think we’ll share, that has a lot of details on how to understand what the service is, get more details on it. I think that’s a great place to start. And then within that, we have all kinds of different tutorials and training materials, resources so that you can really start leveraging the product. So hopefully we’ll be able to share that.

Pete Wright: Check the show notes. Absolutely. Massood, where would you like people to go to learn?

Massood Zarrabian: What we’re going to do is we’re going to put a PowerPoint together that showcases how our solution works with Azure Cognitive Services, Azure Cognitive Search, where you can click through it and see it built instead of show you that. We are also going to give people a couple of links. For sure one to our Azure Cognitive Services solution, two, something about enterprise search and the things to do that make it make you successful. But a few things to education about how to think differently. This is the third decade of the 21st century, right? We need to move on and get away from 1990s.

Pete Wright: Yes we do. Yes, we do. This has been a great conversation, gents, thank you so much for taking the time to teach us all a little bit about Azure Cognitive Search. Liam Cavanagh from Microsoft. We’re just grateful that you took the time to be here, I hope this is not the last time we get to chat.

Liam Cavanagh: Oh, I hope so as well. Thank you so much for having me here today.

Pete Wright: And thank you, Massood, for joining us, number two. Now we’re on a roll, now you’re a regular.

Massood Zarrabian: I am. I have two of them, right, that’s right.

Pete Wright: Thank you everybody for your time and attention for downloading and subscribing to this show, we certainly appreciate all of you. You can learn more about the show over at Subscribe wherever the finer podcasts are served. On behalf of Liam Cavanagh and Masood Zarrabian, I’m Pete Wright, we’ll catch you next time right here on Shared Insights: The Podcast from BA Insight.