Is the Single Index the Next Dinosaur?

A New Dinosaur for Indexing technology

After my last blog, I received several questions about use cases for a federated approach. So, I have written this follow up blog to share real-world examples of the power of federation. I will also introduce the “in-app” experience – the idea of bringing search to where your information workers live.

We work with some of the largest law firms in the world, and many of these firms use NetDocuments as their ECM platform.  (Full disclosure: BA Insight is a NetDocuments technology partner). This particular customer has 100+ million documents in the NetDocuments platform.  Monthly, 5 to 10 million items within the index change. Since the firm uses SharePoint Online, they considered using SharePoint Search for their enterprise search solution, using a BA Insight search connector to crawl NetDocuments content.  Lack of control (SharePoint Search is a black box) and pricing concerns (Microsoft charges for adding external content into the index) steered them away from that option.

Since they scale to billions of records, both Elasticsearch and Azure Cognitive Search looked like options, and they also offer flexibility (relevancy tuning, etc.).  However, the firm doesn’t have any internal expertise with Elastic or Azure.  They would have to hire additional staff to support the solution, and it just wasn’t making sense.  There was also a more critical issue.  With the rate of change in their DMS, even though we do incremental crawls in almost real time, there was still a possibility that search results would display out of date content or miss new content all together.  This was completely unacceptable to the firm’s partners.

That’s when they decided to go with a federated approach.  Leveraging BA Insight for NetDocuments, the firm still searches for existing pleadings, contracts, etc, in NetDocuments, but now with federation they have a window into all content that resides in SharePoint and NetDocuments.

While working on a document from NetDocuments, the associate can quickly look up billing information stored in the SharePoint Search index.  Or, they can search case law information from their LexisNexis subscription (another example of the power of federation), all while never leaving NetDocuments.

Ok, let’s look at another use case where federation shines.

Another customer presented us with a unique search problem.  The firm had four regional SharePoint farms, but they had no unified search capability. This means users had to go to four different places to search, and because of privacy laws, the customer was restricted to creating a single index out of the four.  Otherwise, it would’ve moved data across borders, and they had concerns about the size of the index given SharePoint’s issues with large indexes.

Using our software, the firm is now able to federate the index of each remote farm, providing a single pane of glass, meaning one place to search for all of the firm’s intellectual property.  Just think of the benefits.  This is a case in which going from four to one has a tremendous impact on productivity and the employee experience.

Consider a litigator is San Francisco who is working on a tort liability case.  He/she has an upcoming hearing.  To get ready, this person can now conduct a search and be exposed to helpful content created by an attorney in the New York office (stored on the east coast farm). Even better, with Smart Previews, the lawyer can preview content instantly.  Then, using our workspace functionality, he/she can copy passages to the workspace for repurposing into new content.  Why invent the wheel when the wheel already exists?

A firm’s content is the core of the firm’s intellectual property.  Increased visibility enabled the firm to more effectively leverage and repurpose existing content, saving time and money.

The day of the single index is going away.  Extinct.  Like the Dinosaur.  Multi-index and federation under a single pane of glass offer the most value.  And, it is the future.

In my final blog installment, I will further highlight the value and use cases for “in-app” search.  The concept that search is no longer the destination.

Remember:  Stop sifting.  Start asking.