Skip to content
BA InsightBA Insight
  • Request a Demo
  • Request a Demo
  • Products
    • BA Insight for Amazon Kendra
    • BA Insight for Amazon OpenSearch Service
    • BA Insight for Azure Cognitive Search
    • BA Insight for Elasticsearch
    • BA Insight for Microsoft 365
    • BA Insight for Microsoft Teams
    • BA Insight for NetDocuments
    • BA Insight for ServiceNow
    • BA Insight for Solr
    • WorkHub Search
  • Technology
    • SmartHub
    • Connectors
    • AutoClassifier
    • AppBus
  • Solutions
    • Enterprise Search
    • Customer Portal Search
    • Website Search
    • Search for Legal
    • Search for Life Sciences
    • Clinical Trials
  • Resources
    • Podcasts
    • Videos
    • Case Studies
    • White Papers
    • Product Resources
    • Newsletters
    • Articles
    • Webinars
    • Blog
    • News
  • About Us
    • Company Overview
    • Partners
    • Contact Us
Blog

Adding Machine Learning – Introducing our New Release of AutoClassifier

Posted on February 1, 2017June 28, 2020 by Jeff Fried - CHAIRMAN OF THE ADVISORY BOARD

Last week we announced a new version of our AutoClassifier (you can read the announcement here). This release has many new features and capabilities, including the addition of machine learning into the AutoClassifier.  There is remarkable power in machine learning, and we’re excited to add this to our products.

The trick has been doing it without sacrificing transparency and control.  I’ve been in the industry long enough that this is the fourth auto-classification product I have brought to the market, and I’ve seen many sophisticated techniques fail because they are “black-box” algorithms that you can’t understand or control.  What has made our AutoClassifier successful is that you can get going quickly, understand exactly how and why you get the metadata it generates, and control it as precisely as you want.  Machine Learning is notoriously opaque, so we are introducing it in a controllable way.

Combining Rules-based and Machine Learning in a Single Product

We’ve kept our rules-based classification engine as the heart of the AutoClassifier, and added additional modules.  In particular, we’ve added:

  • Semantic Entity Extraction: a new machine learning based software module that recognizes sequences of words which are the names of things, such as people and company names, or gene and protein names.  It ships with pre-trained recognizers for people, places, and organizations, but it is trainable for a wide variety of additional entities.
  • Integrated Smart Pipeline: allows you to apply different text analytic techniques in any combination.  For example, you might do clustering based on semantically extracted concepts, or auto-tagging using a set of classification rules, and then link in associated data for tags that exceed a specified confidence threshold.

Our AutoClassifier now has multiple text analytics capabilities that can be combined in many different ways.  Its overall function – creating metadata – is the same, but you can now combine rules-based and machine-learning approaches to provide consistent, high-quality metadata.  Organizations can use the resulting metadata to improve findability, discover new insights, and ensure compliance.

rules engineWe reached the conclusion at BA Insight that hybrid solutions give the most flexibility, and the best results, which is what prompted us to come up with this new release of the AutoClassifier.  We allow our customers to apply the best technique, or to combine techniques in innovative ways when that’s needed.  Our customers benefit from leading-edge text analytics technology, and also from the flexibility to apply our product in different ways to different content or different applications.

Machine Learning versus Rules-Based Classification
– the End of a Debate

Back in November I did a session at Taxonomy Bootcamp explaining Rules-Based and Machine Learning-Based approaches to AutoClassification  (you can download the slides from slideshare).  There’s been a battle in the industry between these techniques, and there are pros and cons to each, but neither technique is best for everything.  Here’s a high level summary of the pros and cons from my talk:

rules-based vs ML-Based learning

Which is better? Neither, and both.  If you are just getting started, it’s simpler to use rules.  If you are looking for the most extensive coverage in a wide domain, you’ll want to use some machine learning.  And if you want the best possible results, it turns out that a hybrid combination of both is the winner.

This reminds me of a similar ”battle” 15 years ago between statistical and linguistic approaches to search.  In the end, the best answer was to use both, and all modern search engines combine statistics and linguistics internally.
I believe we’ve been able to make a single product in the AutoClassifier that allows our customers unprecedented power without losing the simplicity that we are often complimented for.  Our customers have told us that we made it simple to combine techniques, and easy to understand.

Other Notable Updates in AutoClassifier V3

In addition to the change highlighted above, there are many new features in AutoClassifier V3:

  • Scripting:  supports VBscript or C# plugins to process the output of rules procedurally, so that you can control any aspect of the processing you want.
  • Scoring and Thresholding:  is now visible in the testbed, extensible by scripting, and extended to cover new scenarios.
  • Tag History and Auditing Reports: have been added to enhance taxonomy management.
  • SharePoint Document Sets: can now be classified as a single entity

We’ll continue to invest in the AutoClassifier, and now that we have an ability to plug in different modules we can do some amazing things.  I expect that our customers will also come up with ideas and innovative solutions as well.  Adding machine learning under the hood is exciting, and a hybrid combination of techniques provides the best of both worlds.

Here Are Some Frequently Asked Questions About AutoClassifier:

Have you added support for SharePoint 2016?

Yes, AutoClassifier V3 supports O365, SharePoint 2013 and SharePoint 2016.  Hybrid SharePoint configurations are also supported.

What kind of content can I process with the AutoClassifier? 

If you can crawl it, you can process it.  Combining the AutoClassifier with our connectors provides processing of content from over 60 different systems.

Can I upgrade from previous versions of the AutoClassifier?

Yes.  Follow the online documentation in the customer portal to upgrade, and contact customer support with any questions.

Do I need a taxonomy?  Where can I get one?

Even if you use machine learning, regular expression matching, or other modules that aren’t taxonomy based, we recommend starting with one or more core taxonomies and rule sets and then add in the other techniques.

  • In most cases, you’ll find you already have a taxonomy in house.
  • You can use a simple list as a taxonomy – for example, your product names and/or project names.
  • You can often download free taxonomies (taxonomy warehouse and BARTOC.org are two good sources).
  • Commercial taxonomies are also available from our partner WAND.

Any of these should be taken as a starting point – expect to tailor and maintain these over time to adapt them to your specific needs. We recommend using multiple simpler taxonomies rather than one giant one. You can often work with a simple “enterprise-wide” taxonomy that’s applied to all content and then apply different department-specific taxonomies to different sets of content.

This entry was posted in Blog and tagged AutoClassification, Metadata, SharePoint.
Next-Generation Analytics
Going Mobile – Introducing our New Release of Visual Refiners
Author Profiles | Blog Home
Tags
  • Artificial Intelligence
  • AutoClassification
  • AzureBA Insight
  • Cloud SSA
  • Cognitive Search
  • Connectors
  • Content
  • Customer Portals
  • DelveDocumentum
  • Dynamics
  • Elasticsearch
  • Enterprise Search
  • European SharePoint Conference
  • Expertise Locator
  • Federated Search
  • Google Search Appliance
  • Hybrid SharePoint
  • InfoApps
  • Information Strategy
  • InfoSites
  • Intelligent Search
  • Knowledge Integration Platform
  •  Knowledge Management
  • Legal
  • Machine Learning
  • Metadata
  • Office 365
  • Office Graph
  • Personalization
  •  Portals and Intranets
  •  SearchFirstMigration
  •  Security
  • SEO
  • SharePoint
  • SharePoint 2010
  • SharePoint 2013
  • SharePoint 2016
  • SharePoint Search
  •  Smart Analytics
  • Taxonomies
  • Unified Information Access
  • Visual Refiners
  •  Webinar

BA Insight logo 2022AI-driven intelligent enterprise search software

Askable knowledge

Reward questions with results.

BA Insight Headquarters
7 Liberty Square, Suite 3
Boston, MA 02109-5812, USA
+1.339.368.7234
sales@BAinsight.com

BA Insight UK Office
London, St James
4th Floor, Rex House
4 – 12 Regent Street
London, SW1Y 4PE

BA Insight Romania Office
C.A Rosetti Street No 17
Office 009ResCo-work01
District 2 Bucharest 020011

Find out how internet-like search can be implemented inside your organization

  • Enterprise Search
  • Search for Legal
  • Search for Life Sciences
  • Elasticsearch
  • SharePoint Search
  • Search Orchestration
  • Enterprise System Connectors
  • Autoclassification
  • Expert Locator
  • Search for Dynamics
  • Search for Salesforce
  • Podcasts
  • Webinars
  • Videos
  • White Papers
  • Product Resources
  • Privacy/Cookie Policy
BA Insight © 2023
  • Products
    • BA Insight for Amazon Kendra
    • BA Insight for Amazon OpenSearch Service
    • BA Insight for Azure Cognitive Search
    • BA Insight for Elasticsearch
    • BA Insight for Microsoft 365
    • BA Insight for Microsoft Teams
    • BA Insight for NetDocuments
    • BA Insight for ServiceNow
    • BA Insight for Solr
    • WorkHub Search
  • Technology
    • SmartHub
    • Connectors
    • AutoClassifier
    • AppBus
  • Solutions
    • Enterprise Search
    • Customer Portal Search
    • Website Search
    • Search for Legal
    • Search for Life Sciences
    • Clinical Trials
  • Resources
    • Podcasts
    • Videos
    • Case Studies
    • White Papers
    • Product Resources
    • Newsletters
    • Articles
    • Webinars
    • Blog
    • News
  • About Us
    • Company Overview
    • Partners
    • Contact Us

Perspectives from our CEO,
Massood Zarrabian

Organizations have long been struggling with how to make knowledge assets available to employees, partners, and customers. Although there have been major technological advances in how this information is captured and made available over the past two decades, these have mostly been around a single business process. For example, when I joined Servicesoft, we were pioneering the idea of using search and classification engines to help transfer information to customers as part of the emerging eService market. At OutStart, we initiated the idea of objectifying learning and making the development of learning component-based so that it could be consumed via a “just-in-time” model. This meant that learners interacted only with learning objects that helped them increase their knowledge, and therefore their value, to the organization. In the last decade, another silo of information capture has emerged in the form of Social Business Software, making it easier than ever to capture nuggets of knowledge that can help others.  The interesting dynamic in all of this is that there are large investments being made to capture information and knowledge assets, but very little of it is actually accessible by employees or customers.

In the ‘90s, multiple software vendors tried to address the issue of information access by providing software to implement portals, whether they were used for helpdesk integration, customer support, intranets, or R&D to help with the collaboration and reuse of IP.  As the technology evolved and search engines and appliances became available, the portals were replaced with implementations of enterprise search. The problem with this approach is that it views the search engine as a ‘one size fits all’ solution, as opposed to viewing it as an enabling technology that helps organizations address a business issue. Failed enterprise search projects became the norm as companies tried, and continue trying, to resolve their information access challenges by implementing new search appliances while they still have underlying issues around integration with other systems, classification and tagging, and a sub-par user experience.

I joined BA Insight because I saw an immense opportunity to transform the way enterprise search is being implemented.  I am sure you know that the amount of content that is being generated is growing exponentially across an increasing number of sources, and the inability to find the information assets our people need is costing us billions of dollars in lost productivity while leading to low morale and customer dissatisfaction.

When I evaluated the opportunity at BA Insight, I found the company to be uniquely positioned to provide a new approach to the unification of information that stops the pattern of enterprise search failures.  We do this by transforming SharePoint, which has become ubiquitous across enterprises, into a unified information access platform that enables fast implementation of search-driven applications at a fraction of the cost of other options while de-risking search projects.

There are many notable and successful search-driven applications available on the Consumer Internet. Many are ‘killer apps’, and as a class they have fundamentally changed the way people interact with information.  However, in comparison, corporate Intranets, customer support portals, helpdesk applications or knowledge management solutions don’t come close to being killer apps, nor do they provide a remarkable user experience.

I found it intriguing that BA Insight has the technology, people, and partners to help catapult enterprise search to the next level.  We replace a people-intensive, SI-oriented approach to implementation with a technology-based approach that is lower cost, lower risk, easier to implement, and easier to upgrade. We do the heavy lifting to make our products work with different versions of SharePoint as well as various versions of other software applications that exist within a customer’s infrastructure.  We automate how content is tagged to improve findability and also provide out of the box capabilities to improve the user experience with how information is found and accessed.

I believe that in order for a company to be wildly successful, it must demonstrate the following five important attributes:

First is customer-centricity. Customers put their trust in startups with a vision, and we must partner with them to make sure they succeed. I am so proud of all of the customers I have had the honor of serving, as well as their achievements, and I consider it the cornerstone of my past success and BA Insight’s future success.

Second is our team. Our people have the experience and the desire to help our customers implement and deploy incredibly powerful applications. We want to help our customers build killer apps, as opposed to search portals. This is a team of high energy, committed, customer-centric experts who have embraced the idea that search-driven applications could be a lot better and are working to change how they get implemented.  I am fortunate enough to have become part of the BA Insight team and am so proud of everyone who works here. In a short period of time, we have made incredible progress on many fronts and I feel strongly that everyone’s loyalty and belief in the company will continue to provide an incredible depth for us and help our customers transform how they work with search technology.

Third is the technology. We have a broad set of capabilities that extend the value of existing SharePoint investments, so we eliminate the need to change the underlying infrastructure, saving a lot of time and money.  We have over 90 out of the box connectors, a world class auto-tagging engine for content within and outside of SharePoint, visual refiners to let users drill down and find information quickly, an active workspace with content assembly, and document preview capabilities.  We also provide a lot of flexibility in how our technology can be implemented.  Our full platform, for example, is particularly well-suited for new projects.  On the other hand, if search-driven applications have already been implemented within an organization, then components of our platform can be leveraged to augment the existing technology to improve results.

Fourth is to be financially responsible. Company success is about investment and return on that investment, and it needs to be measured in two fronts:

    1. ROI, which is the pure quantitative and financial analysis, often measured as productivity gains.
    2. My preferred approach is return on value (ROV), which is qualitative and focused on items such as customer loyalty, employee morale, better decision-making, and the ability to find information faster to increase customer responsiveness. Many of these things impact productivity and are therefore ROI, but they aren’t often measured and are therefore difficult to quantify. Isn’t the impact of finding the right knowledge to be able to do one’s job priceless? If we are smart about how we invest our time, money, and energy based on return on value, then we will naturally bring tremendous value to our customers and market and end up as a growing, profitable company that is resilient through the ups and downs of the economy and market.

And fifth is promoting a strong work/life balance. Work is important, but we cannot forget our families and shouldn’t compromise being with them due to office pressures.  For a long period of time I was a workaholic, but after my first son was born I committed to change. Over the years I have done my best to practice what I preach, successfully balancing work and family life, and I encourage our employees to do the same. I enjoy nothing more than spending time with my incredible wife and our great boys.

These attributes are prevalent at BA Insight, and I am very enthusiastic about the opportunities that lie ahead.  I am extremely confident that we are well positioned to enable the future of unified information access as we help visionary organizations around the world realize the value of the collective intelligence that is being captured every day within the massive volumes of content they produce.

Join the BA Insight Mailing List