Predicting the Future

It’s that time again. It’s become a bit of a tradition to lay out predictions for the coming year at the beginning of January.  For me, it’s a relatively recent tradition I started in 2015.  My predictions focus around search, digital workplaces, and market dynamics involving Microsoft.  This year I started by recording a podcast, where I was interviewed about my 2017 predictions (you can listen to it here).

Some interesting points came out in the podcast that spurred me to write a pair of blog articles. This article looks at how my past predictions turned out; the next one contains my predictions for 2017.

How’s My Crystal Ball?

The first question in the podcast was about my track record – did my predictions for 2015 and 2016 come to pass? Prediction is definitely an art, not a science. Obviously, none of us actually knows the future.  But looking back at my predictions for 2015 and 2016 is informative.

Predictions for 2015

Here are my predictions from 2 years ago and how they turned out. I have the benefit now of also seeing what happened in 2016.

Jeff’s 2015 Predictions Did my prediction come to pass in 2015? What happened in 2016?
Enterprise Software market will grow, rather than crash Yes The market kept growing smoothly
Resurgence of Portals Yes The resurgence continued, and added Intranet resurgence too
SharePoint deconstruction Yes This trend sort of reversed in 2016
Hybrid SharePoint becomes the norm Slower than I thought Happened as I predicted (just later)
App model will keep morphing Yes The App model kept on morphing
Office Graph will cross the “Trough of Disillusionment” No I started seeing some evidence of this
More & different search engines Slower than I thought Happened as I predicted (just later)
Digital Assistants will cooperate No (I was wrong on this and dropped it)
Cognitive Computing will move from buzzword to implementation slower than I thought Happened as I predicted (just later)
Beautiful designs catch on in the enterprise Yes UX focus is here to stay
We can calculate an “accuracy score” from this:
  • 5 happened within 2015
  • 3 showed movement in 2015 but spilled into 2016
  • 2 did not happen in 2015 at all

Not a great start – thought it would be 80% if I were to give myself credit for things that were visibly trending up but just going a bit slow.

What I Got Right in 2015

In January 2015 the conventional wisdom was that the enterprise software market would crash and burn, but I didn’t think it would – and the year proved me right (in fact, it grew by 20%).  Similarly, the portal market had a bit of a resurgence – both in vendor revenues and in the number of projects that identified themselves as “portal” projects – a term that had gone out of vogue and then came back.

The “deconstruction” of SharePoint was evident in Office 365, where BI was removed, user profiles became O365-wide rather than SharePoint-focused, Yammer superseded SharePoint social, and several other services moved out of SharePoint or were replaced by Office 365 Services.  In fact, by October there were many that predicted that SharePoint would be gone as a brand – subsumed into Office365 – and that it was essentially dead.  I never believed this and in fact predicted the pendulum would swing the other way in 2016 (which it did with a vengeance).

Hybrid SharePoint was starting to be adopted, and I predicted it would be “the norm” by the end of 2015, but I got the timing on this a bit wrong. The number of projects grew in 2015 but several things were delayed (such as the release of cloud hybrid search).  It wasn’t until 2016 that I stopped getting strange looks and skeptical questions when proposing hybrid SharePoint, and started seeing almost every large organization consider hybrid as option (either as a temporary path to an all-cloud future, or as a semi-permanent state).  Today there is no doubt that Hybrid SharePoint is mainstream – support for hybrid with SharePoint 2016 is quite robust and the number of projects involving hybrid is large.  At BA Insight we see a lot of hybrid search projects, and the scale of these has gotten pretty large.

The app model was definitely morphing in 2015 – Microsoft pulled auto-hosted apps, renamed Office apps and SharePoint apps to “add-ins”, and came out with the first suite-level APIs in Office 365 (with a different authentication model).  So this prediction was accurate.   In 2016 the changes didn’t stop – the SharePoint Framework and client-side web parts were introduced, sandboxed solutions were discontinued, and the Microsoft Graph unified API came out.   By the end of year, the Office Graph term had been subsumed into the Microsoft Graph (without any actual technical changes), just to confuse everybody more.  I understand that we are in a dynamic, ever-changing world, but SharePoint developers are getting whiplash.  I meet a lot of these developers that are weary of Microsoft changing the model every year or two, and find that the APIs are incomplete and buggy at first – they are therefore wary of the next new API or app model.

The One I Got Wrong

At the end of 2015, most of the predictions I’d made had either happened, were clearly trending in the direction I predicted, or seemed like they still were going to happen.  There was one exception: cooperating digital assistants. I thought that we’d see signs of cooperation amongst our digital agents (such as Siri, Cortana, Google Now, Echo etc).  To me, it just seems natural that if I am using Cortana and you are using Siri, and I want to schedule a meeting with you, our respective agents should be able to talk to each other in that process.  I’ve worked on autonomous agent software and I know that cooperating agents are both possible and powerful, so I predicted that this would happen.  But by the end of 2015 it was clear to me that this was just not going to happen – no sign of standards, cooperating agents, or even friendliness between the vendors in this space.  Perhaps there is too much competition between Microsoft, Google, Apple, and Amazon for cooperation to unfold.  I dropped this prediction.

Predictions for 2016

For 2016 I decided to carry over predictions that hadn’t come true in 2015 – if I still believed they would happen. There were 4 of these (shown in italics below), leaving room for only 6 new ones:

Jeff’s 2016 Predictions
Did my prediction come to pass in 2016?
Hybrid becomes the norm Yes
More & different search engines Yes
Cognitive Computing will move from buzzword to implementation Yes
Office Graph will cross the “trough of Disillusionment” Still slower than I thought
SharePoint will be “back” Yes
Machine Learning will be everywhere Yes
API Mashups will highlight Search No
Resurgence of Intranets Yes
IoT (Internet of Things) will become mainstream Yes
Azure will gain ground on Amazon Yes
Let’s calculate my accuracy score for 2016:
  • 8 happened within 2016 (3 of these were carryovers)
  • 1 is still slower than I expected
  • 1 did not happen in 2016

Not too shabby…..

What I Got Right in 2016

I predicted that 2016 would be a year of “more and different search engines.”  Search is an unsolved problem and there are always waves of innovation, but enterprise search has been relatively stable for years now.  Search engines commoditized at the beginning of the decade – in the sense that the features most people needed were available in every search engine, and prices dropped dramatically.  A wave of acquisitions of search vendors by megaplayers made the focus be on digestion and integration of these search engines into platforms.  But there is still plenty of room for innovation in enterprise search.  For example, companies like Splunk and Palantir have done very well over the last few years (in part by not calling themselves search companies).

In 2016 I counted 12 new enterprise search startups and saw a bigger range of approach.  Google’s Springboard was unveiled, several question-answering systems such as Inbenta appeared; structured-data search (BI style) continued to gained traction with Elasticsearch and Kibana but showed up in new companies like Thoughtspot, and Discovery was a big theme, ranging from progress in Microsoft Delve to new companies like Brainspace.

I don’t need to say much about Machine Learning; you hear about it everywhere now and it is both highly hyped and genuinely progressing quickly. The use of machine learning in search and findability in particular isn’t new and it is still in its infancy in many ways – but it became a very visible marketing theme in 2016, with varying degrees of reality behind the marketing.

Cognitive Computing is a term you may or may not have heard a lot.  I serve on the Cognitive Computing Consortium and saw a number of real and exciting production case studies this past year, so I am confident in my declaration that this has passed beyond the pure buzzword phase.

Am I Cheating?

There are several ways you might cry “foul” on my accounting here:

  • Calling the clear trend.  “Resurgence of portals”? “SharePoint will be back”?  “IoT goes mainstream”? Aren’t these so obvious that they’re not really predictions, just observations?
  • Double-counting. By carrying over predictions I’m counting those predictions TWICE.  And if things were definitely moving in 2015 I’m picking easy ones that have a much higher chance of coming true.
  • Too much wiggle room. I haven’t really defined what these predictions mean in a measureable way, and I’m making the subjective judgement of whether they happened or not myself.

OK, maybe you got me – on at least one point.  Let me plead my case, though:

  • On “calling the clear trend”: these things might seem obvious in hindsight, but at the beginning of 2016 nobody knew that the May 4th “Future of SharePoint” announcements and brand-rebuilding initiatives were on the horizon (except perhaps a few folks at Microsoft).  There were plenty of naysayers about Cognitive Computing and Machine Learning at the start of the year. And it was not at all obvious at the start of the year that the number of intranet-refresh projects would more than double in 2016 over 2015.
  • On the double-counting: I am dinging myself if something doesn’t happen, so this is not biased in my favor.  Carrying over predictions just means that I’m sticking to my guns.  Carry-overs aren’t lay-ups, a prediction can count against you multiple times (I lost 10% in both years on something that I will carry over to 2017), where it can only count for you once.
  • On wiggle room: it does bug me that these predictions seem pretty loose.  I can only plead that I’m pretty new at this, and observe that it seems like all prognosticators do this.  A friend pointed out that I should just have taken better advantage of this wiggle room; if I had counted a prediction as accurate when there were strong signs of movement in the right direction, I could have scored 80% accurate in 2015 and 90% accurate in 2016.  But I don’t want to do this; I will instead endeavor to make predictions for 2017 that include clear measurements.

At least I’m TRYING to be accountable, and to predict things that aren’t obvious.   Let me know what you think about my 2017 predictions.