The Time Has Come for Enterprise Search

Maybe it’s time to challenge the 20 year-old paradigm of making everyone a knowledge worker. For a long time the BI community has assumed that if we give business users the right data and tools, they’ll have the necessary ammunition to do their jobs. But I’m beginning to believe that may no longer be a practical approach. At least not for everyone.

One thing that’s changed in the last dozen-or-so years is that individuals’ job responsibilities have become more complex. The breadth of these responsibilities has grown. I question whether the average business user can really keep track of all the subject area content, all the table definitions, column names, data types, definitions of columns, and locations of all the values across the 6000+ tables in the data mart.

And that’s just the data mart. I’m not even including the applications and systems the average business user interacts with on a daily basis. Not to mention all those presentations, documents, videos, and archived e-mails from customers.

I’m not arguing the value of analytics, nor am I challenging the value of the data warehouse. But is it really practical to expect everyone to generate their own reports? Look at the U.S. tax code. It’s certainly broader than a single CPA can keep track of. Now consider most companies’ Finance departments. There’s more data coming out of Finance than most people can deal with. Otherwise all those specialized applications and dedicated data analysts wouldn’t exist in the first place!

Maybe it’s not about delivering BI tools to every end-user. Maybe it’s about delivering reports in a manner that can be consumed. We’ve gotten so wound-up about detailed data that we haven’t stopped to wonder whether it’s worthwhile to push all that detail to the end-user’s desktop—and then expect him or her to learn all the rules.

One of my brokerage accounts contains 5 different equities. I don’t look at them every day. I don’t look at intra-day price changes. I really don’t need to know. All I really want to know is when I do look at the information, has the stock’s value gone up or down? And how do I get the information? I didn’t build a custom report. I didn’t do drill-down, or drill-across. I went to the web and searched on the stock price.

Maybe instead of buying of a copy of a [name the BI vendor software] tool, we simply build a set of standard reports for key business areas (Sales, Marketing, Finance), and publish them. You can publish these reports to a drive, to a server, to a website, to a portal—it shouldn’t matter. People should find the information with a browser. Reports can be stored and indexed and accessed via an enterprise search engine. Of course, as with everything else, you still need to define terms and metadata so that people understand what they’re reading.

Whenever people talk about enterprise search functionality they’re usually obsessing about unstructured data. But enterprise search can deliver enormous value for structured data. IT departments could be leading the charge if the definition of success weren’t large infrastructure and technology implementation projects and instead data delivery and usage.

The executive doesn’t ask, “What tool did you use to solve this problem?” Instead, she wants to know if the problem has in fact been solved.

Comments

[Kevin Davis]

You're all over it Evan. Some call it Unified Information Access (you know who you are)

The delivery should be on the device the user has at the time they want the answer coupled with the security they are allowed. If there are warning signals after the initial search - assuming structured data KPIs or something like that, then the BI tool can be used to figure out the details. Else why run the BI report if all signs are ok?

[Phil Simon]

Totally agreed, Evan. I had Damien Santer writer a chapter in The Next Wave of Technologies because of its importance. Silly to me how we can find a single blog post via Google in .2 seconds but where's that bloody spreadsheet?

[Kelly Lautt]

I think the most important part of the entry is this:

“Maybe it’s not about delivering BI tools to every end-user. Maybe it’s about delivering reports in a manner that can be consumed. We’ve gotten so wound-up about detailed data that we haven’t stopped to wonder whether it’s worthwhile to push all that detail to the end-user’s desktop—and then expect him or her to learn all the rules.”

I think the context should determine whether the end user gets the critical and targeted data Evan is talking about via email, well information architected portals, search (as Evan emphasizes), properly connected dashboards (that connect to the appropriate reports), or even some alerts that can take the user to a more detailed report (answering part of Kevin's comment).

But I completely agree about the dangers inherent in the current trend to provide more flexible, self-serve, tool-based access to every possible piece of data of potential interest. The assumption seems to be that if that type of access is good for some users, it must be good for all users now that the tools may be less of a barrier. But I think that search should just be one tool of many for sharing, driving and providing the consumable reports.

There’s my quick two cents worth on a rainy Vancouver evening!