5 reasons why Enterprise Search will never be as good as Google

All the time we hear managers saying “we want a search engine as good as Google”. Here are 5 reasons why you can never even get close.

Image from wikimedia commons

 Google is the yardstick for search, and managers seem to want internal enterprise search that works as well and as (apparently) intuitively as google. But there are 5 good reasons why this will never happen (bearing in mind that I am by no means a search specialist).


1) Search engine optimisation – webpages want to be found

Do you have a website? If you do you will be as familiar as I with the deluge of Spam emails offering to optimise my website for Google search. SEO (Search Engine Optimisation) is big business, and the owners of webpages are doing lots of work on Google’s behalf to ensure their pages are indexable and findable and optimised for search.

But who, in an organisation, optimises their documents and sites for internal search? Let me tell you who – Nobody; that’s who.  Unless you are very lucky, few if any people think about the issues of findability when they publish content.

Google is successful in finding sites because those sites want to be found. They are often very keen to be found, because they are trying to sell you something. The search results at the top of Google’s list are often the ones most desperate to be found. Many documents in your enterprise system do not want to be found, often for issues related to confidentiality as described below.

2) The fact that the web is interlinked html pages, whereas your content is usually isolated word documents (if you’re lucky!)

Sometimes it’s not even Word documents – I know organisations that save their critical knowledge in pdf form!

The difference between interlinked web pages and isalted documents is critical. Google can crawl through the web of interlinked sites, can understand the context of a site partly through its links, and can identify authoritative or important sites based on the number of links that point to them. The search engine results at the top of the list are often the ones with the most backlinks.  The components of the page are also obvious to Google – the title, the first level headings, the metadata – and these also are used to understand what the page is about.

Your documents are not linked. Each stands alone. Each has to be searched and indexed separately. There are no backlinks. There is no visible structure to the document, other than to the human eye, and the search engine cannot tell a footnote from a level 1 heading.

3) The hordes of search engine specialists employed by Google.

How many search engine specialists do you employ? None, right? Google employs tens of thousands. That’s one of the reasons their search works better than yours.

This is especially an issue if you are planning to use Semantic search, or to optimise customer search of your knowledge base. In these cases you will need a search engine specialist to build and evolve the ontology, track and improve the search accuracy, and define the synonyms and stop words.  However managers often neglect this aspect, and assume a search-engine is a one-off purchase that will run itself.

4) Google doesn’t do “security levels”

Google assumes everything is available and visible to everyone. It doesn’t do passwords or access restrictions or security levels. It searches everything that is not on the Dark Web.

A lot of your documents are effectively on the dark Web – they are in secure folders on Box, or Dropbox, or SharePoint. I consulted recently to an organisation that had 300 separate databases or document management systems. They had opened about 6 of these for indexing, the rest were effectively “dark” as far as search was concerned.

5) The web doesn’t do version control

Every webpage on the web is the only version. Rather than storing a webpage as version 3.5 and writing version 4.0, you just rewrite and publish the page. Every page on the web is the current version, and is constantly under development. Google only returns one version of the page – the current version.

You don’t treat documents in this way. Very often, unless your document management is very good, you will have multiple versions of the same document stored in different places.  One of the bugbears of enterprise search is that it will often find all these version in your search results.

So the next time your managers ask “Why can’t we have search like Google” – 

you can reply – “Yes, we can, IF

  • You move all content out of documents onto wikis
  • You keep only one version of every document
  • You train all staff in search engine optimisation
  • You hire a team of search engine specialists, and
  • You make all documents open to everyone”.
Then see what they say!

Enterprise search can work, but it will never work like Google.

View Original Source (nickmilton.com) Here.

Search or Browse? Which is best?

Should you optimise your knowledge base for search or for browse? Answer to an online question.

I received the comment below on my blog post on “Knowledge documents vs project documents“.

“I found this a helpful blog. I am working on the implementation of a matter management system within a large law department. One of the things I’ve been considering is the appropriateness of organizing documents within matters using a folder structure versus metadata and tags. 

“I have a hypothesis that it makes sense to use a folder structure to organize “project” or “matter” documents – because the folder structure operates as a story map, and helps people understand what is involved with the specific project/matter. Browsing for project documents seems intuitive. But it makes sense to use metadata to classify “knowledge” documents because users are likely going to want to find knowledge documents through “search” as opposed to browsing. 

“I’m interested in hearing reactions to that hypothesis.”

Firstly I am pleased the blog post was helpful! Then I have two answers for you below.

This is the dilemma of optimising for search or for browse when it comes to maximising findability of knowledge and information. We assume “users are likely going to want to find knowledge documents through search” – but is that true?

  • This study found 14% of users start with search
  • This found less than 5%
  • This found that 59% of web visitors frequently use the internal search engine to navigate on a website and 15% would rather use the search function than the hierarchical menu. 

Some conclusions from these studies are as follows:

So the first part of the answer is this

You have to optimise for browsers. 

From the results above, browsers are the majority. Also search has its limitation: a lack of discovery. A user relies on search to find specific information he or she already knows or suspects to exist. Rarely does a user search for something he or she doesn’t even know to search for. browsers see more and buy more – often things they did not know they wanted.

I like the idea of structuring your folders like a story map, or you could use the model of the knowledge supermarket (see my post on How a Knowledge Supermarket helps the knowledge customer to find what they need). Supermarkets are predicated on findability, and one of the reasons they put the bread and the wine at the back of the store is that it forces you to see more of the store, to browse, and ultimately to buy more.

Also, try to introduce a standard folder structure, so your lawyers can move from one matter to another and still find their files stored int eh same way.

However the second part of the answer is this

You have to optimise for searchers as well.

This means getting a good tagging system and a good metadata structure. Good metadata and a good folder structure are not mutually exclusive – you need both. Some people will browse and find more that they came for; some will search, and need to find exactly what they asked for.

Like so much in KM, this is not an “either-or”, this is a “both-and”.

Cater for the browsers and for the searchers, and you might just please all of the people all of the time.  

View Original Source (nickmilton.com) Here.

Feedback loops in the Knowledge cycle

Last week I described a “Pull cycle” for knowledge – let’s now look at the feedback loops in that cycle.

You can find description of the cycle here. This is a cycle based on knowledge demand (unlike the supply-side cycles you normally see) and includes the following steps;

  • The cycle starts with a problem, and the identification of the need for knowledge to solve the problem (the “need to know”)
  • The first step is to seek for that knowledge – to search online, and to ask others
  • Seeking/asking is followed by finding
  • However generally we tend to “over-find”. Unless we are lucky, or there is a very good KM system, we find more than we need, so the next step is to review the results and select those which seem most relevant in the context of the problem.
  • This found knowledge then needs to be integrated into what is already known about the problem, and integrated into solutions, approaches, procedures and plans.
  • Finally the integrated knowledge needs to be applied to the problem.
In the picture above, I have added the monitoring and feedback loops to each step, which work like this:
  • After the behaviour of asking/seeking, you feedback firstly whether there was any asking/seeking (so the organisation can track the behaviours of knowledge seeking) and you monitor and feed back the topics that people are asking about or searching for. You can for example analyse questions in a community of practice, or queries to a helpdesk, and you can analyse search terms from the corporate search engine logs. These will give you some ideas of the knowledge needs in the organisation, which knowledge supply has to match. 
  • After the finding step you feedback whether  you found the knowledge you needed, or not. This feedback will help identify gaps in the knowledge base where the knowledge needs are not being met, and can trigger the creation of new knowledge assets of articles.
  • After the review step you feed back whether the knowledge was relevant or not. In reality this feedback will be merged with the previous step. 
  • After the Integration step you feed back on the quality of the knowledge you found. This feedback will identify knowledge assets or knowledge articles which need to be updated.
  • After the Apply step, you feed back whether the knowledge was actually applied. This will help identify the most applicable and useful knowledge assets and articles. 
  • Finally, after the problem solution step, you feed back how much difference the knowledge made, and how much value was realised through problem solution. This allows you to track the value of the entire pull cycle and the entire knowledge management framework
Many of these monitoring and feedback loops are well developed in the customer-focused KM approaches such as KCS (Knowledge Centred Support), but any KM approach can apply these as part of their own KM metrics framework.
It is through the feedback associated with the steps that you can tell whether KM is actually working.

    View Original Source (nickmilton.com) Here.

    Is your knowledge base more like a sock drawer or a supermarket?

    There are three models for a knowledge base – which one is most like yours?

    before & afterYour online Knowledge base is where you store your documented knowledge, It is a repository – but more than that, it is a knowledge resource for others. Someone looking for documented knowledge comes to the knowledge base to search and browse.

    So what do they find?

    Generally knowledge bases fall into one of three categories. Let’s call them the underwear drawer, the library and the supermarket display.

    The underwear drawer (see top picture), if you are anything like me, is the place you pile all your clean washing, generally with the newest washing on the top. The drawer is easy to fill – you just cram everything in – but you know that the hard work will be done when you search (often early in the morning, in the half-light) for a set of matching underwear with no holes. All the work is done when searching, and very little is done when storing. The knowledge base equivalent is the uncontrolled filing structure, where you rely on a good search engine to find what you want. Dump it all in, then search for what you need.

    The library (or the organised underwear drawer, see bottom picture) is a managed and structured repository. You know the category, you know the title, and you find the book. Or you know the drawer, and the relevant section, and you find a rolled set of underwear of the right colour. The work is distributed between the seeker and the storer. You categorise when you store, and you browse to the right place when you search. The knowledge base equivalent is the organised and tagged knowledge base, where you can browse or search for the knowledge you know you need.

    The supermarket goes one step further (see my post from last year on the knowledge supermarket). In my local supermarket, for example, you can find a section that displays pasta, pasta sauce, Parmesan cheese and Italian wine, all within the same attractive display. Without searching, you are presented with all the ingredients for an Italian meal. Similarly with curries – curry sauces, poppadoms, nan bread, Cobra beer, lime pickle – all in one display. Lots of work is done by the storer, so as to minimise the work for the seeker, and as a result, they pick up the Impulse Buyer – the person who was not actually looking for this material in the first place, or who had forgotten that they need lime pickle with their poppadoms. The knowledge base equivalent is the Knowledge Asset; the one-stop shop for knowledge on a topic – the wiki page or portal that gives you everything you need to know, whether you knew you needed or not.

    So what’s the lesson for Knowledge Management?

    I believe there are three reasons why a supermarket is the best of the three models for your knowledge base.

    1. Firstly the main barrier for KM is not supply, but re-use. Many companies have no difficulty in creating knowledge supply, but all companies struggle with re-use. Therefore if we are to lower the barriers, let’s lower the barriers for the seeker and the re-user. Let’s invest in knowledge packaging, and the creation of knowledge assets, so that there is no excuse not to re-use.
    2. Secondly, although the search engine vendors will say that the search engine can do all the finding work for you, most people start by browsing rather than searching when they are shopping for something. Supermarkets are built for browsers, unorganised underwear drawers aren’t. 
    3. Thirdly, a search result will not return the “unknown unknowns” – the things you did not know to search for. The supermarket, on the other hand, is well designed for ensuring you find the impulse-buys which were not on your shopping list. 

    Think about the knowledge user when you design your knowledge base, and don’t make them or their search engine do all the work.

    View Original Source (nickmilton.com) Here.