Monthly Archives: May 2010

“Enterprise Search” and “Federated Search” – what’s the difference?

Posted on by
1

Federated Search or Enterprise Search

In this post I look at the difference between Enterprise Search and Federated Search …

Enterprise Search

In Enterprise Search, content is indexed locally. That is, the content is made available to the search engine, and the metadata, and if possible, the content itself, is indexed and stored in a database.

Federated Search

Federated Search, utilises the search results that are provided by an “external” system. (“External” refers to another document management system, or search application.)

Different Indexing Methods

Jeroen Mom (author of the blog that inspired this post) points out a few interesting things about the different indexing methods. Because, in Enterprise Search, the documents are available by the indexer, all the metadata and content can be crawled and indexed. With a Federated Search, the results are what was returned from the external system.  This relies on the external system’s indexing capabilities.

On the other hand, to access, and index documents locally (in an Enterprise Search) usually requires loading the document to a locally accessible location. To do this can chew up a lot of bandwidth. Then there is the corresponding CPU utilization to do the actual indexing. And to keep the index up-to-date, further crawls/indexing need to be performed regularly. If the documents are relatively “close” (within the same subnet) then this should not be too much of a problem.

However, when the documents are located in, or indexed by, remote (outside the firewall) systems, a federated search has certain benefits. All that is being transferred is the query and the results that match that query. The indexing process is being handled by the remote system. And this includes the regular indexing required to keep the index up-to-date.

Should you use Federated Search, or Enterprise Search

So the decision whether to use Federated Search or Enterprise Search, depends on:

  • the resources available (how much grunt does your indexing server have?),
  • the availability of the content that needs to be indexed (can the indexer access the content directly)
  • the bandwidth available
  • the amount of confidence that the search results returned contain 100% of the content that meets the search criteria. (Are you happy with the results that the remote system provides).

Federated search versus Enterprise search, can the one go without the other?

Would you like to learn more?
Here are some resources that I handpicked fom Amazon…

(Note – these are affiliate links.)

Related articles

Getting access to all the information

Posted on by
Reply

Don't lose access to all your information

In 2010, NZ CIO magazine published an article.

The article described a problem where 74% of the local public sector organisations admitted to holding some digital information they can no longer access”. This is incredible!

This means that out of every 100 pieces of digital information that they are holding, they can only use 26 of them!

Why? Because they do not have a defined method for retaining and storing their digital information.

out of every 100 pieces of digital information that they are holding, they can only use 26 of them!

Archives New Zealand (a government organisation that is the “official guardian of New Zealand’s public archives”) have been tasked with creating a digital continuity strategy.

This will ensure that valuable information is preserved and migrated when necessary to the latest formats and media, appropriate metadata is attached, and documents that are no longer relevant are securely deleted.

Wow – that is precisely the way that they can improve the “findability” of digital information. Metadata that describes the information (whether it be film, document, etc) is essential.‘

Everyone speaking the same language

Further to that, there are a couple of other very important things to take into consideration.

One is the “vocabulary“. Do all departments/organisations speak the same language? All the various departments involved need to ensure that they use a common vocabulary when describing something. Otherwise, there will still be a very large number of documents that are “unable to be accessed”.

Another thing that is very important is “buy in“. With many different departments, it is important that they all agree to the above-mentioned strategies. Having only 2 out of every 10 organisations following the new strategy means that there is still a lot of information that can’t be found.

Tackling the Problem

Fortunately, Archives NZ has addressed these extra points of consideration.

They have drawn up a Digital Continuity Action Plan. The focus of the plan in the first year is to raise awareness and understanding of the problem.

Excellent! Communication and awareness are critical for success.

Part of this communication process will involve “refining the language used”. This will be done by looking at which words are used by CIOs, technology managers, record keepers and so on.

I am very keen to see how this initiative transpires. I’ll be watching with interest, and if I see anything of note, I will let you know.

Related articles
  • Digital Continuation StrategyArchives NZ – Digital Preservation Strategy
  • Strategy reviewAnalysis of Current Digital Preservation Policies: Archives, Libraries and Museums

Keen to read more?

Where in the World have I worked?

Posted on by
Reply

world-sepia

I’ve worked in three different countries, in both the Northern, and Southern Hemispheres. I have worked in quite a few different places, with many different customers.

Below you can see a Google map in which I have pinpointed the companies that I have worked for, and the customers that I have worked with. Feel free to zoom in, and have a look around.

 

 
 

Alternatively, click on this link to my “Working Map” – something I created for my LinkedIn profile. This takes you directly to the map, and is somewhat easier to read.

My Working Map

My Linkedin profile