Taming Off-line Content

In my posts on the long tail of enterprise content it appears I neglected to address the issue of content stored on the desktop, flash drives and other storage devices (off-line content) of employees. Although it is difficult to assess the value of these content, I believe that, for the most part, these content simply extend the long tail to the right of the curve. The challenge with this situation is still the same as for any content in the long tail: how does the organization ensure that content that meets its criteria for risk (or other value attribute) is placed under control of the designated systems. However, content stored on an employee's chosen device, outside of the control of the enterprise content management system, presents an additional challenge for discovery, retention and destruction. Locating this content when the time comes to take action, such as destruction, can be a problem. Think about a document that an employee as copied to their flash drive for "safe-keeping", if the organization is required to retain this document for 3 years, how does the organization ensure that all copies of the document including the one taken off-line is destroyed at the end of the retention period?. The organization can certainly define policies for document handling and also block port access to prevent external storage devices from been attached to company issued computers etc. It may also consider blocking e-mail with attachments and locking down computer hard disk so nothing can be stored on it (good luck with that one!).


Gaining Control: Self-Containing/Describing Content

So far organizations have managed to get a good handle on content sent as e-mail attachments (at least somewhat), however there does not appear to be an easy answer for off-line content without doing something so drastic that any productivity gain an organization had anticipated is largely eroded. The more I think about this issue the more I'm convinced that the silver bullet (if there was ever such a thing) is to have content that can tell you everything about itself. Content that can carry around with it more just its metadata (e.g. author, department, type, age etc) but also information such as: (1) its access control list; (2) the application that was used to create it (not just the mime type or document extension); (3) what business processes it participates in; (4) what organization owns it and (4) what organizations are allows to read it etc. These metadata must be persisted with the content and readable by any operating system and/or application software.

The implementation of this concept will require cooperation by leading vendors in both application and operating system software and standards organizations. Existing technologies such as XML, DRM contain aspects of what is required to implement this concept but portability of DRM solutions today remains an issue for various reasons. WinFS, a technology from Microsoft (MSFT) has some of the critical operating system support needed for this, it has however had its own share of problems. XML is really not self-describing especially because to consume a piece of XML you must first understand its structure and the relationship between the elements to get any meaning out of it.

In any case, while we wait around for the industry to provide the necessary infrastructure support for true self-describing content, organizations can either reward employees for not taking content off-line or punish employees who do so. I'm not advocating either approach to solving the problem, just a suggestion.

Is SharePoint a Disruptive Technology?

First, a brief history/overview/introduction, with the release of Microsoft Office SharePoint Server (MOSS) 2007, Microsoft (MSFT) delivered a product which provides some of the enterprise content management features already available through the leading Enterprise Content Management (ECM) vendors, but at a fraction of the price. SharePoint was first released in 2001 as SharePoint Portal Server and targeted at the Enterprise Portal and Collaborative software market. MOSS 2007 however, positions Microsoft to directly compete with the leading providers of ECM software.

AIIM, the Enterprise Content Management Industry Association, defines ECM as the technologies used to capture, manage, store, preserve, and deliver content and documents related to organizational processes. ECM tools and strategies allow for the management of an organization's unstructured information, wherever that information exists. An ECM software system typically has the following components: (1) Document Management, (2) Digital Asset Management, (3) Document Imaging, (4) Business Process Management/Workflow, (5) Record Management and (6) Collaboration.

The ECM software market is currently estimated at about $14b with revenues expect to reach $67b by the year 2010. The leading providers of software in the industry are FileNet (an IBM company) and EMC Corporation (EMC), with market shares of 22% and 13% respectively. Both became major ECM vendors through acquisitions. EMC, the leader in information storage, acquired Documentum, the leader in enterprise content management software, in October of 2003, in a move designed to strengthen its Information Lifecycle Management strategy, for $1.7b. The acquisition gave EMC access to technology which manages enterprise content that can be stored on EMC’s storage devices. In a similar move three years later, IBM acquired FileNet to advance its On Demand strategy. The acquisition of FileNet according to IBM will make it the clear industry leader in the growing Enterprise Content Management and Business Process Management. On July 26, 2007 MSFT reported that its Office SharePoint Server business unit generated almost $1b in revenue for the fiscal year. This announcement, which is an unprecedented move by Microsoft as it does not generally comment on revenues for individual products, is a clear indication that SharePoint has crossed the chasm to the mainstream of ECM software. It is no longer a technology for the innovator and early adopter types, but is now a stable mainstream ECM software product for the early majority to consider.

The Disruptive Innovation Theory by Clayton M. Christensen, points to situations in which new organizations can use relatively simple, convenient, low-cost innovations to create growth and triump over powerful incumbents. The theory holds that existing companies have a high probability of beating entrant attackers when the contest is about sustaining innovations. But establised companies almost always lose to attackers armed with disruptive innovations. SharePoint is a relatively simple, convenient, low-cost ECM platform that has brought 100million non-consumers into the ECM world. ECM purist will argue that SharePoint is not a "real" ECM platform/solution or that it is not an "Enterprise" Content Management solution. This is certainly true especially when SharePoint functionalities today are lined up against the ECM definition provided by AIIM. However this is the nature of a disruptive technology.

Enterprise Content LT: Getting a Handle on it

Given the challenges of the LT of enterprise content, organizations are probably tempted to manage the long tail by controlling its spread. This is a losing proposition at best, organizations need to put in place an content management strategy with the following components (1) Content repository Agnostic; (2) Make all content available available (3) Help Employees find the available content in the organization and (4) Multi-product solution.

Content repository agnostic solutions are harder to come by chiefly because each vendor tool stores data using proprietary formats. In addition these applications are tightly coupled to a chosen repository. One way to work around this issue is to invest in adaptors to help connect the different systems in use.

Making all content available and helping employees find the content can be achieved by deploying a search solution along with a taxonomy/folksonomy solution. These should allow access to the disparate data/content repositories in the organization from a single search user interface.

Employee content management needs may vary by various dimensions (functional grouping, department, tasks etc); therefore a short-list of enterprise content management tools to be used and supported in the organization should be provided for employees to choose from based on their needs. Adaptors may be needed to ensure that content can be accessed across these systems.

Enterprise Content Management & The LT Challenge

LT forces in enterprise content, (1) Democratization of Content Creation (2) Democratization of Content Distribution (3) Connecting Supply and Demand, gave rise to the vast amount of content and digital assets created in the course of doing business. In essence, these forces drove content creation towards the tail of the content-value curve.



The x-axis of the graph is the amount of content and the y-axis indicates the value of the content as defined by the organization, this value will vary across the organizations and between organizations, examples of value to consider include (1) Risk Factor; or (2) Regulation factor.

The Head of the curve is where you will find content that an organization considers high value, access to this content is typically restricted, a retention policy is applied to the content and it tends to participates in one or more business processes. Organizations have managed thise content by deploying advanced enterprise content management (ECM) systems such as IBM FileNet and EMC Documentum.

The Long Tail of the graph, which I believe contains the vast majority of content created in the enterprise, is the domain of several other applications most notably of which is SharePoint/MOSS 2007. This tail is the result of a combination of (1) the availability of content authoring tools especially Microsoft Office; (2) the lowered cost of distributing the content created and; (3) search technologies which allows all content stored to be indexed and searched. The deployment of these applications was largely limited to the departmental levels in organization and the central IT groups did not pay much attention. The deployments of these applications became so widespread that several niches developed and most of the valuable content that IT investments in Advanced ECM systems was deployed to cater to was no longer finding its ways to the advanced ECM systems.

The growth of the tail presents several challenges to the organization. For central IT groups, it is an infrastructure/support challenge, largely because the applications that enabled the tail where not central managed. Central IT is reigning in the uncontrolled deployments of these applications by incorporating them into the organization’s IT infrastructure and planning its deployment across the organization.

The LT is also a risk management and compliance nightmare for organizations. Organizations continue to struggle with what to do about the content in the tail as the value of content increases i.e. moving the content in the tail to the head. Companies would like to leverage the capabilities of their advanced ECM platforms and years of investment in developing business process, retention policies and compliance initiatives for these platforms. CIOs want to mitigate the risk and address compliance concerns while allowing employees to continue to collaborate in the knowledge creating activities which drives innovation and business results.

The challenge for organizations can be summarized as follows (1) moving content from the tail to the head as value increases (2) making all content readily available within the constraints of the access rights established for the particular content and regardless of the system in which the content is stored and (3) helping employees find content to promote sharing of best practices and reuse.

The Long Tail Forces in Enterprise Content

In my previous post I introduced the idea of The Long Tail (LT) of Enterprise Content, in this post I'll discuss the tree LT forces in enterprise content.

- Democratize Production
- Democratize Distribution
- Connect Supply and Demand

Democratization of Content Production
The democratization of content production in organizations started with the release of word processing software for the PC. As the use of PCs as a productivity tool increased with deployments across the enterprise, employees began to do their own word processing tasks. Further advancement in the PC hardware and software industry: (1) inclusion of hard disks in PCs (2) networked computers (3) Windows OS (4) Computer Server significantly lowered the cost of producing and storing content which led to more content that ever been created.

Democratization of Content Distribution
With the release of Portal/Collaboration software such as SharePoint Portal Server and Plumtree, employees began to collaborate in ways they were not able to in the past. Granted, employees had been able to share content through e-mail or by putting them on file servers, the content on these file servers were however not indexed therefore finding anything was a nightmare. Portal/Collaboration software however democratized the distribution of content, employees can now share and collaborate on content in ways they are not been able to do before. For example with SharePoint Portal Server (SPS) any employee can create a document library ( a repository for content), invite others to collaborate on the content and the content in the document library is available to anyone using a web browser.

Connecting Supply and Demand
Another effect of Portal/Collaboration software was that is connected previously un-findable content to the demand for the content through the enterprise search solution that was a common component of the portal software. For example with SPS, employees could now search for content stored on either file servers, SPS Document libraries, Exchange Servers, Lotus Notes and several other repositories by simply entering a search term in SPS search UI. The search component of portal software helped to connect the supply of content with the demand for it.

The Long Tail Of Enterprise Content

Organizations have applied the 80/20 rule in addressing the challenges posed by the vast amount of content and other digital assets created in the course of doing business. CIO's have generally focused their enterprise content management efforts on content considered to be highly valuable ("the hits") to the organization. A content's value was largely determined by one of the following factors; (1) an organizations need to mitigate risk; (2) expert opinions (3) regulations governing the organization's industry and (4) participation in business process automation applications.

The "modest sellers" of enterprise content, content deemed not so valuable, as defined by the organization, was largely left unmanaged; at best some organizations provided a file server where employees could store this content. In recent years however, organization have come to realize that the combined value of these modest sellers was equal to the value of the highly valued content. This is the Long Tail (LT) of enterprise content and managing the volume of content produced by employees is a huge, enterprise content management, challenge for organization.


Chris Anderson coined the term “The Long Tail” to describe how the internet has made possible a world in which the combined value of modest sellers (in the world of movies, books and music) equals the sales of the top hits. The long tail has been applied to other industries and I will explore its applicability to enterprise content and in defining an enterprise content management strategy.