Month: June 2009

Posted on

Cloud Control: What to Look Out For When People in Your Company Use Web-based Software

As an enthusiastic user of SaaS (“Software as a Service”) applications, I’ve increased my own productivity via the cloud. But while wearing my Information Governance hat I see companies becoming sensitized to information control and risk management issues arising from SaaS use. In particular:

  1. Company intellectual property (“IP”) frequently leaks out through employees’ SaaS use, often when subject matter experts within a company naively collaborate with “colleagues” outside the company; and
  2. Company information may be preserved indefinitely rather than being deleted at the end of its useful life, thus remaining available for eDiscovery when it shouldn’t be.
Productivity versus control over IP and information lifecycle management
Productivity versus control over IP and information lifecycle management

To illustrate these concepts, I’ll describe a bleeding edge cloud service I recently ran across that could cut both ways, producing both impressive productivity gains and control threats. It’s the as-yet-unreleased Pi Corp “Smart Desktop” project from EMC’s Decho unit, based here in Seattle. Decho is best known for its Mozy online backup solution, but it also provides a home for Pi Corp which ex-Microsoft executive Paul Maritz founded and lead for six years before it was purchased by EMC and added to Decho in 2008. (Maritz was since tapped to take over the CEO position at EMC’s VMware subsidiary.)

Pi Corp’s Smart Desktop project is described by EMC’s CTO Jeff Nick in this video taken at EMC World last year. In a nutshell, Smart Desktop is meant to:

  • provide a central portal for all of an individual’s information collected from all of the information sources they use;
  • index and classify that information so it can be used more productively, for example, when a user begins performing a particular task the user will be prompted with a “view” (dashboard) of all of the information the system expects they will want, based on the user’s past performance and the system’s predictive intelligence algorithms;
  • “untether” information so that it is available to the user from any of the user’s devices, including mobile devices, and interchangeable across different sources; and
  • enable secure sharing such that people can share just the information they wish to share with those they want to share it with.

Much of this is not new. Online file synchronization services like Mozy competitor SugarSync already offer central portals that enable users to share information across their devices, including mobile devices, and selectively share documents with other users. But the indexing, classification, and predictive prompting promised by Smart Desktop really stand out. This could be an addictive secret sauce that turns Decho into the Facebook of business SaaS.

Once I’ve had a chance to evaluate Smart Desktop I’ll take a harder look at its Information Governance implications. Problems could arise for employers — albeit through no fault of Decho — if Smart Desktop (or Mozy, or another file sharing service, for that matter) is used by employees to share their employers’ IP with people outside of the company, or people within their company who have not been properly trained and cautioned about maintaining IP security. Similarly, if Smart Desktop (or Mozy, or another SaaS) enables employees to preserve company documents beyond their deletion dates, or to access company documents after they are no longer employees, this could prove difficult in eDiscovery or IP secrecy scenarios, where such information could become a costly surprise late in the game.

But for now I’ll presume that because Decho’s parent EMC has a strong Information Governance focus, Decho will ultimately provide not only the access controls that they currently envision, which will enable secure sharing across devices and users, but also group administration features that make it possible for companies to retain control over IP and information lifecycle management. In particular, I predict Decho will provide dynamic global indexing of information which enters any user account within a company’s user group, thereby making company information easy to find, place holds on, and collect for eDiscovery. I also predict Decho will offer document lifecycle management functionality, including automatically enforced retention and deletion policies.

And while I’m making a Decho wish list, two more items:

  1. The contents of employee hard drives shouldn’t remain a mystery to the company until they are imaged for eDiscovery. I’d like to see an offering for Mozy Enterprise customers wherein all information which passes across all company-owned laptops can be backed up (or at minimum indexed) in a single company-managed repository in the cloud in order to strengthen the company’s control over IP and document lifecycle management.
  2. Smart Desktop’s indexing, classification, and intelligence sound somewhat like the cutting edge document clustering methods used in eDiscovery. I’d like to see its indexing and classification data anonymously aggregated and pooled across users and re-used to dynamically speed up and enhance the experience of new users and users who are starting new work flows.

In sum, companies concerned about information control and risk management issues arising from SaaS use should consider:

  • What cloud backup, synchronization and collaboration services are employees using, and how are they using them?
  • What obligations has the company clearly spelled out for employees, in employee manuals or individual employment contracts, regarding their use of these services?
  • What network management options does the company have, or could it obtain, for logging, monitoring, controlling, or blocking transfer of company information to such services?
  • If the way employees are using these services now poses an unacceptable risk, how can employees be moved to safer services?
Posted on

TREC and the gold standard for document review

Ron Friedman recently blogged an excellent critique of TREC Legal Track’s effort to objectively assess eDiscovery document review practices. Like Ron, I commend TREC Legal Track while wishing to offer comments that may contribute to their success. Like me, Ron is an attorney with long experience working in the technology sector, although for comparison with his math background I can only claim four years of college courses concerning statistical methods for assessing human behavior.

Benchmarking is valuable almost everywhere.

Benchmarking is valuable almost everywhere.

I strongly recommend reading Ron’s post for the benefit of his insights, whether or not you are already familiar with TREC Legal Track. I’d also like to offer my own observations about TREC Legal Track’s finding of low consistency between document classification decisions made by subject matter experts, who are spoken of as “gold standard” reviewers, and ordinary legal document reviewers. (In TREC Legal Track’s study, ordinary reviewers were 2nd and 3rd year law students. In real life the subject matter expert role is played by in-house or outside counsel, while much of the actual review work is performed by contract or outsource attorneys.)

Generally speaking, quality control processes involve benchmarking against some standard. Mechanical processes can be meaningfully benchmarked by physically sampling output (this is the essence of Six Sigma, in particular). For example, as machine parts come off an assembly line, samples can be selected and measured and the variance between their actual size and target size monitored not only to detect defects but to flag the processes responsible for defects. Human processes can also be benchmarked in a variety of ways. (This is in part the province of ITIL, the “Information Technology Information Library,” and the basis for the idea of “service level agreements”.) For example, those responsible for a customer service center may track the number of issues handled per hour, the type of issues handled, the number of resolutions or escalations per issue, revenue gained or lost per issue, etc.

Unfortunately, “responsiveness” and “privilege” are not only somewhat subjective in document review, standards for responsiveness and privilege will vary from case to case. For this reason standards need to be developed “on the fly” for each case, and these standards will by necessity be arbitrary (aka subjective) to some degree even if consistently applied. The good news is that the latest generation of document clustering software applications incorporate tools for developing consistent document review standards on the fly. Through an iterative feedback loop, the humans educate the machines to look for documents with certain characteristics, while the machines force the humans to refine their conception of responsiveness and privilege to a degree that the machine can reliably model it. After enough iterations have passed and the machine has reached some measurable standard of consistency, the humans can step back and let the machine do the rest of the review work. The machine does it more consistently than human reviewers could themselves, and at a much lower cost.

With document review the very idea of defining a “gold standard” for classification is less useful than it sounds. For instance, even if a panel of leading legal scholars could be formed for each eDiscovery matter, the mere fact that someone legitimately may be called a leading scholar doesn’t mean that their views will be consistent with anyone else’s — just well reasoned. But a “gold standard” is not what’s important here. What’s important is that in each case the attorneys responsible for responding to a document request do everything they can to carefully define and consistently enforce reasonable document review standards. This is what the current crop of document clustering applications are intended to do. That is the current model, anyway. I don’t pretend to be able to name the vendors who can or cannot deliver on this promise, although I think this will be the number one question in eDiscovery technology before long.

UPDATE: I discuss TREC’s role in forumulating new legal procedural rules for e-discovery in a later blog post, Catch-22 for e-discovery standards?

%d bloggers like this: