Skip to main content
Find a Lawyer

Managing an Online Document Review

Electronic discovery projects are, by nature, chaotic. When dealing with electronically stored information (ESI) from dozens of custodians, a variety of vendors and service providers, hundreds of different file types, several pages of search terms, and an uncooperative adversary, you have all the ingredients for confusion and disorder. The job of the legal team is to bring order to this confusion and counter uncertainty with structure and management control.

How should an organization go about accomplishing such a feat? There are eight key steps legal teams should undertake to successfully manage on online document review. Some will be familiar to legal teams who are already establishing a more skillful approach to electronic discovery projects. More often then not, however, document reviews are fraught with problems and can expose corporations and their outside counsel to risk of sanctions. Additionally, they can waste time and money without properly controlling the quality of the review process.

While there will always be problems with which to contend, following the steps below (listed here and elaborated in the remainder of the article), will result in a well-managed document review:

  1. Review Strategy

  2. Resource Planning and Timelines

  3. Review Workflow and Reviewer Procedures

  4. Data Analysis and Review Set Assignment

  5. Review Team Training

  6. Review Progress Tracking

  7. Quality Control

  8. Production Set Creation and Tracking

1. Review Strategy

Review strategy consists of the high-level decisions and activities that guide the online document review. The strategy needs to include the key objectives of the review and all considerations around time, resources, cost and risk. These key activities will drive all subsequent planning and management decisions.

In many cases, one primary strategic objective is simple: get the review and production done. Produce the right documents to the other side or investigating government agency and do it on time and on budget.

But review strategies will vary widely depending on the type of matter being managed. Conducting an internal investigation, reviewing and producing documents to comply with a second request, or coding documents that are being examined for multiple product liability matters require very different approaches. For example, if it's important to research discovery documents to assess the case, build fact patterns, or identify documents for depositions or exhibits, the reviewers will probably need to invest more time in evaluating and coding the documents (as opposed to making two relatively quick decisions on responsiveness and privilege).

Another key question to ask while plotting strategy for the online document review involves technology: how much does the legal team want to rely on search technology to code documents or to determine which ones will be produced?

In some cases, the answer may be "not much." A live attorney will need to review each document in order to accurately assess it, as the risks of producing privileged or sensitive information may be too steep to justify a more aggressive approach. In other matters, an aggressive approach that takes advantage of technological advances might offer the best balance of time, cost, and risk.

2. Resource Planning and Timelines

As with any project, it's important to begin with the end in mind. The most common "end" of an online document review is handing over a few DVDs that contain production documents to a FedEx representative who will deliver them to opposing counsel. Assuming there is a discovery cutoff date established for this hand-off, it's important to work backwards from that delivery date to build the document review schedule.

Larger discovery project schedules should span dates for preservation, collection, review and production activities. For the document review portion, the schedule should specifically include milestone dates for data publication to the online review tool, reviewer training, first document production to opposing counsel and final document production. Even if the dates are estimates, they provide some sense of the timelines and enable the team to plan and manage the online document review more effectively.

At its most basic, creating a schedule involves some simple calculations, including:

  • How many documents need to be reviewed?

  • By what date do they need to be reviewed?

  • How many documents per attorney per hour can be reviewed?


Creating a timeline includes marshalling resources for the review and clarifying roles and responsibilities. These resources (usually a combination of associates and legal assistants from outside counsel) will coordinate, manage and conduct the online document review. Other key members of the team include the electronic discovery services provider who hosts the data on its review platform and possibly other consultants that specialize in online document review projects.

It's also important to consider what equipment and facilities are necessary. This is imperative for large-scale reviews involving contract attorneys. In these cases, the review coordinators will need to arrange for appropriate space, computers and internet connectivity. Testing the performance of the review application in the review environment is a must, as results can vary widely and impact the efficiency of the review.

3. Review Workflow and Reviewer Procedures

Outlining the review workflow is paramount in creating an efficient, accurate plan to achieve document review objectives. Some of the key decisions around review workflow derive from the work product needed from the review team. Will the review team be performing a first pass review for responsiveness followed by a review for privilege? Will reviewers be issue-coding documents, and, if so, will they do it while also looking at the documents for responsiveness or privilege? Is there a need for reviewers to redact documents? Will there be a second pass review of responsive or privileged documents (perhaps by supervising attorneys)?

Based on these decisions, the review managers can configure the review tool to display appropriate tags or categories. They can organize the documents into appropriate folders and then communicate specific directions to the reviewers for how to conduct the review.

Of course, the review workflow can get much more complex. The review team may use technology to organize the documents into groups that will facilitate a higher quality review. Examples of more sophisticated workflows abound. It may be valuable to identify potentially privileged documents and assign those to higher level attorneys for review. Expert witnesses may have to review certain documents. The data collection may have foreign language documents that must be identified, translated, re-inserted into the workflow and reviewed.

Ideally, those coordinating the review will create a process diagram that captures the desired workflow (using a program like Visio or Word) and share that diagram with everyone involved in the project. Count on going through a couple of iterations of this workflow as new information or ideas emerge.

4. Data Analysis and Review Set Assignment

As the electronic discovery services provider is publishing documents to the online review application, the managers of the review team should examine inventory reports on the data. These reports are often integrated in the review tool or available as required from the services provider. They can be extremely valuable in revealing information about the document population, which will then help in expediting the review.

In any document population, there will be clusters of files - log files or broadcast email messages or graphics, for example - that will be clearly non-responsive. Identifying these patterns and bulk categorizing the items as irrelevant can save a tremendous amount of time by eliminating thousands of documents from the population that must be reviewed. As an auditing mechanism, reviewers can help validate that all of these items are non-responsive by quickly scanning through lists of the documents to make sure the item names fit the non-responsive profile.

In addition to understanding such data patterns, it's important to identify early on in the process how to organize the documents for review as this has a substantial impact on review efficiency. Often, legal teams organize the review by custodian, assigning one or more reviewers to review all the documents for a given custodian. This approach makes sense because that is often the way the documents will be organized for production.

Often times, however, review teams can improve on this approach by using concept or keyword search technology to group documents together for review (this process is also referred to as concept foldering or clustering). The advantage with this method is that documents about certain issues, transactions, products, people, etc. can be grouped together, enabling the reviewers to make document decisions more efficiently and consistently. Though not as flashy, other mechanisms, such as item sort, can accomplish a similar objective.

5. Review Team Training

Training for an online document review has two components - administrative training for the review managers, which focuses on configuring and administering the functionality available in the review tool, and the actual training of the reviewers on the online review tool.

The training should include appropriate documentation and contact information for technical support. The training materials should offer detailed reference and screen shots on how to use the online review tool accurately. When necessary, the training documentation should be customized for each separate review team (contract attorneys, privilege review team, second pass review team, expert reviewers, etc.).

As always, the clearer the review guidelines are, the better. If you want reviewers to code e-mails and their attachments the same way every time, it's important to communicate that clearly and capture it in training documentation. If reviewers need to enter an annotation each time they mark a document as "questionable," communicate and document that.

Finally, while it can be a distinct advantage of online document review to include review team members who are located remotely, a dispersed review team generally requires greater communication. It's important to identify whether the online review tool can accommodate a dispersed review environment or if the team must be managed by old-fashioned e-mail and conference call communications (consider setting up a reviewer distribution list).

6. Review Progress Tracking

After the review team has commenced review, the review managers can use the online review tool to monitor progress and ensure they are on target to meet production deadlines - and if necessary, make adjustments that will get them back on track.

Progress tracking should take place directly in the online review tool (most have various tracking and reporting functionality) or via reports supplied by the electronic discovery services provider. Tracking should take place on a few levels. First, the manager needs to monitor individual reviewer productivity to determine whether:

  • Are reviewers working productively?

  • How many documents per day or per hour are they completing?

  • Are certain reviewers lagging behind, and maybe in need of encouragement?


Those managing the review also need to monitor the progress toward production goals. If a review manager needs to complete review of documents for five custodians in 10 days, he or she needs to have clear reports telling them the total number of documents for these five custodians, at what rate the documents are being reviewed, and what percentage of the documents is complete. Based on these metrics, the manager can keep the project on track or re-assign resources to ensure the production deadlines are met.

If managing a multiple-phase review (say all responsive and privileged documents are receiving a second pass review prior to production), the legal team must monitor where documents are in the workflow and use the online review tool to re-assign documents to the next appropriate team.

7. Quality Control

A major advantage of online document review is the ability to use management controls to identify inaccuracies or inconsistencies. Prior to any production of documents, the review managers should run through systematic procedures to make sure the data set is right. In the online review tool, review managers should implement sample quality control checks such as the following:

  • Completeness (ensuring that every document has been reviewed)

  • Completeness of second pass review

  • Consistency in the coding of duplicates

  • Consistency in the coding of document chains and threads (if required).


Additionally, using keyword or concept search technology will enable reviewers to implement a host of other checks for privilege (by searching for attorneys' names against documents slated for production, for example), or other sensitive topics or documents.

As part of the quality control process, the manager should develop a checklist of quality control procedures that is right for the review and ensure the procedures are executed and signed off on a regular basis. Also, the review platform or service provider should offer reports to systematically identify discrepancies between the planned versus actual results in each phase of the review.

8. Production Set Creation and Tracking

The final step in managing an online document review is coordinating and tracking document productions, planning production formats, and identifying where the data deliverables (including privilege logs) are going. Often, a copy of the production documents will go to the other side as .tiff images and a load file that will be imported into their litigation support tool. Other formats, such as native file production, are a whole additional issue that may need to be considered. It's often necessary to send data exports to outside counsel -- and potentially a copy to in-house counsel as well. Finally, the legal team needs to consider how long they will need the data to stay available in the online review environment, in case they need it for later research or reference.

It's vital to identify the production format at the beginning of the online document review process, (and required with the amendments to the Federal Rules of Civil Procedure coming in December) so the manager knows what production expectations must be met. In addition, the review timeline should include a pilot production to the other side. Load files and data exports can be tricky even for experienced professionals. Ensuring the files are delivered as required by the requesting party and the courts will play a critical role in a company's defense, as well as reduce the risks associated with evidence tampering or missed deadlines.

By following the steps outlined for an effective document review, companies and their outside counsel can save a tremendous amount of time and cost in the electronic discovery process and focus on the most important matter at hand -- winning results.

Courtesy of Peter McLaughlin of Fios, Inc.

Was this helpful?

Copied to clipboard