Heather Pruger, my co-blogger on The Daily Record’s Generation J.D. posted Are Document Reviewers About To Be Replaced? last week. It’s a good read, and important for both new lawyers (who need jobs) and practicing lawyers (who deal in electronic discovery–which is most of us, even if you don’t think you do). Here are some things to think about:
Magistrate Judge Andrew Peck (U.S. District Court, Southern District of New York) wrote an opinion and order on February 24 (See the full opinion here). The opinion deals with the production of electronic documents during discovery. What normally happens is this: a lawyer sues a big corporation, and sends out requests for production of evidence. The requests ask for all sorts of things, including company documents, spreadsheeets, memos and e-mails that deal with the litigation. Typically, the corporation will have its law firm find responsive documents. To that, the law firm brings its best and brightest and lowest paid associates, and even temp hires (those poor new lawyers who are still on the job market) to cull through thousands or millions of pages of documents, looking for responsive documents. It is rarely done with actual documents anymore–the documents are uploaded to a computer network, and there is a software program that helps reviewers code the documents. Document Review is typically considered “yucky lawyer work.” Sure, it’s fun for a day or two, but it will wear you out.
Well, in Peck’s case, a gender discrimination matter, the case was in its early stages and the parties were permitted limited discovery to determine whether the court had jurisdiction over one of the defendants.
So, it deals with computer-assisted coding, which Peck describes:
By computer-assisted coding, I mean tools (different vendors use different names) that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with (i.e., training by) a human reviewer.
Unlike manual review, where the review is done by the most junior staff, computer-assisted coding involves a senior partner (or [small] team) who review and code a “seed set” of documents. The computer identifies properties of those documents that it uses to code other documents. As the senior reviewer continues to code more sample documents, the computer predicts the reviewer’s coding. (Or, the computer codes some documents and asks the senior reviewer for feedback.)
When the system’s predictions and the reviewer’s coding sufficiently coincide, the system has learned enough to make confident predictions for the remaining documents. Typically, the senior lawyer (or team) needs to review only a few thousand documents to train the computer.
Some systems produce a simple yes/no as to relevance, while others give a relevance score (say, on a 0 to 100 basis) that counsel can use to prioritize review. For example, a score above 50 may produce 97% of the relevant documents, but constitutes only 20% of the entire document set.
Counsel may decide, after sampling and quality control tests, that documents with a score of below 15 are so highly likely to be irrelevant that no further human review is necessary. Counsel can also decide the cost-benefit of manual review of the documents with scores of 15-50.
The parties agreed to use computer-assisted coding, but they disagreed about the scope of the coding.
Don’t panic (famous words from The Hitchhiker’s Guide to the Galaxy which, really, is a good manual for life, the universe and everything). Peck only says this: ”This judicial opinion now recognizes that computer-assisted review is an acceptable way to to search for relevant ESI in appropriate cases.” The fact that the parties agreed to use computer-assisted coding is of paramount importance–if they didn’t agree, and say one side wanted to do it manually, and the other side wanted to use computer-assisted coding, then the court would have had to get involved. My bet is that a court would always allow manual review if the responding party wanted it (despite studies that show manual review is less reliable than computer-assisted coding), and that a court would likely allow computer-assisted coding if the responding party wanted it and could show that their process was reliable.
The most important thing to note is that document review jobs will vanish tomorrow. The cost to create and enable computer-assisted coding is extremely high, and is only likely to be efficient in cases with millions of documents. Those cases are out there, though, and as time goes on, they will probably go the route of computer-assisted coding. But, it won’t happen overnight, particularly where it is still very misunderstood, and where there are a lot of older stick-in-the-mud lawyers who don’t trust computers.
This Opinion appears to be the first in which a Court has approved of the use of computer-assisted review. That does not mean computer-assisted review must be used in all cases, or that the exact ESI protocol approved here will be appropriate in all future cases that utilize computer-assisted review.
New lawyers should still look for document reviewing jobs–it’s a good way to get your foot in the door at a large firm, it’s a good way to get some experience, and if it pays your rent, it’s definitely good.
When I mention e-discovery to other lawyers, the usual reaction is for the eyes to glaze over (maybe it’s me, maybe it’s the topic–I’m not going to commit to one or the other). E-discovery is simple–it is just like normal discovery, but it includes electronic stuff–e-mails, native files (for example, the non-printed Excel spreadsheet or Word document) and other computerized documents. If you are one of those law firms who try to ignore it, you’re doing your practice a disservice. It’s important to have a working knowledge of e-discovery, because you need to request it, or else it will probably be requested of you.
As for computer-assisted coding–this is the wave of the future. Costs are high, and they will probably always be high as the coding algorithms get more sophisticated. But it’s a good way to do business, it is reliable, and in the right case it will save you money.
If you have questions about e-discovery, computer-assisted coding, or other technology-related lawyer services, contact us at 443.850.4426, or online. We specialize in helping lawyers to prepare and present PowerPoint and other technology at trial, and website/blog enhancement.
Thank you for reading our blog. If you find it interesting, give our other blogs a chance. We also publish the Maryland Car Accident Lawyer Blog. Here are the stories for the past week:
And, don’t forget that we write a post every other Monday for Maryland’s legal newspaper, The Daily Record. Find our posts (including a new one this upcoming Monday) at Generation J.D., a blog intended to provide advice for younger lawyers.
And, by all reports, the current American Bar Association (ABA) president is unsympathetic to their plight. Not very nice, but the law grads need to make the best of their situation. So if you have a nice unframed J.D. certificate (because you can’t afford to frame it), and lots and lots of law school debt, click here for our Generation J.D. blog post, “How to find a job in seven (not always easy) steps.” Or, if you have a job, but want to know what a Corvair has to do with the whole kerfuffle, click the link.
Have a great week!