Avoid Common TAR Issues:

Understanding TAR technology is helpful, but doesn’t always give enough education to evaluate things like the parity of types of sample documents, the impact of using production data versus one’s own data, and the type of seed documents. It is important to understand some of the common TAR issues so they can be spotted and remedied.

Issue 1: Cleat Explanation of Technology and Process

If someone is unable to name the TAR tool or algorithm used, then there is an issue. This is a clear sign that he/she doesn’t understand the analytics or AI process. The lack of understanding indicates that the output from these processes are potentially flawed.

Issue 2: Document Selection – How and Why

In TAR’s early days, training documents were selected basically at random. Now it has evolved to a point where people are choosy about what documents they use fo training. While this is a positive thing, it should make people pause to thing about the opposing party’s choice of documents. Are certain types of documents being under represented or over represented? A TAR system must understand what responsive and non-responsive documents look like so it needs to see numerous examples of each category. If a too small sample is used, the TAR system has the potential to incorrectly categorize the documents. It is also vital to use a variety of documents in training. If the type of seed documents (training documents) used do not mirror typical types of documents expected, this should cause suspicion. The parity of seed document samples is also important. While the ratio of responsive and non-responsive doesn’t have to be perfectly equal, there shouldn’t be a large discrepancy. If 10 times the number of non-responsive versus responsive documents are used, the TAR model can be distorted.

Issue 3: How Performance Is Measured

People often mention common TAR metrics like recall and precision, but rarely clarify to what they are referring. It is imperative to always understand which documents are related to these statistics.

Gaining a deeper understanding of TAR will help avoid common TAR issues.

 

For more Tidbits & Thoughts, please click here.