Measuring Legal Service Value, Part 2 Legal Ethics Column, Tuesday June 1, 2018.


What makes a great law firm? How can one quantify just how great a firm is, and compare it to its competitors? Last time in this space I suggested that legal service value has four elements (full paper here):

  • To the extent that a firm gets good legal results for its clients, it has effectiveness value.
  • To the extent that the firm’s fees are low and easy to pay, it has affordability value.
  • The more the firm’s practices minimize clients’ time and stress costs, the more client experience value it has.
  • Finally, if the firm’s work has many benefits and few costs for people other than its clients, it has high third party value.

In principle, a firm’s performance on these four elements of value can be quantified. If we could actually create accurate charts like these for legal services providers, we would have more empowered consumers, better self-awareness of strengths and weaknesses within law firms, and more evidence-based regulation. This project can also move us toward a more meritocratic legal profession, in which individual success is less dependent on racial and socioeconomic privilege.

An accurate quantification of value will require multiple metrics, and the array of metrics will depend on the legal niche in question. This column outlines three basic types of metric for quantifying legal service value: output metrics, internal metrics, and input metrics. Output metrics measure what comes out of the legal service provider. Internal metrics examine what happens inside the firm. Finally, input metrics analyze the people who walk in (or log in) to the firm each morning.

Output Metrics

Output metrics may analyze the outcomes created by law firms, or their work products. A simple example is win/loss rate, which can be informative in some administrative litigation niches if properly designed. Judges in England & Wales are now evaluating the advocacy of all lawyers who appear before them in criminal matters.

The best proof of a pudding is in the eating. Output metrics are, in principle, the bestway to evaluate legal service value. If we can tell from its outputs that a firm consistently produces great, highly affordable results for its clients, while leaving them happy and doing good in the world, we can be confident that it is a great law firm.

Unfortunately, output metrics for legal services are methodologically troublesome. They must be valid — they must measure what we are actually interested in instead of something else. Measuring affordability value by comparing the average bills of three law firms over the last 10 cases handled by each would not work, because any differences would be just as likely to reflect the complexity of the cases they handled. Especially in contested matters, the outcome depends on a wide range of factors other than the inherent value of the legal service provided.

One egregious example of an invalid output metric is the personal injury “Litigator Awards” handed out by the “Trial Lawyers’ Board of Regents” in the United States. This metric is based on whether or not a firm has settled cases for more than certain amounts within a set period. However no account is taken of the inherent strength of the cases in which these results were obtained. Thus, receiving a “Litigator Award” may well speak more to the prowess of a personal injury firm in attracting high-value cases than it does to the effectiveness of its advocacy for those clients.

Even if valid, an output metric may still lack reliability if the sample size is too small. For example, surveying past clients can be a good way to assess client experience value. However, because clients have idiosyncratic expectations of their lawyers, and some client evaluations are unreasonable or biased, the results cannot be relied upon unless a sufficient number of clients were surveyed.

Internal Metrics

Looking at what happens inside law firms is an an alternative to scrutinizing outputs in the effort to identify value. Most consumers aren’t interested in what goes on inside — they care about the outputs. However certain practices and structures demonstrably increase the chance that high-value services will be delivered. Checking for these practices and structures can be a helpful way to identify the firms likely to consistently produce great value, especially when output measures are methodologically impossible.

Some internal metrics focus on processes — what lawyers, and other people involved in legal service provision, actually do when working for a client. Common sense practices such as actively listening to clients and giving them a single point of contact within the firm have been empirically demonstrated to produce client satisfaction.

Structural attributes can also be the basis for good internal metrics. A firm that is free from harassment, racism, sexism, conflict, and excessive turnover is likely to produce better value.

The methodological challenge of internal metrics is that the information is private. Self-reporting is one way to get at this data. Peer file review and audit are somewhat more forceful techniques sometimes used by regulators and legal aid funders.

Input Metrics

How many years of experience does the lawyer providing the service have? What were her bar exam scores? How many practice hours have the firm’s staff accumulated in the niche? These are examples of input metrics, focused on the attributes of the people who do the work.

The advantage of such input metrics is that the data are relatively easy to gather and compare. The disadvantage is that this data seems to have a weak relationship at best with meaningful elements of value. Potentially more fruitful are surveys identifying the knowledge or skills that are most important for other lawyers to possess. If a variety of substantive knowledge (e.g. of the Criminal Code) or a personal skill (e.g. persuasive writing) demonstrably helps practitioners provide high quality services in a certain niche. If we have data about the extent to which different practitioners have these attributes, that can help us quantify and compare the value they offer.

A Steep Path, Worth Climbing

The value of a legal service is a complex aggregate of its effectiveness, affordability, client experience, and third party effects. Measuring it accurately requires gathering data from multiple sources, using a variety of methodologies. A “tyranny of metrics” that are misleading or counterproductive might be even worse than the status quo, in which the true relative value offered by different firms is mysterious, even to insiders.

However the author’s view is that it is entirely possible – and entirely worth the effort — to create accurate, objective methods to quantify the value propositions of different firms in different legal niches.

For a more detailed account, please see

Measuring Legal Service Value, Part 1 Legal Ethics Column, Tuesday April 5, 2018.


If you work at a law firm, how good is that firm? If you’re a client or potential client, how good are the different legal services providers that you might choose to patronize?

It’s too difficult, at present, to answer these questions in an objective and reliable way. This is most obviously true for individual people with legal needs. They generally confront a mysterious landscape populated with apparently indistinguishable law firms, as well as proliferating alternative sources of legal services.

However, even experienced corporate clients, and lawyers themselves, lack solid information about the respective merits of different legal service providers. To evaluate quality, they must often rely on opaque, methodologically suspect rankings or else anecdotal impressions of firms.

We need better ways to quantify and compare the value propositions offered by different providers of legal services. This column proposes a definition of legal service value, and next time in this space I’ll propose and categorize metrics for quantifying it. A detailed paper on these topics is available here.

Elements of Legal Service Value

There are four types of value offered by a law firm:

  1. Effectiveness Value means accomplishing clients’ legal goals and protecting clients’ legal interests. In litigation, this can mean maximizing (or minimizing) the award or settlement, or minimizing custodial sentence in a criminal defence matter. In uncontested matters, effectiveness means accomplishing the client’s legal goals, and minimizing all associated risks, to the greatest extent permitted by law and the client’s instructions.
  2. Affordability Value: More affordable legal services cost less. The way a firm structures its prices— for example by offering price certainty or deferred billing — also affects the affordability value the firm offers.
  3. Client Experience Value: Legal services are more valuable when they have a more favourable impact on the client’s time and the client’s mental state. Timeliness, communication, and understanding of the client’s life and/or business are valuable in and of themselves. If two firms are equally effective, and equally affordable, the firm that creates a better experience for its clients offers better value.
  4. Third Party Value: In addition to the client, there are various others who can be affected positively or negatively by a legal service and the way it is performed. Firms that create access to justice through pro bono work, or contribute to social goals like diversity and the rule of law, offer better value than those that do not.

The aggregate value offered by a service provider is represented by the space occupied inside this figure. The greater the area, the greater the total value offered by that firm.

The Value Profiles of by Different Firms

Different providers of a certain legal service (e.g. an initial public offering, or representation in an employment law dispute) have strengths and weaknesses in different elements of value. Quantifying elements of value separately, as this model seeks to do, allows consumers to make well-informed decisions, based on differing individual sensitivity to price and different aspects of value. Unlike rankings, this method can help consumers who are hunting for bargains, or those who are interested in results above all else.

The “All About Results” firm is the best of the four at getting the job done (high effectiveness value). The “No Frills” firm offers great affordability but modest scores on the other three elements of value. The third firm is “Service First” – its effectiveness and affordability are mediocre but it offers an outstanding client experience. Finally, the “Model Citizen” firm offers wonderful contributions to social goals backed by solid, if unspectacular results on the other three elements of value.

How to Quantify Value? 

This framework is of little use unless it can be filled in with real numbers. What data would quantify the extent to which competing legal service providers provide different elements of value? Next time in this space, I will consider output metrics, internal metrics, and input metrics – practical ways to measure and compare the value offerings of different legal services providers.

This blog is based on a working paper entitled “Measuring Legal Service Value,” which is available at

Shady Billing: Closing the Hall of Shame Legal Ethics Column, Jan. 30 2018

Shady Billing: Closing the Hall of Shame

Only “fair and reasonable” fees and disbursements can be charged by lawyers to their clients. This rule is uncontroversial, and applies across the country. Nevertheless, the following billing practices are used by some Canadian firms, and not clearly forbidden by regulation:[1]

Continue reading Shady Billing: Closing the Hall of Shame

Access to Justice Levies for Lawyers: Putting Our Money Where Our Mouths Are Legal Ethics Column, Dec. 12 2017

Tyrell Moodie, accused of drug offences and facing several years in prison, was denied a Legal Aid Ontario certificate because his income of $16,211 per year exceeded the cut-off threshold. Legal aid services for refugees in B.C. and Ontario were threatened with drastic cuts in 2017. Self-represented litigants are now the majority in many family courts, mostly because people cannot afford the legal assistance that they would love to have, and legal aid won’t pay for it.

Every media story about a legal aid shortfall includes a quote from a lawyer, pointing the finger at the government for inadequate funding. However, every time the legal profession points its finger at the state, three fingers are pointing back at the legal profession. As trustees and beneficiaries of the legal system, lawyers should make a more tangible contribution to ensuring its accessibility.

Continue reading Access to Justice Levies for Lawyers: Putting Our Money Where Our Mouths Are

Get Creative in Billing Clients for Dispute Resolution

Canadian Lawyer, November 20, 2017.


Most individuals looking for legal help in a dispute would love to scrap pre-paid, uncapped time-based billing, and lawyers should be open to that.

Imagine a list of clients’ top 10 pet peeves about law firms. Pre-paid, uncapped time-based billing would rank high. Why do so many firms in niches such as family law, estate litigation and employment law stick with this much-unloved system? How can these firms realistically and profitably move past it?

Risks for personal plight law firms

Distinctive business practices in a sector of the economy can often be traced to specific risks confronting firms in that sector. Pre-paid time-based billing addresses two distinctive risks forfirms in the “personal plight” area. (Personal plight is a sociological term for law practices in which (i) the clients are individuals and (ii) the legal needs arise from disputes.)

First, labour requirement risk reflects the unpredictability of what is required to resolve a certain file. How many hours will it take to get a certain matter settled or adjudicated? How many of those hours can be served by lower-cost juniors and clerks and how many will require higher-cost senior lawyers? The answer depends on factors that are hard to predict at the outset, including the behaviour of the other side or the client. Labour requirement risk is eliminated by a retainer allowing the firm to be paid for every hour that turns out to be necessary.

Second, non-payment risk is the chance that the client will not voluntarily and promptly pay the agreed-upon fee. Collection issues are common in personal plight legal practice. The clients are individuals with shallower pockets than corporate or government clients. The life crises that create these legal needs (e.g. loss of employment or divorce) will also often affect their ability to immediately pay for legal services. Non-payment risk is eliminated by a sufficiently large pre-paid cash retainrier.

What’s wrong with pre-paid time-based billing

The advantages of pre-paid time-based billing come at a heavy price to access to justice, client satisfaction and even firm economics. Clients don’t like not knowing how much services will cost before they agree to purchase them. They don’t like having to take it on faith that time-based dockets are truthful. And they definitely don’t like having to deposit cash retainers for services before they receive them.

Personal plight law firms that find workable alternatives to pre-paid time-based billing find it easier to attract future clients and more pleasant to deal with existing clients. They also spend less time and energy dealing with the client complaints that time-based billing tends to generate.

The problem is easy to state: How can personal plight law firms ditch pre-paid time-based billing — and reap the rewards of doing so — without choking on labour requirement risk and non-payment risk? Three leading alternatives often work more effectively: contingency billing, flatter fees and legal expense insurance.

Contingency billing

Contingency billing is a powerful pricing model that preserves accessibility while reducing risk for the personal plight firm. Because the firm can deduct its fee from the client’s settlement or judgment funds, non-payment risk is eliminated even though no up-front retainer is required from the client. The firm must still deal with labour requirement risk, but carrying a portfolio of contingency cases with different characteristics mitigates that.

Most people with personal plight legal needs would much rather pay on a contingency basis than through a pre-paid time-based retainer. Already ubiquitous in personal injury and class action cases, contingency fees could be adopted by more firms in areas such as employment law and estate litigation. Some provincespermit their use in family law cases not involving children.

However, when a personal plight client is seeking a significant non-financial remedy, contingency billing loses most of its appeal. In such cases, there is no “pot of gold” at the end of the litigation rainbow from which the firm can reliably collect its fee. Moreover, the alignment of incentives between client and firm — which is one of the contingency fee’s chief attractions — breaks down if the client needs the firm to pursue non-financial objectives, for which the firm will not be compensated. Thus, for matters such as criminal defence, family cases involving children and contested refugee matters, personal plight firms must seek other alternatives if they wish to move beyond pre-paid time-based billing.

Price certainty through flatter fees

To varying degrees, different fee models can also provide something that all consumers want: price certainty. Completely open-ended time-based retainers provide no price certainty whatsoever. At the other end of the spectrum, completely flat, advertised fees (e.g. “we will handle any Human Rights Tribunal case for $4,000”) offer complete price certainty to clients.

In general, the more price certainty a fee model offers the client, the more labour requirement risk it imposes on the firm. However, in research for my new book, I found firms using innovative models that offer clients significant price certainty without creating intolerable labour requirement risk.

A model that works for many firms is the “milestone fee.” A client will be quoted a flat fee to take a matter to a certain litigation stage, such as discovery or a preliminary hearing. Further stages are quoted and billed if necessary.

An innovative monthly billing plan was described to me by Toronto employment law sole practitioner “YY.” At the outset of a case, YY estimates how many months and how many of his hours would likely be required to take it through adjudication. He divides his projected total fee for the case (based on his hourly rate) by the estimated duration. The result is the amount that the client will pay each month. If and when the matter settles, the client pays nothing further. YY’s clients enjoy reasonable price certainty and deferred payment, and YY limits both labour requirement and non-payment risk.

Legal expense insurance

Best of all would be a personal plight fee model that offers certainty and deferred payment to the client, while freeing the firm of labour requirement and non-payment risk. The good news is that legal expense insurance is such a fee model; the bad news is that personal plight law firms cannot adopt it unilaterally.

A legal expense insurance, or LEI, policy is paid for by the client or the client’s employer with a price-certain monthly premium. If a personal plight legal need arises, the insurer will reliably pay the bill, so the firm experiences very little risk. The Unifor Legal Services Plan may be Canada’s most comprehensive legal expense insurance plan for personal plight legal services.

Unfortunately, few Canadians have access to union- or employer-sponsored benefit plans as generous as Unifor’s. Meanwhile, LEI plans available for individual purchase typically offer little or no coverage for dispute-related legal needs. This leads some to argue that a state entity such as Legal Aid Ontario might have a role to play in scaling up LEI coverage.


Pre-paid and open-ended time-based billing is a natural response by personal plight law firms to the labour requirement risk and non-payment risk that are inherent in their work. However, moving past it can make a law practice more successful as well as more accessible. Canadian personal plight firms have a variety of options, both new and old, for doing so.

Noel Semple is a faculty member at Windsor Law. His new book is entitled Accessibility, Quality And Profitability For Personal Plight Law Firms: Hitting The Sweet Spot. It is available free online from the Canadian Bar Association Futures Initiative.

Generalism and Access to Justice: Jack of All Trades, Master of None? Legal Ethics Column, October 6 2017

Generalism and Access to Justice: Jack of All Trades, Master of None?

The rise of specialization is among the biggest changes in the practice of law over the past hundred years. Most lawyers and paralegals are increasingly able to focus on a smaller number of legal niches. That is good news, for practitioners and also for clients. However, I will suggest here that generalist legal professionalism has an enduring role in fostering access to justice.

Continue reading Generalism and Access to Justice: Jack of All Trades, Master of None?

Personal Plight: Mending the Market Legal Ethics Column, Aug. 11 2017

Personal Plight: Mending the Market

“Personal plight” legal services are those provided to individual clients whose legal needs arise from disputes. Personal plight areas such as family law, refugee law, and human rights are the site of Canada’s worst access to justice problems.

The market for personal plight legal services functions poorly, as Malcolm Mercer and Amy Salyzyn have shown in this space. A key problem, I suggest here, is that it is too difficult for consumers to shop intelligently. This undermines healthy competition and legal professionalism, in addition to access to justice. Regulators can and should mend the market for personal legal services.

Continue reading Personal Plight: Mending the Market

Attorney on Amazon? Online Marketplaces for Legal Services (Law and Technology at Windsor Law Blog)



For an individual with a legal need, shopping intelligently for a law firm can be a frustrating experience. It is difficult to get any objective information about price or quality, and comparison-shopping is arduous. Are online marketplaces, which play an increasingly central role in the consumer economy, part of the solution to this access to justice problem?

The Canadian Bar Association thinks so.  The CBA’s Futures Report called for a “full-blown technology-enabled marketplace where sellers of legal services can present their offerings, credentials and fee structures.”  In such a marketplace, consumers would be able to “choose the types of services they wish to purchase,” and “investigate a seller’s quality and reputation.”  Online legal services marketplaces may be most appealing for individuals and small businesses, which usually lack internal legal expertise and long-standing relationships with firms.

This vision — an eBay for consumer law firms — could have significant appeal both for clients and for legal practitioners. Online marketplaces can connect firms and clients who would otherwise never find each other.  Today, many people with legal needs and some capacity to pay for legal services, nevertheless go without legal services. This is often because they cannot find a firm that is offering the service package the consumer wants and that is willing to accept the individual as a client.  After interviewing hundreds of self-represented litigants (SRLs), Windsor Law’s Prof. Julie Macfarlane reportedthat

A significant number of SRL’s describe ‘shopping around’ for a lawyer but with no success. Some SRL’s complained that while they were willing to pay for legal services, they could not find a lawyer willing and competent to take their case on. These respondents described placing numerous phone calls to lawyer’s offices – sometimes as many as 15 or 20 …


An online marketplace can connect more paying clients with firms, to their mutual benefit.

Online marketplaces are especially useful in facilitating long-distance retainers, between clients and firms that are geographically far apart. Long-distance retainers are increasingly feasible thanks to technological improvements and relaxation of interjurisdictional practice rules.  Online marketplaces can help lawyers target practices at sub-niches (e.g. Islamic family law or employment law for athletes), drawing clients from around the province or even around the world.

A sufficiently robust online marketplace could reduce law firms’ reliance on paid advertising and referral fees to connect to clients.  These traditional “offline” methods are not only expensive for law firms, but also problematic for access to justice and legal professionalism.  At best, they provide clients with little or no useful information about alternative firms.

Online comparison shopping is much easier for prospective clients than visiting multiple law firms and trying to distinguish between them.  Comparison shopping also fosters competition, which helps consumers in the long run. Increased competition may sound like bad news for lawyers. Indeed, for firms whose services are effectively commodities – e.g. simple wills or residential property conveyances – online marketplaces could heighten price competition and squeeze profits.

However, most law firms’ offerings are differentiated in some way. Family law firms, for example, don’t just sell divorces “off the rack.”  They make very different claims about their approach to files, the experience and attributes of their lawyers, etc.  Research suggests that, for differentiated offerings, a move to online marketplaces can actually make consumers less sensitive to price, and more attentive to quality differences.  This is apparently because good online marketplaces make it easier for consumers to get information about quality.



Steps toward Online Marketplaces : Public and Private

How close are we to the CBA’s vision of a “full-blown technology enabled marketplace” for personal legal services?   Steps have been taken in this direction both by legal services regulators and by private sector entrepreneurs.

Public sector efforts to connect consumers with firms online have come in two varieties. Directories are name-based: they let users look up practitioners, and see contact information, license status, and (sometimes) disciplinary history and practice areas.  Many of these are now specialized—the  National Directory of Professionals Assisting SRLs, operated by Windsor Law’s National Self-Represented Litigants Project, is a great example. Referral sites, by contrast, are need-based: a user answers questions about his or her legal need and a list of practitioners in the appropriate area is generated.

Private sector entrepreneurs have also responded to consumer desire to shop intelligently online for legal services.  Private sector sites generally obtain advertising revenue from law firms and list contact information. Some serve as intermediaries between client and firm. The American site Avvo, for example, sells fixed-fee service packages delivered by lawyers under contract with Avvo.  Private sector sites offer both name-based directories and need-based referralpathways.  Others specialize based on legal niche (e.g. family law) or client demand (e.g. for fixed pricing).



The Distance Left to Travel

Despite these steps, online marketplaces for legal services still face significant challenges. Providing reliable quality information is one challenge.  Marketplaces like eBay and TripAdvisor succeed because users trust the information they provide about the quality of different options. No online marketplace for legal services has yet managed this feat, although Avvoprobably comes closest. Personal legal services are credence goods. Ratings from prior inexperienced consumers would be of limited value in evaluating quality, even if users could trust that those ratings are not being manipulated by the firms or the online marketplace itself.

Users of online marketplaces also expect to find clear price information.  Flatlaw delivers, but most legal services for contested matters in this country are still billed by time or on a contingency basis. Even if firms could be convinced to disclose their hourly or contingent rates, this information still wouldn’t tell the consumer what the final price will be.

In other sectors of the economy, the most successful online marketplaces have all been private-sector ventures, and this may also be true for legal services.   However, regulators have an essential role to play.  Good information about price and quality is the lifeblood of an online marketplace.  For law firms, regulators have a unique capacity to define and gather this information (e.g. through mandatory annual reports from lawyers), and to make it public in a responsible way.

Understanding and measuring law firm quality is a major challenge for regulators.  Providing useful and accurate price data to consumers, without encouraging a “race to the bottom” or collusion among firms, is just as difficult.  Regulators in the UK are currently world leaders in tackling these problems.


Will shopping intelligently online for legal services ever be as easy as ordering a pair of shoes on Amazon?  I’m not holding my breath. Still, online marketplaces are definitely a “space to watch” as the legal industry slowly lumbers forward into the future.

Long Live the Law Practice Program

I am struggling to understand the justification for the recent committee recommendation to end the Law Practice Program. The LPP is the Law Society’s alternative licensing program predominantly used by candidates unable to find articling positions.

The committee‘s central rationale seems to be that the LPP is “perceived as second tier.” They acknowledge that (i) “there is no evidence to suggest that the LPP is in fact second-tier” and (ii) the LPP is “of very high quality and may, in fact, excel over articling in a number of areas” in terms of preparing candidates for practice (para 59).

A regulator ending the LPP because it’s perceived as second tier to articling is like a regulator banning Chevrolets because they are perceived as second tier to Cadillacs. A regulator which does so must, at very least, have a realistic plan to ensure that everyone will be able to drive a Cadillac/get an articling position.  I can’t find any such plan in this Report.

The committee could have proposed reforms to expand the articling stream to accommodate everyone. For example they could have proposed that every licensed lawyer be required to either serve as an articling principal, or else contribute x% of his/her law practice income to a fund used to compensate lawyers who do serve as articling principals.

In the absence of any such plan, ending the LPP simply eliminates a path into the profession which is disproportionately used by equity-seeking and relatively disadvantaged candidates.  Perhaps more importantly, it also deprives equity-seeking/ disadvantaged would-be-clients of 200+ new lawyers per year who would be more likely to serve them than articling-track lawyers are.

The Report’s only other serious argument against the LPP is that we can’t decide who should pay for it. It costs roughly $17k per candidate.  At present a portion of this is absorbed by LSUC. Articling stream candidates pay a large share, due to the equalization of costs for LPP-stream and articling-stream candidates.

Who should pay is a tough problem, and there’s a convincing argument that the articling-stream candidates shouldn’t have to subsidize LPP-stream candidates to the extent that they currently do.  Personally, I think LSUC fees should be increased, and made progressive based on licensee income, in order to fund LPP and other A2J-enhancing initiatives.

But even requiring LPP candidates to pay the entire $17k per year themselves would be better than completely depriving them, and their would-be clients, of the opportunity to practice for which they have already invested so many years and so many tens of thousands of dollars.

The perception of second-tier or stigmatized status for LPP and its candidates is unfortunate. LSUC should fight this inaccurate perception, not surrender to it. But even if they can’t or won’t fight it, a professional path perceived as second tier is better than no path at all.