Applied research turned into tools, models, systems, and data.

Multidimensional Quality Metrics (MQM) has its roots in the work I did on the unreleased LISA QA Model 4.0. After I was approached by Hans Uszkoreit to continue working on a shared framework for evaluating human and machine translation, I moved to Berlin and began work on what became MQM as part of first the QTLaunchPad and then QT21 projects.

MQM started by comparing all then-existing publicly declared quality evaluation frameworks to find their common elements and differences. From there, I and my team created a master vocabulary and definitions that allowed us to declare all frameworks in a shared conceptual space and network of hierarchically linked error types (now called MQM Full). The key innovation was the ability to subset the master list of error types and represent it as differing degrees of granularity to meet specifications-driven quality requirements.

In addition, we introduced the Verity dimension (later renamed Audience appropriateness), a categorical approach for dealing with the need to adapt translations for different audiences that was entirely missing in then-current quality approaches. For example, if a manual written in English refers to a ground wire as “bare copper,” a correct translation into German as “bare Kupfer” would actually be incorrect because ground wires in Germany are sheathed in green and yellow insulation. The error types we developed to hand such issues were a genuinely new intellectual contribution in the field of translation quality evaluation. Although these issues were well known in translation theory, they had not previously been applied to systematic evaluation.

In 2014, it became apparent that MQM had to be unified with the TAUS Dynamic Quality Framework (DQF), which was then emerging as a shared way to exchange quality information. Over an intense period of collaboration in 2014 and 2015, both MQM and DQF changed, with DQF becoming a fully-compliant subset of MQM. The DQF subset found widespread industry adoption, but has now been replaced by MQM Core.

Since 2015, MQM has been managed by the MQM Council, which started as an informal management group, but which has evolved into a corporation with the mandate of promoting and furthering the development of MQM.

The Global Revenue Forecaster translates research on language, market behavior, digital access, and company performance into a practical planning tool. Its purpose is not simply to rank markets, but to help organizations reason about where additional multilingual investment may create measurable commercial opportunity.

The project combines external market data with company-specific inputs so that results reflect the scale and circumstances of an individual organization rather than a generic global ranking.

As a project, it illustrates a recurring theme in my work at CSA Research: taking analytical findings that would otherwise remain in reports or datasets and turning them into a reusable system that supports concrete business decisions.

The B2B Global CX Calculator is intended to help organizations evaluate where language and localization choices affect the experience of business customers across markets.

It brings together research findings and structured data in a form that allows users to explore market differences rather than treating language support as a simple yes-or-no requirement.

The project is designed around a practical question: given limited resources, where will improvements in multilingual customer experience matter most for a particular business?

The B2C version of the Global CX Calculator applies the same general research-to-tool approach to consumer-facing organizations, where the relevant populations and decision criteria differ substantially from business markets.

It is designed to make complex multilingual market data usable for planning by allowing organizations to compare countries, languages, and customer populations in a consistent analytical framework.

Keeping the B2B and B2C systems distinct makes it possible to model the assumptions and evidence appropriate to each rather than forcing both into a single generic view of global customer experience.

The project measures which languages and locales companies support and how deeply they localize their digital presence. The aim is to move beyond anecdotal comparisons and create a consistent basis for comparing companies within and across industries.

At its current scale, the analysis covers more than 3,000 brand websites. The resulting data can support company-level reports as well as industry-level benchmarking, including comparisons of language coverage and localization depth among peer organizations.

The project combines web-scale data collection, multilingual analysis, normalization across inconsistent site structures, and visualization intended for both research and client-facing use.

CSA Research has historically estimated changes in the size of the localization industry using a model built around observed changes in company revenues. That approach is being replaced because company-level revenue movements are an imperfect proxy for changes in the market as a whole and do not adequately represent the uncertainty in the available evidence.

The new work uses a Bayesian approach so that multiple forms of evidence can be combined explicitly and uncertainty can be represented rather than hidden behind a single deterministic estimate.

The project is therefore as much a methodology redesign as a market-sizing exercise: It starts by identifying where the older model makes claims stronger than the underlying evidence can support, then builds a framework intended to produce more defensible estimates as new information becomes available.

The project explores how generative AI can make a large proprietary research catalog more useful when it is combined with data and context supplied by an individual client.

Rather than treating a language model as a general-purpose source of answers, the system is intended to use CSA’s existing research as the analytical foundation and bring client information into that context so that outputs are specific to the customer’s situation.

This creates a bridge between a traditional research business, in which findings are published as reports, and an interactive model in which accumulated research can be applied dynamically to a client’s own data and questions.

The LISA QA Model 4.0 was a planned major update of the existing LISA QA Model 3.1. Widely deployed in the language sector, the LISA QA Model was one of the first shared error-count scorecards. However, by 2010, it was increasingly outdated and LISA realized the need to replace it with a flexible, open, and modular framework that the scorecard application could draw upon, but which would be available for integration into commercial and open-source localization tools.

I began work on building the system and had developed full specifications and built the first improved error typology for use in the LISA QA Model. Together with a colleague, we had started building an application to read and deploy arbitrary metrics. The forthcoming release had been announced.

However, it was never to be. LISA declared insolvency in early 2011, just before the planned launch, and with its demise, the LISA QA Model ceased to exist. However, no good ideas go to waste, and in early 2012 I was approached by Hans Uszkoreit to continue work on the concept at the German Research Center for Artificial Intelligence in Berlin. There, the principles and ideas behind the LISA QA Model 4.0 project came to fruition as Multidimensional Quality Metrics, which has since come to be the most widely-adopted framework for translation quality evaluation. The LISA QA Model may no longer exist, but its children are alive and healthy.

The New Advent Corpus grew out of academic work on a large collection of public-domain translations of writings by the early Church Fathers. The source texts had been gathered and maintained by another compiler; with permission, I prepared them for computational use through segmentation and conversion into TEI.

The result is a reusable scholarly dataset rather than a one-off research artifact. It preserves the underlying collection while adding structure that makes the material more useful for corpus analysis and other forms of computational research.

The project also reflects a separate strand of my work from commercial research: building open resources that other researchers can inspect, reuse, and extend.

View project record →