How you can assess the quality of a translation

You are a direct client that wants to translate a text from an unknown language towards your own language. How can you assess that your translator delivered a good quality translation if you don't know the source language and you have no experience as a translator? If the translation was very bad, a mere glimpse would tell you that the translator is an amateur. But, accepting that there is no such thing as a perfect translation, and what we are looking for is fair value for our money: How do we assess that the translation delivered is a fair value?

To answer this question we must first define what is a good quality translation, and to do this we must establish what we are going to assess and what is a text.

Although it might seem obvious, a text isn't only a group of words correctly organised; far from that, it's a series of messages that convey meanings, and so, what we must assess is the fitness of the translation to convey such meanings.

Thereby, the quality of a text can be assessed by two features: 1) Its INTELLIGIBILITY (the translation is understandable) and 2) its FIDELITY (the message transmitted by the translation corresponds exactly to the original message).

In this way we can have a very unintelligible text that has been translated with great fidelity, but producing a very unintelligible text; or a very intelligible text translated with scarce fidelity and still produces a highly intelligible text. Both texts are of low quality, the first one because the translator didn't ask the client to correct the unintelligibility issues of the source text before translating, producing something unintelligible (don't ask me how he did it), and in the second case because, either some of the original messages are missing or the translation delivers different messages to those originally expressed.

Unfortunately, to assess the quality of a translation through its intelligibility and fidelity, according to an experimental study ⁽¹⁾, requires the work of at least three or four reviewers, due to problems with individual subjectivity. Also, it is clear that for assessing intelligibility you need monolingual reviewers (they are not influenced by the source text) and for assessing fidelity you need bilingual reviewers.

Thereby, for the average direct client it is not practical to perform an expert assessment of a translation. Nevertheless, if the translation is into the clients own language, he will be able to assess, up to a point, its intelligibility, if he reads it as a proofreader; and if he has both texts reviewed by a bilingual person (another translator) he can assess, up to a point, its fidelity.

When the TSP is a recurring provider and the quality of his work has been proven, either through previous assessments or by fulfilling the clients expectations, in some cases, a self-proofreading by the translator should be sufficient (i.e. those specialties and levels of difficulty where he has been proven or has fulfilled expectations). See: What is a competent translator?

In those cases where the source text is sensitive or can give rise to legal consequences, and specially if the translator hasn't proved his competences in that field previously, it is highly convenient to have the translation reviewed by another translator, although this raises the costs.

One requirement of the EN 15038⁽²⁾ Standard is that every translation should include a proofreading step performed by a third party, however these translation standards have been created to regulate the behaviour of translation companies and agencies where handling big volumes of work increases the odds of mistakes. An individual translator follows a different work process, which doesn't mean that it would not be convenient for him/her to have third party reviewing.

What elements define intelligibility?
- Reads clear and easily; has no stylistic infelicities.
- Absence of grammatical errors.
- Use of terminology appropriate for the field.
- Correct syntactic arrangements.
- The message is clearly transmitted. It is not necessary to read twice.

What elements define fidelity?
- The translation conveys the same meanings of the source text.
- There are no misapprehensions or mistranslations of words or sentences.
- All sections have been translated completely, no additions or omissions (in some cases a translator may divide a sentence in two or join two sentences in one to avoid unintelligibility problems in a passage).

Why so many reviewers are required for an expert assessment?
Because when reviewers assess intelligibility and fidelity of a translation many personal factors, like their education and competences, their experience, their cultural standards, their values, their linguistic slant, etc., affect the results.

Is it possible to perform unbiased assessments of a translation?
One of the first "unbiased" assessments (of interest to a direct client) has a subjective origin. This is that the translation should be intelligible for the client. If the translation is towards the client's language he will be able to assess if it is intelligible or not. Although his assessment can be biased, only he or the readers that he represents can assess if they understand the messages transmitted by the translation. Someone could say that the client can suffer cognitive issues in reading, but odds are higher that the translator didn't try harder to write something clearer (tight deadlines or self-imposed urgency are the main causes for this kind of problems). In any case, this is only one facet of the quality of a translation.

Other unbiased elements which can be assessed by a client are the following:
- Spelling errors
- Punctuation errors
- Syntax errors
- Grammar errors
- Non compliance with usage, conventions or instructions.
- Use of words that are obviously wrong when connected to the context.
- Consistency in the use of terminology throughout the text. ⁽³⁾
- Errors related to formatting, fonts and design or layout.

The more errors a text has, lower is its quality; however, some errors are critical and others are of much less importance. A linguistic purist will be fussier than a pragmatic one, and what one might consider critical the other won't. In spite of this, they both should agree that if an error changes the meaning of a phrase (its message), it's a major error. If an error might lead to liability, health, or safety issues, it's a critical error. Everything else might be tolerable. However, and although it can't be classified as critical, a text with such a syntax that it must be read many times to get a grasp of it, can end up being intolerable, and so, isn't fit for use.

Quantitative measurements of quality try to determine an index or quality indicator, that can be useful for comparing translations (as when testing translators), but can't be used to assess if a translation is good or bad, because you can't define a value under which a translation is bad, that is, won't perform its intended purpose, or what is known as "Fitness for use". In the end, the quality of a translation is something subjective, and the fewer the reviewers the more subjective it gets.

Are language conventions unchangeable?
When conventions of the source and target languages are different, one usually should respect the conventions of the target readers. However, in some cases the source text can bring an implied message, specially in advertising texts and website texts, where transgressing conventions is a way of catching your attention towards key words or messages. For this reason the assessment of conventions must be done considering both the source text and the target readers.

Can we use Word's spell checker to assess a text?
Usually we can, but only if you use your brain also, because as any machine, Word doesn't think.

As an example:

“Eye ewes my spiel chequer so much, I own lee OK shun ally maker miss take.”

According to Word’s spell checker (2011) this phrase is perfect (for the US and the UK).