The impact of specialised translator training and professional experience on legal translation quality assurance: an empirical study of revision performance

Figures & data

Table 1. Number of participants per profile group.

	1	2	Total
LL	10	2	12
LT	10	5	15
T	5	5	10
T0	5	2	7
Total	30	14	44

Figure 1. Screenshot of TranslationQ’s interface.

Figure 2. Average number of total, justified and missing corrections per profile group.

Table 2. Average correction scores per profile group.

	All	Justified (of which mandatory)	Unnecessary (% over total)^6	Missing mandatory
LL1	32.60	14.40 (8.00)	18.20 (55.83%)	4.00
LL2	18.00	11.00 (7.50)	7.00 (38.89%)	4.50
LT1	30.70	15.70 (8.50)	15.00 (48.86%)	3.50
LT2	25.00	13.40 (8.00)	11.60 (46.40%)	4.00
T1	23.00	13.20 (7.60)	9.80 (42.61%)	4.40
T2	14.80	9.80 (6.80)	5.00 (33.78%)	5.20
T01	20.60	10.80 (6.00)	9.80 (47.57%)	6.00
T02	11.50	6.50 (5.00)	5.00 (43.48%)	7.00
Overall	25.61	13.07 (7.55)	12.55 (48.98%)	4.45

Figure 3. Average number of total, justified and missing mandatory corrections per profile group and error group.

Table 3. Average correction scores per profile group and error group.

	Legal content corrections				Other corrections
	All	Justified (of which mandatory)	Unnecessary	Missing mandatory	All	Justified (of which mandatory)	Unnecessary	Missing mandatory
LL1	18.60	11.00 (6.80)	7.60 (40.86%)	2.20	14.00	3.40 (1.20)	10.60 (75.71%)	1.80
LL2	14.50	10.00 (6.50)	4.50 (31.03%)	2.50	3.50	1.00 (1.00)	2.50 (71.43%)	2.00
LT1	18.10	11.90 (7.20)	6.20 (34.25%)	1.80	12.60	3.80 (1.30)	8.80 (69.84%)	1.70
LT2	17.40	11.00 (7.00)	6.40 (36.78%)	2.00	7.60	2.40 (1.00)	5.20 (68.42%)	2.00
T1	14.80	10.20 (6.40)	4.60 (31.08%)	2.60	8.20	3.00 (1.20)	5.20 (63.41%)	1.80
T2	8.20	7.00 (5.40)	1.20 (14.63%)	3.60	6.60	2.80 (1.40)	3.80 (57.58%)	1.60
T01	9.20	8.20 (4.80)	1.00 (10.87%)	4.20	11.40	2.60 (1.20)	8.80 (77.19%)	1.80
T02	7.50	5.50 (4.00)	2.00 (26.67%)	5.00	4.00	1.00 (1.00)	3.00 (75.00%)	2.00
Overall	14.98	10.05 (7.00)	4.93 (32.93%)	2.00	10.23	2.50 (1.16)	7.20 (70.44%)	1.84

Table 4. Over-corrections and errors remaining in revised TT per profile group.

	Total over-corr.		Legal content over-corr.		Other over-corrections		Errors remaining in revised TT
	Corr.	Avg. (% of total corr.)	Corr.	Avg. (% of total corr.)	Corr.	Avg. (% of total corr.)
LL1	7	0.70 (2.15%)	6	0.60 (3.22%)	1	0.10 (0.71%)	4.70
LL2	-	-	-	-	-	-	4.50
LT1	2	0.20 (0.65%)	1	0.10 (0.55%)	1	0.10 (0.79%)	3.70
LT2	-	-	-	-	-	-	4.00
T1	3	0.60 (2.61%)	3	0.60 (4.05%)	-	-	5.00
T2	1	0.20 (1.35%)	-	-	1	0.20 (3.03%)	5.40
T01	5	1.00 (4.85%)	1	0.20 (2.17%)	4	0.80 (7.02%)	7.00
T02	-	-	-	-	-	-	7.00

Table 5. Average time devoted to the revision task per profile group (in minutes).

	1	2	Overall
LL	99.50	100.00	99.75
LT	82.40	75.00	78.70
T	72.00	72.00	72.00
T0	83.00	49.00	66.00
Overall			82.43

Table 6. Revision scores and times for the best and worst revisions.

Ranking	Participant	Justified corr. (incl. mandatory)	Unneces. corr.	Missing corr.	Over-corr.	Remaining errors in TT	Time
1	LT1–3	18 (11)	22	1	-	1	70
2	LT2–2	21 (10)	11	2	-	2	70
3	LT1–10	19 (10)	22	2	-	2	84
4	LL1–3	18 (10)	22	2	-	2	90
5	LT1–7	17 (10)	7	2	-	2	75
40	T2–3	9 (6)	8	6	1	7	75
41	T01–3	17 (5)	14	7	1	8	90
42	T02–1	7 (4)	3	8	-	8	40
43	T02–2	6 (6)	7	6	2	8	58
44	T01–4	6 (4)	1	8	-	8	85

Figure 4. TT holistic assessment per profile group (average values).

Table 7. TT holistic assessment per profile group (average values).

	LL		LT		T		T0		Overall
	1	2	1	2	1	2	1	2
Semantic accuracy*	2.60	3.00	2.70	2.60	2.00	4.00	4.00	4.00	2.95
Language and style	3.30	3.50	2.90	2.80	2.60	4.20	4.20	3.50	3.30
Terminological and functional adequacy	2.70	3.00	2.90	3.00	2.60	4.40	4.20	3.50	3.18
Overall	2.70	3.00	2.80	2.60	2.20	3.80	4.20	3.50	3.00

*Scale: 0-1 = unacceptable/very bad; 2 = poor/bad; 3 = borderline; 4 = acceptable/good; 5 = excellent/very good.

Table 8. Illustrative comments about the quality of the revised TT (our translations).

Predominantly negative	Predominantly positive
LL1–1: I detected several issues of semantic accuracy and internal consistency (the same term is translated in multiple ways, which is unacceptable for a short text), and low thematic competence (deficiencies of background legal knowledge).	T01–1: Although I indicated several potential improvements, there are barely any substantial errors that could affect the correct comprehension of the text.
LL1–7: In general, the target text reflects lack of knowledge of basic legal notions, as it has been drafted closely following English phraseology. This lack of knowledge could have been overcome through a minimum amount of terminological and conceptual research, but this was apparently not done. As a result, the target text is comprehensible but of a rather poor quality.	T01–2: Despite some inconsistencies, there are no serious mistranslations and the target text is easily readable.
LT1–3: In general, the target text lacks in textual and functional adequacy. It does not conform to the phraseology and conventions of its genre. There are also several terminological and phraseological inconsistencies. The final text does not seem to have been prepared on the basis of the necessary prior research.	T01–4: The issues corrected do not hinder text comprehension. The translation seems to be the work of a specialised translator.
LT1–8: In my opinion, terminological accuracy and register adequacy are mediocre, and there are some ortho-typographic errors related to the use of caps and punctuation. I have also identified mistranslations which, given the type of text, are particularly serious due to their potential legal implications.	T01–5: The target text is perfectly understandable and fluid to a certain extent, even if translation solutions are not totally systematic and one sentence reads rather intricate.
LT2–4: The quality of the target text is very poor. There are terminological errors, seriously misleading false friends, multiple terminological and phraseological inconsistencies, calques from English and fluidity issues that considerably hinder target text comprehension.	T2–3: I find the target text easy to read in general. Given its descriptive nature and low level of specifics or specialisation, the terminology is appropriate for the intended register. There are some minor consistency issues, but they do not hinder text comprehension in context.

Table 9. Participants who considered that the TT was produced by an MT system (proportions per profile group).

	1	2	Overall
LL	50.00%	50.00%	50.00%
LT	90.00%	100.00%	93.33%
T	60.00%	60.00%	60.00%
T0	20.00%	100.00%	42.86%
Overall			65.91%