With the announcement this week of the new initiativeGALE (Global Autonomous Language Exploitation) backed by DARPA, we seem to be entering a new phase in the development of MT (machine translation). But what can we really expect from this latest combination of government funding, academic activity, and new theoretical approaches?
First, MT isn't just a toy or hobby. Major corporations, IBM and Google in particular, are now actively and openly pursuing this Holy Grail of artificial intelligence. Further, the government, in particular the Pentagon, spurred by the needs of the war in Iraq and the larger War on Terror, is investing heavily again in the promise of a system or device that will provide instantaneous translation or interpretation for anyone, anywhere.
Second, the new, now popular theoretical approach to MT: statistical machine translation. This approach leverages the vast quantities of documents in many languages available on the Web for use as parallel texts, which are then subjected to statistical analysis to find matches, which in turn are used as reference when creating a new translation. Of course, the older rules-based approach is also incorporated so as to produce natural-sounding results in the target language.
All this sounds great, but so far it has yet to produce anything useful. Google's recent victory in the NIST-sponsored annual competition was impressive, but it is worth noting that the performance of their software was 50% similar to the reference Arabic/English text translated by a person. The other 50% was different, and only someone who knows both Arabic and English can tell which half is right and which half is wrong.
So as impressive as these results are, they hardly represent output that can replace what a human produces. Also, the reference text in the NIST competition was a newspaper article, hardly material considered difficult or demanding for human translators. The NIST will eventually have to provide more realistic texts, including research articles, engineering specifications, and patents, to name but a few, before Google and its fellow MT developrs can be confident that their systems are on par with human performance.
Finally, there are some other underlying issues that remain undiscussed or undisclosed as of today. One: how will Google or similar systems deploy their translation service? Uploading files to a Google server for processing might raise questions of privacy and security, especially if the content is proprietary or secret. Second: how will these systems contend with bad writing, neologisms, slang, idioms, and deliberate attempts by writers or speakers to be indirect, subtle, discreet, or even to intentionally hide meaning in language? Human translators have a difficult time dealing with such issues, so the machine translation systems probably have a long way to go in this important aspect of language. Last: how will the MT system's output be checked? Until it reaches 100% accuracy, someone will have to evaluate and correct the output, and that may prove to take more work than simply having a human translate the material in the first place.
Machine translation continues to look like a "magic bullet," as we refer to it on the Language Realm Blog, one that is regularly reported and hyped, but just doesn't seem to materialize. What does appear, little gadgets that provide translation of a few hundred phrases or the online translation tools like Babelfish, are more entertaining than useful. In effect, MT is like pornography: there is no clear definition for what it is, but we will all certainly know it when we see it. But unlike pornography, it has yet to be seen.