Mantra Inc. delivers first automated system for manga translation
We have exciting news for manga enthusiasts and all fans of comics! Japanese start-up Mantra Inc. has delivered the first automated system for instant machine translations of manga. The Mantra Inc. translation engine features image and character recognition. Better still, it can also consider context. As such, this impressive development has important implications for machine translation in general.
Comics are hugely popular in many cultures across the globe. From Marvel’s superheroes to Charles M. Schulz’s iconic Peanuts, comics and comic strips have been enjoyed by billions worldwide. Unfortunately, they are uniquely difficult to translate and so tend to be available only in their domestic markets. But the reach of comic book series is about to expand, thanks to the latest technology.
Table of contents
Why are comics so difficult to translate?
In many ways, comics represent the perfect storm for machine translation. The speech bubbles are certainly problematic. The words spoken by the characters are often split into multiple bubbles. These don’t tend to be neatly aligned horizontally or vertically but rather distributed randomly across the page.
In addition, speech will alternate between characters.
To achieve an accurate translation, it is necessary to identify which character is speaking and to parse the structure of the story. Otherwise, it isn’t possible to understand the context of each section.
Context
Translation software must be able to distinguish between male and female characters in order to insert the correct pronouns and ensure that the dialogue is sequential. It must also be able to utilise the context of previous and subsequent speech bubbles and tackle ambiguities such as words with multiple meanings.
All sentences in any scene must be analysed before producing a suitable translation.
Comics and manga feature very few words and so the choice of words is crucial.
Those words must fit into the speech bubbles without the necessity to resize or reposition them. Even if it were possible to resize the bubbles, changing them could negatively impact the graphic content of the publication.
The Mantra Inc. Solution
The team at Mantra have evolved a system that recognises and processes the scene, reading order and visual cues of comic book stories. They believe that they have developed the first multimodal, contextually aware translation system. To achieve this, they have been forced to overcome the usual limitations of NMT translations which tackle one sentence at a time and do not consider context. Text recognition, machine translation and image processing have been assimilated into a single system by the Mantra developers.
This significant advancement was delivered via automatic corpus construction. The AI software was trained using original manga stories and their translations. An incredible two million sentences were used to train the system.
The Mantra Inc. system recognizes Japanese manga texts and translates them into the target language using the trained neural machine translation model. The original Japanese texts are removed from the page. Translated texts are then added to the cleaned page. Font sizes are optimised to perfect the presentation.
Cloud-based system released
The Mantra Engine has been now been released. It is a cloud-based service that enables the swift rendering of manga into English and Chinese. Further languages will be added in the future. Aimed at both publishers and distributors, the system can reduce the time taken to translate manga by up to 50%. It is user-friendly and enables sharing for all parties involved in a project. Multiple stakeholders can participate in the translation process simultaneously.
The implications of the Mantra Engine
Manga has proved to be prohibitively expensive to translate, confining the distribution of many titles to Japan. English and Chinese speaking manga fans could soon be able to access more publications than ever before. There is no doubt about the appeal of manga, as many titles feature in the top twenty best-selling comics of all time. Indeed, One Piece, created by Eiichiro Oda, has sold more copies than any other comic on the planet.
But the implications of the Mantra Engine extend beyond improving the accessibility of manga. The system presents the possibility that all comics could reach larger audiences and reach them quickly.
Naturally, this AI system represents a significant advance for machine translation. As such, it could be the beginning of a new era. One in which neural machine translation is far more effective. One day, NMT could provide accurate and appropriate translations when images are integral to the meaning and context of the source material.
The future of NMT translation
Only time will tell whether the Mantra Inc. system is a game changer. It could massively increase the distribution of manga and any publications that are heavily dependent on their visual content such as children’s books. The ability to analyse context and to distinguish gender could take NMT translation to a whole new level.
New publishing perspectives in Japan
Are you seeking to sell the rights for the publication of your manga in Japan?
We have the expertise and partners you need. Please contact the Word Connection team for information about our foreign rights agency and proofreading services.
