icer2020

The very first run needs to wait for the machine translation model to be downloaded so it won't be quick. For the subsequent runs, the model will be reused as a cache.

You can get them displayed by running: $ subaligner -lgs (on the first column). Notably, it is not the comprehensive list.

Hi, stanchsk,

After installation, simply passing in the source and target language codes should do the job. E.g., $ subaligner -m dual -v video.mp4 -s subtitle.srt -t eng,spa

Or just translate without synchronisation: $ subaligner_convert -i subtitle_en.srt -o subtitle_es.srt -t eng,spa

Subaligner provides a one-stop solution on automatic subtitle synchronisation and translation between 1,000+ language pairs.

Thanks and your select&annotate&render approach is definitely a cool solution. Some editors such as Wiki-authors may prefer plain text editing so I feel there could be a middle ground where users can toggle on or off the raw markups.

It does not produce audio itself but markups used for generating speech audio.

Very handy indeed. I wonder if a plugin can be made for Wiki2SSML on Narration Box.

Wiki2SSML eases the burden of voice editors preparing scripts in SSML, widely understood by modern speech synthesizers including Amazon Polly, Google TTS, IBM Watson TTS and Microsoft Azure TTS, etc. It is powered by WikiVoice which provides an unobtrusive solution of blending voice-tuning markups with free texts and creates seamless experiences of editing scripts and voices in one go.

s/DWT/DTW/g

Just realised another user reported that it did not work well for Russian movie and Polish subtitles. Nonetheless, it doesn't stop you from training your own subaligner with those media assets you possess.

Yes, subaligner should work for audio files as it does for video files.

Oh good to know! Never tried that combination before. Maybe this was due to the model pre-trained with the speech in English. Nonetheless, have you tried switching off the stretch with "-so"?

Aeneas is used for stretching pre-synced subtitle cue blocks, which is still experimental in subaligner. If durations of cues are correct in you case, there is no need to use this feature and passing in flag -so will switch it off. So why not get rid of stretch and see the difference?

What I know is Aeneas is using DWT which does not guarantee triangle inequality and for low voice or noise, DNN can handle those better with good enough model capacity.

The model was trained with features of human voice bound to a frequency range so it may work for "cross-language" sync. Why not give it a go and check the quality? It won't change the content of original segments but only shift them along the timeline if there are gaps.

A useful link. Need to do some comparison against different genres. Subaligner is just yet another tool and not one of a kind.

Oh nice to have that on the roadmap.

Subaligner does respect the original frame rate. How the player is going to interpret the subtitle timecode is hard to know though.

Now you can customize and train a new synchronizer using your own subtitles and audiovisual content: https://github.com/baxtree/subaligner

Containerised Subaligner for synchronising a subtitle file to its companion video.

Hi, tsukikage,

It works more effectively on subtitle segments with initial inaccurate timecodes and gaps (non-speech) in between. The scenario you described is equivalent to when you have a sequence of words with no associated timecodes. Subaligner is not implemented for tacking this problem but it has incorporated forced alignment at the second aligning stage. This feature is experimental and only for English atm but feel free to give it a go. There is a nice summary of forced-alignment tools if you only have a sequence of words as the input: https://github.com/pettarin/forced-alignment-tools

Subtitles can be out of sync due to a variety of causes including latency introduced by Speech-To-Text on live streams or manual rectification within the post-production process. Subaligner provides a one-stop solution on automatic subtitle alignment based on Deep Neural Network and Forced Alignment.

WikiVoice is a lightweight but expressive markup language used by voice editors for fine-tuning synthesised speech while editing raw scripts. Wiki2SSML turns WikiVoice into SSML which is a W3C standard widely supported by TTS vendors.

Feedback, contribution and PRs are all welcome.