DuoBook: Generate bilingual stories to learn any language

by celltalk • 1 year ago

95 80 1 year ago

celltalk20• 1 year ago

My best friend and I had been trying to learn Dutch for a while. About a month ago, we picked up some Donald Duck books in Dutch, thinking they'd be fun and simple to start with. But soon enough, we found ourselves constantly switching between the book and our phones, struggling to follow the story—definitely not the immersive experience we had hoped for...

That’s when I remembered those books with one language on one page and the translation on the opposite page. Inspired by that concept, I thought, why not use AI to create something similar, but even more interactive?

So, we built DuoBook.

Here's how it works:

1) Start writing your story in your language.

2) Select the language you want to learn.

3) AI helps complete the story, side-by-side with your native language.

It’s still early days, and it might not be perfect, but it's genuinely helping us—and we hope it helps you too!

Check it out: duobook.co

WalterGR• 1 year ago

That’s when I remembered those books with one language on one page and the translation on the opposite page.

One term for this is "Parallel Text" (https://en.wikipedia.org/wiki/Parallel_text).

Cool idea! I've been thinking about learning German - I'll have to give this a try.

celltalk20• 1 year ago

Please do! Let us know :)

gus_massa2• 1 year ago

Do you have a few samples that can be read without registering?

Does it highlight the matching words?

celltalk20• 1 year ago

If you go down on the main page, you will see an example of Spanish to English.

And, we don’t have language alignment yet. Still, it highlights some of the “hard” words.

vunderba2• 1 year ago

Nice job. I'd recommend "publishing" some bilingual books that have already been made on the platform as a better pattern for discovery since content creation requires signing up.

celltalk20• 1 year ago

Thank you! That’s a great suggestion maybe we can show the latest generated stories by other users.

wingerlang• 1 year ago

Is this a real origin story or a fake one? (Obviously written by AI)

celltalk20• 1 year ago

Hahaha I know it sounds fake but it is actually true.

https://imgur.com/a/nTbBgdJ

atoav• 1 year ago

At least with German Donald Duck the language used there is.. special in a good way. The early translator Erika Fuchs devised a whole new variant of German that:

- uses words and phrases that are either of rare historic origin or completely made up new ones

- verb forms so uncommon that verb form she used frequently (the Infliktiv) has a second inofficial name: the Erikativ

- she frequently borrowed from the biggest writers and poets in the German language in her translations

- for the younger figures there is an entirely made up youth slang that is both appealing and incredibly entertaining to read

The english originals are utterly boring to read in comparison. Her work has a literaric and entertainer quality of the kind that made generations realize there is no real border between serious high brow literature and comics.

benatkin2• 1 year ago

At first I had trouble coming up with an idea for a story. I think it could use more examples. I chose "Ayn Rand gets added to Mt Rushmore" and it made a nice story about it. The next one I might try is "Dingoes permitted as pets around the world". Basically anything you can imagine. I like the vocabulary quiz. I did Spanish and it had a new word and a couple that I'm fuzzy on, and that was useful practice.

Edit: Year of Linux on the Desktop is another.

Edit 2: For Year of Linux on the Desktop it did 2024 as that year. Might want to add the current date to the prompt and say that to have stuff imagined in the future be after that. Another thought is to have the LLM suggest a story prompt for you.

celltalk20• 1 year ago

Thank you for also trying the practice mode! Super useful feedback. I will add them to the TO-DO list.

yurishimo• 1 year ago

Hey! I'm also learning Dutch and trying it out. A couple of things I discovered. For reference, I'm a native English speaker and speak Dutch at a B2~ level but my reading vocabulary is probably closer to B1.

The "long" story is not that long. I was expecting something closer to 1000 words or more. Using your Donald Duck example, I doubt the long story was more than 4 or 5 comic book pages. I started with the magic mushroom example, intermediate difficulty, and long length.

I also generated another story using the "lost puppy" prompt, but this time advanced difficulty and long. The difficulty was ramped up which I appreciate, but the length was even shorter than before!

The speech synthesis is garbage, but I'm sure you already know that. For a free service (for now), I understand why some limitations are in place, but it doesn't look good for your product unless you're targeting beginners. I'm not sure what you're offering is useful to anyone above that level. 3 stories per day for me is about 10-15 minutes on advanced (including the generation waiting time).

I wish you luck with the project! Imo, your time is best spent now optimizing your spend so you can provide higher quality audio to go alongside the text. I'm sure you will have plenty of content generated, but surfacing that to users without it feeling icky might also be a challenge.

bossyTeacher2• 1 year ago

This is such an amazing idea and I would pay for it right now as finding text for A1-A2 is virtually impossible if your language is not incredibly popular

celltalk20• 1 year ago

How much would you be willing to pay for it for a monthly subscription?

We were thinking about a pricing page with few pro options. For instance, longer text generation, access to better LLMs, or much better TTS etc.

jasonpeacock• 1 year ago

Related is Prismatext, which intermixes the new language into the text of a book in your native language, so you get in-place, contextual learning of the new language:

https://prismatext.com/

vunderba2• 1 year ago

I built something like this for my own personal language studies about a decade ago called "Kindle Swap". You fed it an ebook in ePub/MOBI format along with a list of known vocabulary and it would sprinkle them into the book so you could read it on your Kindle at your leisure later.

https://specularrealms.com/projects/kindle-swap

RestartKernel• 1 year ago

This seems useful for related languages, but even then I wonder whether it is a good idea to blur the lines required for code switching.

kriro• 1 year ago

I'd suggest an extra tier where you can generate a very tiny story without signup to try it first.

As is, I jumped out of the funnel already.

celltalk20• 1 year ago

I know it’s a friction, but we were (maybe still are) a bit paranoid of abuse or bot attacks, as we are using a paid api to roll this thing. One suggestion others gave was to show stories generated by other users, I think that might work. What do you think?

oniony• 1 year ago

Just have the beginner puppy book available in all languages (or major ones).

random42• 1 year ago

1. Generate a story.

2. Make it available for all (major) languages.

3. Profit!

Also, integrate with captcha + rate limit to prevent abuse. Authentication alone is not strong enough deterrent to a motivated adversary.

yuvadam• 1 year ago

Provide a free experience that doesn't hit any paid APIs and doesn't require any registration or friction, you'll convert much better.

carstenhag• 1 year ago

Have beginner and intermediate examples (1 page) available. Also helps show what the real UI looks like. Because the current tiny demo UI is probably not the real one, not sure?

Also, pricing is not mentioned anywhere on the start page or before signup

InsideOutSanta• 1 year ago

Why not offer a whole library of stories generated by other users that have been reviewed by a human and deemed good?

I'm the target audience for this app, but I don't really want to generate my own stories; I want to read. Generating stories is just additional work for me, and I don't know how good the story is before I generate and read it. If I had access to a library of human-reviewed stories, I'd just read those, knowing that they are at least okay. You wouldn't even have to review them yourself; you could have a system for users to rate stories.

You're selling this as an AI product, but I don't care about AI. I care about learning a new language by reading stories.

j454• 1 year ago

- Stories generated by other users would be interesting.

- You could look at running a locally hosted model. There are some good story writing ones, albeit unsure for languages.

- Help visitors generate example stories for language pairs on your website ... if a language pair already has one, maybe show a pre-existing one.

.. if it's a new language pair being tested, inform the user it may be shared with others to let them see how the system works?

rzz3• 1 year ago

I also jumped out of the funnel and had this “I got tricked” feeling after selecting the story parameters. I’m someone who would use this, too. But I’m not giving you my personal information unless I can see what it does.

askl• 1 year ago

You don't have to confirm your email address there. Just enter something random. That's usually what I try on sites like this, when it doesn't work just close the tab and move on.

altern8• 1 year ago

Yes, same here.

ErigmolCt3• 1 year ago

Yeah, same here. I was intrigued but bounced the moment it asked for sign-up without showing anything

outside12342• 1 year ago

Yes, consider something like letting the user keep track of "minutes reading" or "word points" after generating a story as the point where you log them in.

There has to be user value to login or nobody will do it.

celltalk20• 1 year ago

There is a progress and practice page for users, maybe we should emphasize this better in the main page?

DHolzer• 1 year ago

consider telling us upfront about the login requirement and/or have the three demo stories prepared in all combinations, saves you money too.

wahnfrieden• 1 year ago

For Japanese specifically I made an iOS/macOS native app for learning by reading. It curates a library of RSS feeds and some books rather than generating them with AI because there are still issues with the naturalness of LLM-made Japanese writing. I've found many learners are still apprehensive of using generated content.

To calibrate the content to your reading level, rather than generating the content, it tracks your comprehension and shows you how much of a given webpage or book you already understand.

It has optional Anki integration if you don't want to use the built-in ones. I work on this full-time now and am about to launch a manga reading mode, plus Netflix caption lookups.

https://reader.manabi.io

hliyan• 1 year ago

Tried this with my native language. Many errors. Probably because the language is only spoken by 20 million people (Sinhala). Suggestion: ability to highlight one word and have its translation highlighted. Could catch errors like the one I encountered: "A robot lives alone on Mars" got translated to "A robot was born alone on Mars"

celltalk20• 1 year ago

Sorry to hear that. Do you think we should exclude it then? I think highlighting the word is second most requested feature so far. Thank you for the feedback!

j454• 1 year ago

Highlighting the word is extremely cool. Using these examples as feedback into the system ("how did the translation do?"), and letting users provide feedback that you can improve your model on with this feedback loop would be ideal.

hejsansvejsan• 1 year ago

Interesting bug: when pressing the text-to-speech button to hear my Swedish story read out loud, what I get is a pretty good rendition of the text read as if it were French.

celltalk20• 1 year ago

Thank you! We will have a look at it, but TTS right now is browser based. I think it would be cool to have better TTS (OpenAI or Google) for a new paid tier.

dustincoates• 1 year ago

Great stuff, congrats!

A couple of suggestions: - I'm learning Hebrew and I'm at the beginner stage, so it would be good to have niqqud. Even with the STT, it's helpful at this stage. - For the STT, every time I tried it, it just said something that sounded like "Dodd."

ErigmolCt3• 1 year ago

Kinda curious how good the AI translations are. If it's just straight-up machine translation, that might get rough fast depending on the language pair

flipchart• 1 year ago

Please split Portuguese into European and Brazilian

sagon_ren• 1 year ago

I tried it on a not so popular African language, pretty good result.

outside12342• 1 year ago

Good approach. I basically do the same thing with Apple Books. I read in Spanish and then when I don't know a word I highlight the whole sentence and click Translate.

celltalk20• 1 year ago

Exactly, that's what more or less the aim was. We will try to make the word by word following happen.

dinkblam• 1 year ago

very nice but:

1.) a "long" story is still only like 20 sentences 2.) a real translation for each sentence is nice but often you are still left wondering what each word means. a "word by word" literal translation would be more useful either as an option or additionally. or the ability to click on any word and see the translation (bonus points for the declinations / conjugations too)

ordzo• 1 year ago

I tried it on Icelandic and the grammar was horrible.

999900000999• 1 year ago

This is neat.

But, please focus on 3 or 4 languages. Do those well.

Asian languages ( apparently Cantonese isn’t on the list ) are very hard to machine translate.

As is this is just Chat GPT + AWS Translate + AWS Text To Speech. Along with Firebase for user management and a very nice UX front end.

To turn this into a product I’d select maybe 3 languages, French , Spanish, German and hire advisors for all 3. Work on creating a few stories edited by your advisors and add basic gamification/quizzes.

I like the idea though

shadowvoxing• 1 year ago

Disagree. I'm learning Polish -not the most popular of languages- and I think this idea is fantastic and much appreciated.

celltalk20• 1 year ago

Thank you both. Great that you like it! :)

hs586• 1 year ago

I wanted to work on a similar idea recently for my personal use, but I wanted to use books and stories that already have the texts in the two languages that I am interested in (e.g. Harry Potter).

So you pass in two texts and get back some form of aligned text. If you have some knowledge of the language you are trying to learn and are ok without a perfect sentence-to-sentence alignment, then this would work.

My motivation is to improve my wife's and my knowledge of each other's languages while reading books to our daughter.

For a moment I got excited that someone else had already built it :)

j454• 1 year ago

This is a great design to line up the similarities and context from sentence to sentence.

endofreach• 1 year ago

Whenever i am in a new country, i hit the first bookstore in the city centre. Child section often has disney books or similar. Sometimes even bilingual versions. I recommend it.

No idea how to order my food, but if i ever need to find my son, who's a clownfish, and was kidnapped, and i need to do it in italian, i am ready.

jamesdutc• 1 year ago

Congratulations to the creators for successfully releasing this product!

(I'm trying to start with a positive tone, since I have only negative things to say about the site itself. I want to make sure that I'm coming across a critical without coming across as mean.)

I spent a few minutes generating a couple of sample stories using their prompts for the pair that I'm most qualified to evaluate “English”→“Chinese (Traditional)” and just wasn't very impressed. Honestly, I think the approach is largely a dead-end.

Let's set aside that “Chinese (Traditional)” is not a language, and that someone with experience learning or teaching Chinese ought to know this (and, as I will argue, knowing this is critical to producing high-quality educational materials!) That the creators of this tool aren't particularly familiar with the languages themselves is probably much less consequential than that they don't really appear to be familiar with the pedagogy of teaching or learning languages.

One would anticipate that the languages that most learners want to learn are subject to broad market forces, and that, as a consequence, these languages already have a variety of high-quality, human-written primary texts and educational texts (many of which may even be free-to-access!) For the language pair I tested, this is definitely true, and I would encourage every learner to start with those materials (and to avoid anything AI-generated.)

(Of course, if I wanted to learn a less-common language where materials are hard to find this might be marginally useful—e.g., Telugu probably has more total speakers than Italian, but my local high school probably has an Italian class—but I would wonder whether the training set would be good enough to accurately reproduce the language. I suppose if I wanted to learn an endangered language, where they may simply not be enough native speakers to maintain a rich catalogue of written language, then someone could train an AI to reproduce this language to aid in learning, but a similar question arises as to whether this kind of preservation or reconstruction is sufficiently “faithful.”)

It's absolutely the case that AI tools are at a point where (for common languages) they are able to reliably generate grammatically accurate language, independent of its factual accuracy. Indeed, while I could spot fluency issues in the sample stories I reviewed (since, of course, “Chinese (Traditional)” is not a language,) I could not spot outright grammatical errors. (This is an impressive accomplishment for AI models!)

But this is really a solution looking for a problem (and, in my opinion, finding the most obvious but also least useful.)

Contrast these randomly generated story with the equivalent from a human-generated educational resource. In the case of a human-generated educational resource, the quality of language may actually be worse than than that in the AI generated resource (even in the face of sloppy AI writing tends to be!) In fact, in the case of Chinese (“Traditional” or otherwise,) this is absolutely guaranteed to be the case for an introductory text. Almost all introductory texts will be written in a very choppy, repetitive style: e.g., 「那隻狗很可愛。我養的狗也很可愛。」

(It's likely the case that even intermediate and advanced learning materials will not resemble actual primary texts. e.g., I was reading the news the other day and came across the sentence 「北捷重申，無論任何年齡，各車站閘門前的黃色標線內一律禁止喝水等飲食行為，除非是身體不適或母乳哺育」 which is perfectly appropriate for an intermediate learner… except 「閘門」 is simply not useful or appropriate textbook vocabulary!)

So why is the human-generated educational material better? Well, there's a lot of design to writing these kinds of materials. How do we teach and reïterate the most broadly useful grammatical structures and vocabulary? How do we teach this in a way that maximises retention? (And, often, how do we expose the learner to useful cultural background that will help them when they visit a region where the language is spoken?)

All of this is visible in human-generated materials, yet none of this is evident in these AI-generated materials. It is, in fact, this design that makes these materials useful in the first place. In the absence of it, we end up with vocabulary lists that define 「狗：dog」 next to 「呈現：to emerge」 where a human educator would align the difficulty of these terms to the order and process in which a human learner would learn them. Similarly, a human educator knows how to evolve a student's fluency with language and understanding of tone and register, taking them from 「媽媽: mother」 to 「母親: mother」 perhaps even strategically including 「媽咪: mommy」 or even 「阿母 a-bú: mother （台）」 to engage the student. (Real educators do this very often, and students tend to really like it when they get “fun fact”-style local flavour!) I have not seen anyone attempt to introduce any of this design into AI-generated learning materials, and I suspect this is why they always come across as being so bland and mushy. Instead, the AI-generated materials are creating only rote practice items (which is why their prompts typically include things like “limit the generated text to use only vocabulary as published in the prep materials for such-and-such language proficiency exam.”) This kind of practice is, indeed, useful, but it's debatable whether it's measurably more useful than just spaced-repetition with flashcards.

Now, contrast these materials with primary texts (i.e., written language artefacts produced for an audience of native speakers.) Primary texts are often very difficult to incorporate into language learning, especially for languages like Chinese. This is probably because at the introductory level, the materials simply aren't dense enough for an adult learner, and at the advanced level, probably because these materials are far too challenging given the amount of specialised terminology and vocabulary used. (There are, in fact, very appropriate materials that sit between these extremes, such as news magazines or short stories written for middle schoolers, but these materials can be hard to access.)

The benefit of the primary text is that it is very close to the actual goal of the learner: I really don't want to read a story about a lost dog, and I only do it, because with enough practice reading such drivel, I might eventually read ‘Dream of the Red Mansion’ or ‘Red Sorghum.’ As a consequence, what most learners will reach for are “graded readers” which are adaptations of well-known works with simplified language and grammar. I'm on the fence with how well AI can create these for us. On the one hand, there is a pedagogical and creative dimension to producing a good graded reader. The former may be possible to approximate with additional prompting (“use only vocabulary from this list; use only grammatical structures familiar to a learner at this tested level,”) but I'm not sure about the latter. The reader is probably losing a lot when we simplify Gandalf to ‘Run away now!’

So while I'm quite hopeful that AI technologies can improve language learning, this kind of tool just doesn't seem to add anything to what already exists and is already much better.

The approach is just too obvious. I think it's too focused on finding a way to adapt something we know that AI can do well (generate grammatically correct text) to something we want to be able to do more cheaply or effectively (teach language learners how to read) without really considering how to solve this problem.

celltalk20• 1 year ago

I totally disagree. We know AI can code well, still Cursor and Windsurf worth billions of dollars, because it works.

drillsteps5• 1 year ago

Lifelong foreign language learner here.

What would really help is a way to find stories at a pre-defined learner level in a particular language.

Ie you say you're studying Japanese, your learner level is such and such, give me an example of an article at that level (from a blog, news article, anywhere).

Regardless of good LLMs are rn it's still a slop if not verified by human speaker. Ability to find texts in open access written by human beings would be so much more helpful.

Of course a lot of them are not REALLY written by humans anymore but one problem at a time :)

PebblesRox3• 1 year ago

I like it! I came back today after making a story yesterday and it had not saved my language.

Also, I’m using it in the phone and don’t see a way to get the translation of just one word since I can’t hover.

celltalk20• 1 year ago

Wow great to hear! There is translation for only some words not all though. We’ll try to make language alignment happen.

PebblesRox3• 1 year ago

Another feature I would love would be the ability to get a printable version so I can hand it to my kids. With that it would be great to have an option for a line-by-line translation (maybe side by side so they could cover half the page to hide it) and then another option without a full translation but with a little dictionary at the bottom so they can look up unknown words.

And my preference in general would be for a more literal, word-for-word translation so I can learn what each individual word means.

dgimla20• 1 year ago

Yet another frontend for ChatGPT. You can do all of this on ChatGPT.

gxonatano• 1 year ago

Almost every time I see claims to the effect of "learn any language" or "learn all languages," on HN or otherwise, it's about a smallish subset of languages. No matter how many languages you think you support, you don't support them all, so why make such a wild claim?

DuoBook: Generate bilingual stories to learn any language

Related Stories