Show HN: AI to help you write fast. modes: gen, legl, hp, research, got. [OSS]

by jeffshek • 7 years ago

3 2 7 years ago

jeffshek2• 7 years ago

I built writeup.ai to learn more about machine learning. I underestimated the amount of time it would take (thought 1 month, took 3, classic 3x off) ... alas, the feature creep of my judgement was hard to manage.

I learned WAY more about deploying ML in production environments than neural networks. ML DevOps was surprisingly hard. the ecosystem is a bit fragmented with managing TensorFlow / PyTorch / Docker / Nvidia dependencies. since there's SO MANY ongoing changes they have a lot of out-of-date documentation.

the backbone of the algorithms running this is from gpt-2 (openai's text generation algorithm). i experimented with almost all of the other text models including bert, transformer-xl, xlnet. i looked into ctrl (salesforce). most of them had incredibly high memory requirements, speed wasn't the fastest, or the output wasn't as good as gpt-2. this isn't a knock on the other algorithms, they were designed with the focus of answering questions in test sets (reading comprehension, etc).

i fine-tuned gpt-2 medium w/gradient checkpointing for five different datasets: legal, harry potter, game of thrones, song lyrics, academic research. training was relatively straight forward, but the data scrubbing was incredibly tedious. i'm still training/updating the research models.

reddit has made some amusing harry potter and game of thrones fanfiction w/this so far: https://www.reddit.com/r/FanFiction/comments/d5s9yh/i_made_a...

app hosted on google cloud. everything is run via docker containers (which is also hard w/nvidia). autoscaling on usage. the main inference runs off from either cascade lake (cpu, surprising reasonable performance), k80s, or p100s. i experimented with the speed and profile of them a lot.

open-sourced at https://github.com/jeffshek/writeup-frontend, https://github.com/jeffshek/open

i'm going to write a detailed blog post about the code/deployment/infrastructure soon if you're interested. love to answer any questions and get feedback! thanks!

jeffshek2• 7 years ago

No one will ever see this self pep talk, but posting on HN and finding no one cares about your project makes me quite sad.

Just have to pick your head up and continue.

Show HN: AI to help you write fast. modes: gen, legl, hp, research, got. [OSS]

Related Stories