saip

Access to parallel corpora is a limiting factor in general. A good way to train a language translator is to use an open source dataset (several here http://opus.nlpl.eu/) to train a base model, and then fine-tune it with a smaller dataset specific to your domain.

In this case, the author claims pretty good accuracy, almost on par with Google Brain's!

  On my test set of 3,000 sentences, the translator obtained a BLEU score of 0.39. This score is the benchmark scoring system used in machine translation, and the current best I could find in English to French is around 0.42 (set by some smart folks as Google Brain). So, not bad.

A GAN style approach to learning and generating variants could be interesting as well. It could generate a couple of hundred plausible versions. Then you have another network that is trained to differentiate between fake and real colored photos which picks the best version.

Agreed. I imagine this has applications in compression as well. You could stream a movie (or a football game) in black and white and enable each device to color it on the spot. A similar technique could also be done for HD/3D/VR.

Hey, thanks! I'm one of the co-founders at FloydHub.

Our focus has been on individual data scientists thus far. We'll be rolling out a teams plan soon since folks doing deep learning at work (with on-prem or cloud infra) tend to run into more or less the same difficulties as individuals. If you have any feedback, I'd love to hear :) sai (at) floydhub.com

FloydHub | Cloud Infrastructure Engineer | San Francisco, CA | Onsite | Full-time | Salary: 100k-125k (0.5%-1% equity) | https://floydhub.com

FloydHub (YC W17) is building a Heroku for deep learning. We enable data scientists to do deep learning in the cloud with a few simple commands and without any of the infrastructure or DevOps hassles.

Every day, we handle training, scaling and serving of several thousand deep learning models on our GPU clusters and manage TBs of data. As an infrastructure engineer, you will be responsible for building and scaling our GPU cloud infrastructure. Not simply operating the system, but be part of architecting and building scalable and secure cloud infra.

Keywords: AWS/GCP/Azure, Kubernetes, Docker, Mesos, Terraform, Packer, Python.

Requisites: 3+ years of cloud infrastructure experience

We're a small team (4 engineers), agile and very early stage (YC W17). We're Stanford/CMU grads, with experience leading deep learning research at Bing/Microsoft, large engineering org (Location Labs, Avast) and infrastructure at LinkedIn. We're backed by some of the best VCs and angels in town (https://floydhub.com/about). If joining an awesome 4-person team doing AI/infra excites you, come join us! As a founding team member, you will have the opportunity to considerably impact not only the technical direction of the product, but also the culture of the company!

Email us: careers@floydhub.com or find more info at https://angel.co/floydhub/jobs/240935-senior-infrastructure-...

Likewise, any cloud infrastructure engineers in the Bay Area that might be looking, shoot me a mail - sai at floydhub dot com

DevOps is indeed a huge bottleneck in deep learning. Provisioning machines, installing drivers and packages and managing their dependency hell distracts focus from the core deep learning. At FloydHub (I'm a co-founder), we're building a zero-setup deep learning platform.

Spinning up a Jupyter notebook with Pytorch 0.2 is as simple as `floyd run --env pytorch-0.2 --mode jupyter`. All the steps you mention in your comment are automated.

DevOps hassles is, of course, just the first of many hurdles to doing effective deep learning. Experiment management, version control, reproducibility, sharing & collaboration, etc. are also other important problems.

FloydHub (YC W17) is building a Heroku for deep learning. Data scientists can train and deploy deep learning models in the cloud with a few simple commands and without any of the DevOps hassles. Instead of worrying about provisioning GPUs, installing drivers, and managing software dependencies, focus on what matters - the science itself.

We're small, agile and very early stage (YC W17). We're backed by some of the best VCs and angels in town. If joining a 4-person deep learning/infra startup excites you, come join our core founding team!

FloydHub | Senior Infrastructure Engineer | San Francisco, CA | Onsite, Full-time, Salary: 100k-125k (0.5%-1% equity), https://angel.co/floydhub/jobs/240935-senior-infrastructure-...

- Help us scale our massive GPU clusters and manage TBs of data. Expertise: AWS/Azure/GCP, Python, Docker, Kubernetes, Terraform

FloydHub | Growth and Product Engineer | San Francisco, CA | Onsite, Full-time, Salary: 75k-115k (0.25%-0.75% equity), https://angel.co/floydhub/jobs/245161-growth-and-product-eng...

- Help us grow 5x, 10x and 50x our current scale. You will lay foundations for our community building initiatives. Expertise: data driven growth mentality

FloydHub | Deep Learning Researcher and Writer | Remote | Full or Part-time, https://angel.co/floydhub/jobs/245449-ai-researcher-and-writ...

- Deep learning engineer/data scientist to help implement and write about the latest projects and research. Expertise: deep learning, writing

Email us: careers@floydhub.com

If you want to try out fastText without having to do any local setup, see https://github.com/floydhub/fastText.

FloydHub[1] is a deep learning PaaS for training and deploying DL models in the cloud with zero setup.

[1]https://www.floydhub.com Disclaimer: I am one of Floyd's co-creators

Hey! Glad you brought this up. We've gotten quite a few requests from students enrolled in Udacity courses and have been helping get them set up & run their class projects on Floyd.

Here's our instructions for the Self Driving Car Engineer nanodegree program: https://github.com/floydhub/CarND-Term1-Starter-Kit. Happy to do the same for your class as well! How can I reach you? Feel free to to mail us directly: founders@floydhub.com.

We've also reached out to folks at Udacity to see if we can offer any official support for the courses.

Thanks - fixed now! :)

When we started out, we were just building a bot (droid) to manage our deep learning workflows (scheduling jobs, logging results, etc.)

So, Flow + Droid => Floyd. Also, a hat tip to one of my favorite bands :)

You mean bash on Windows? The CPU version of dl-docker will work on Windows! But unfortunately, the GPU version does not. You would need GPU passthrough, which is currently not supported. See https://github.com/NVIDIA/nvidia-docker/issues/197

We’ve definitely felt the bane of forgetting to turn off some really expensive GPU instances. Efficient scheduling and spinning instances up/down as required is one of the first things we built to cut down on the costs.

Git is an apt analogy. The search space of hyperparameters is usually fairly large for most DL algorithms, so a good amount of experimentation is required to tune them. Things can start to get haywire without end-to-end version control of code, data, parameters, results, environments, etc. Definitely one of the core problems we solve.

That’s awesome! ML to some extent, and DL to a large extent are still not considered mainstream enough by many. The big players (Google, Facebook, Baidu) seem to recognize this and invest appropriately, but the others not so much yet. That’s hopefully a sign of a burgeoning market!

Good luck with your Udacity class! If you want to play around, take a look at our guide for Neural Style Transfer: http://docs.floydhub.com/guides/style_transfer/

Our Individual plan has no monthly fees, and is purely pay-per-use @ <50% of AWS pricing.

Defending against big players with almost infinite resources is always an interesting problem. Kind of have to hope they don’t come at you head on :) That said, there might be other aspects that come into play. For example, the market is fragmented with big providers each having their own frameworks and infra that they prioritize (Google - Tensorflow/GCP, Facebook - Torch, Microsoft - CNTK/Azure, Amazon - MXNet/AWS, Baidu - PaddlePaddle, etc.) Since there is yet no clear winner, which will likely be the case for the near future, vendor lock-in is not really desirable. We intend to be backend / framework / language agnostic.

I’ve heard FBLearner Flow is pretty cool for running/managing/sharing ML pipelines inside Facebook. Never seen or used it myself, but Microsoft had a similar internal tool called AEther that was very cool too. We’ve definitely taken inspiration from AEther in building Floyd.

Here’s an anecdotal story about how awesome AEther was (been a long time, so a little fuzzy on details): In 2011, Harry Shum was the VP of the Bing division at Microsoft. It was the early days of Bing (~10% market share, ~$2bn annual loss, etc.) - we had good talent, but were lagging behind Google in tech. In one of our all-hands meetings, Harry jokingly announced that if we beat Google in our core relevance metric (called NDCG), he’d take the entire Bing team, approx. 300 people strong, for a fully paid trip to Las Vegas.

Sure enough, a year later, Bing did beat Google in our core relevance metric (http://www.insideris.com/microsoft-bing-beats-google-in-the-...) and all 300 of us went to Vegas for a weekend as promised. (Spoiler: Google did eventually beat Bing back later)

The success and rapid acceleration in relevance gains was attributed in large parts to the introduction of a new tool called AEther (in addition to improving ML tech and hiring top talent). AEther was an experimentation platform for building and running data workflows. It allowed data scientists to build complex workflows and experiment in a massively parallel fashion, while abstracting away all the engineering concerns. I used it a ton on a daily basis and loved it. The AEther team claimed that it increased the experimentation productivity of researchers and engineers by almost 100X. Even now, when I ask ex-Bing data scientists working at other companies about what they miss the most from their time at Microsoft, AEther is almost always in the top 3 answers.

Having seen how awesome AEther was from the inside, one of our goals is to bring its benefits to the rest of the world as well. However, having talked to a few individual data scientists and researchers over the last month, their preference seems to be CLI over GUI (while bigger companies like it much better). May be its one of those things you have to get used to, or may be our implementation is clunky. So we’re making the GUI an enterprise only feature for now, while we continue to help individual data scientists through our CLI.

Haha, there’s no magic and it’s difficult to say with any certainty that we’re going to make it. When we started out, we were just scratching our own itch. The AI community is amazingly open and fast paced. May be because of that, the tooling around it isn’t as mature as for, say, software development. More companies, IMHO, tend to be focused on building the next big thing _with_ AI, rather than the next big thing _for_ AI. With the increasing popularity of the space and the relative infancy of the state of tooling, we believe now might be a good time to tackle this problem.

That said, it’s hard to say how things are going to turn out. This is my first startup after working in the corporate for 6 years and it’s been amazing so far. Learning a lot, and excited for what’s ahead! :)

Thanks for the comment! There's lots of challenges to be solved in this space, and I'm sure there's room for all of us. Excited to see what you guys are up to. I will look forward to your beta release "real soon" :)

Fully agree! In the longer run, I believe there might be some great opportunities wrt infrastructure. GPUs instances are super expensive (now combine with long runtimes). Self hosting infra at scale can drastically drive down prices 10x+ (we're already 2x cheaper than AWS). Till we get there, there’s a few lower hanging fruits to pick, like streamlining development processes and making things easier. Glad you like the Jupyter Notebooks!

Nothing fancy - bought some cheap stock art and my sister, who's a UX designer, modified and brushed them up :)

We picked this as a starting point to assess demands, e.g. CPU vs. GPU, before scaling out. We'll be adding more tiers very soon. What kind of instance would suit your needs?

Sounds good! Give it a spin and let us know what you think. If you want a reference - here's a guide we wrote to run Neural Style Transfer (http://docs.floydhub.com/guides/style_transfer/)!

React + Redux for frontend. react-bootstrap + Bootswatch Paper theme for the UI. This was my first foray into anything web related, so it's good to see positive feedback.

Thanks for the catch. My web development skills are definitely sub-par. Fixed the landing page now, to some extent. Hopefully it's not rendering it unreadable.

Thanks, that's an awesome comment! Agreed. IMHO, end-to-end reproducibility is important from multiple angles - provenance for research, enabling collaboration, driving down costs by eliminating redundant runs of the same jobs, etc.

We were initially calling ourselves Heroku + Github for DL, but realized that was too confusing, haha.

@Jupyter NB, we charge continuously right now. Charging for compute time only is possible, but an interesting engineering challenge (sandboxing, scheduling, etc.) - We’ll take this as a feature request! :) We’re all in the Oregon data center (us-west-2) now.

Thanks - glad you found it useful! The attention and feedback that I got from building dl-docker has been terrific. Definitely one of the reasons we started working on this seriously :)

Yes, we currently run on AWS! The p2.8xlarge has 8 GPUs, not 16 :) So, it would still boil down to $0.9/hr/GPU. Also, utilizing 8 GPUs concurrently is pretty difficult/inefficient for most jobs since benefits from parallelization tapers off quickly.

We are only using p2.xlarge (1 GPU) for now. Driving down the cost is really important to us. We use reserved instances, spot fleets, etc. to be at <50% of AWS pricing. Lots of interesting challenges to be solved there wrt. effective scheduling and fully utilizing resources.

We’ve been thinking about our own infrastructure. It would really drive down the cost, but obviously, comes with its own challenges :)