codeviking

Runway | Multiple Engineering & Research roles | Remote (global) | Full-time

Runway is building AI to simulate the world — merging art and science. We're a team of researchers, engineers, artists, and designers shipping the models and infrastructure behind a new generation of creative tools.

We're hiring across a lot of teams right now (engineering, research, product, design — see the full list below), but a few we'd especially love to talk to you about:

  - Engineering Manager, API — https://jobs.ashbyhq.com/runway-ml/b5d136bb-4143-400b-914a-0ae7377ad8ed
  - MTS, Backend Engineer, API — https://jobs.ashbyhq.com/runway-ml/8489a08a-f0cf-4418-968c-c7c803d538bc
  - MTS, Research Engineer — https://jobs.ashbyhq.com/runway-ml/98e4b160-3bbe-4acd-b106-ad75ccc49675

Full list of open roles: https://runwayml.com/careers

I'm a big fan of lightweight, automated tests. Despite that, I still default to manual verification. Usually I do both.

Automated tests omit a certain type of feedback that I think remains important to the development loop. Automation doesn't care about a poor UX; it only verifies what you tell it to.

For instance, I regularly contribute to a CLI that's widely used at $WORK. I can easily write tests to verify the I/O of a command I'm working on that assert correctness. Yet if I actually try to use the command I'm changing, usually as part of verifying my changes, I tend to discover usability issues that make the program more pleasant to use and the tests would happily ignore.

Also, there's certainly cases where automation isn't worth the cost. Maybe because the resulting tests are complex, or brittle. I've often found UI tests to lie in this category (but maybe I'm doing them wrong).

Because of these things I think manual testing is the right default. Automated tests should also exist; but manual tests should _always_ be part of the process.

Ai2 | Senior Software Engineer | Seattle, WA | ONSITE / HYBRID |

Ai2 (https://allenai.org) is a Seattle based non-profit AI research institute founded in 2014 by the late Paul Allen. We pursue foundational AI research and innovation to deliver real-world impact through large-scale open models, data, robotics, conservation, and beyond.

My team is a group of software engineers redefining how researchers and engineers use state of the art GPU clusters. We own and actively develop Beaker, a GPU-first job orchestration system used by Ai2 researchers to manage and execute frontier research workloads, such as large-scale, distributed pre-training and online reinforcement learning. We’re also responsible for Ai2’s on-premise GPU servers from the bare-metal up, operating a high performance storage cluster and designing and developing critical systems that teams across the institute rely on for pushing forward cutting-edge, open science.

We're looking for a Senior Software Engineer to join our team. You should be proficient in Go and Python and have prior experience operating and configuring linux servers in a professional setting.

https://job-boards.greenhouse.io/thealleninstitute/jobs/6157...

Ai2 | Senior Software Engineer | Seattle, WA | ONSITE / HYBRID |

Ai2 (https://allenai.org) is a Seattle based non-profit AI research institute founded in 2014 by the late Paul Allen. We develop foundational AI research and innovation to deliver real-world impact through large-scale open models, data, robotics, conservation, and beyond.

My team (ReOps) maintains the software and servers that allow research teams at Ai2 to execute machine learning workloads on high performance, SOTA GPU clusters.

Most of our time is spent contributing to Beaker (https://blog.allenai.org/beaker-ed617d5f4593), a GPU-first job orchestration system that was authored at the institute. We also spend a fair amount of time configuring and operating the underlying on-premise GPU servers.

We're looking for a Senior Software Engineer to join our team. You should be proficient in Go and Python and have prior experience operating and configuring linux servers in a professional setting.

https://job-boards.greenhouse.io/thealleninstitute/jobs/6157...

This inspired us to do a little exploration. We used the top cited papers of a few authors to produce a list that might be interesting, and to do some additional analysis. Take a look: https://github.com/allenai/author-explorer

But falling behind is very different than "being done." I think the original tweet is very much an exaggeration, and agree with the point made here.

Google is no where close to "being done." Sure, their answers aren't perfect. But they've managed to deploy them at scale. They're probably available globally. They're fast. And they probably see way more eyeballs than OpenAI's system.

It's going to take a long time for folks to deploy advanced techniques like this at the scale required for something like Google. And if anyone has the resources to do this, it's Google. So I suspect Google will just learn from these examples and integrate them into their existing offering, which will probably eclipse any chance at disruption -- both because of their existing market share and because of the computational firepower they have to make this happen.

Maybe you're right. But I'm not convinced.

I feel like the mass centralization of content is starting to unwind a bit. As things scale the generalized sources usually become less valuable to me. With more content comes more noise, and that noise is hard to sift through. And while Google isn't perfect, they're better at sifting through this noise than most sites are.

Take StackOverflow as an example. When it first emerged I found it really useful. Answers were generally high quality. There were valuable discussions about the merits of one approach versus another. Now it's a sea of duplicate questions, poor answers and meandering discussions. I rarely visit it anymore, as it's rarely helpful. And I regularly have to correct information others glean from it, as it's often wrong or incomplete.

So I suppose this all goes to say that I'm optimistic that things are headed in the right direction. I imagine things will ebb and flow for some time. But I believe Google and other search engines will always have a role to play, as there will always be new, valuable things to discover.

The Allen Institute for Artificial Intelligence | Multiple Research & Software Engineering Roles | REMOTE | https://allenai.org

AI2 is a non-profit research institute founded in 2014 with the mission of conducting high-impact AI research and engineering in service of the common good.

Our headquarters are in Seattle, WA. Employees are free to work remotely or on-site.

We have multiple roles open for both Researchers and Engineers. You can find a full list of open positions here:

https://allenai.org/careers#current-openings-ai2

The Allen Institute for Artificial Intelligence | Multiple Research & Software Engineering Roles | REMOTE | https://allenai.org

AI2 is a non-profit research institute founded in 2014 with the mission of conducting high-impact AI research and engineering in service of the common good.

Our headquarters are in Seattle, WA. Employees are free to work remotely or on-site if preferable.

We have multiple roles open for both Researchers and Engineers. You can find a full list of open positions here:

https://allenai.org/careers#current-openings-ai2

The Allen Institute for Artificial Intelligence | Multiple Research & Software Engineering Roles | REMOTE | https://allenai.org

AI2 is a non-profit research institute founded in 2014 with the mission of conducting high-impact AI research and engineering in service of the common good.

Our headquarters are in Seattle, WA. Employees are free to work remotely or on-site if preferable.

We have multiple roles open for both Researchers and Engineers. You can find a full list of open positions here:

https://allenai.org/careers#current-openings-ai2

Yup, I definitely agree that they're harder (and noted this). But I'm not sure I agree with your second point. Or rather, I think there's some nuance to it.

Sure, using AI to treat people without a human in the loop would clearly do harm. But using AI as an assistant, to help a doctor make the right diagnosis, seems like it'd do the opposite. It'd help doctors serve a larger patient population, make less mistakes, and probably equate to less harm in the long run.

Anyway, I think we can all agree that using AI for anything other than ad targeting is a net win.

Which is why it's important for folks to start applying AI to more interesting (but harder, more nuanced) problems. Instead of making it easier for people to write emails, or targeting ads, it should be used to help doctors, surgeons and scientists.

The problem is that these problems are less profitable. And that the companies with enough compute to train these types of models are concerned about getting more eyeballs, not making the world a better place.

I haven't jumped on the GraphQL train yet, largely for a lot of the reasons the original author calls out. I see the benefits, but they don't outweigh the costs of converting our existing API surface area.

Like most of the tools we choose to use (or not use) there are trade-offs. The original tweet and post fail to recognize why GraphQL might make sense, even with its caveats. GraphQL makes the API more flexible for the front-end to consume. This reduces the number of requests a UI might need to make in order to render something, which makes clients (particularly mobile ones) faster. It also means a team of specialists working on the UI can probably add or adjust features faster, as the backend is more dynamic.

So if you're serving a certain audience (lots of clients where network requests are expensive) or have a large, specialized front-end team that's distinctly separated from the team that's responsible for the API, then GraphQL might be worth the trade offs. Sure, it'll come with some downsides, but all things do -- it's our job to be careful and deliberate about the tools we choose to use.

AI2 | Full Time, Seattle (REMOTE or ONSITE) | Engineering Managers and Software Engineers | https://allenai.org/careers#current-openings

AI2 is a non-profit research institute working to apply AI research and engineering efforts towards the common good. Part of this, of course, involves writing a lot of code.

You might write code for scheduling machine learning experiments across both on-premise and cloud hardware, or work on a platform we have for handling inference at runtime. Or you might contribute to Semantic Scholar, an AI powered, open academic search index or projects like EarthRanger and Skylight, that use technology to combat illegal poaching and fishing activities.

We're a small, open institution with a lot of really smart, motivated people. There's no shortage of fun problems to work on, and folks are given a ton of autonomy to drive and shape their projects.

Take a look at our job offerings, and send me a note if you have any questions:

https://allenai.org/careers#current-openings

sams [at] allenai.org

Yup, that's right.

Yup, this is a known limitation:

What are the limitations? There are several known limitations. Tables are currently extracted from PDFs as images, which are not accessible. Mathematical content is either extracted with low fidelity or not being extracted at all from PDFs. Processing of LaTeX source and PubMed Central XML may lack some of the features implemented for PDF processing. We are working to improve these components, but please let us know if you would like some of these features prioritized over others.

But we intend to fix this!

I agree!

Maybe we'll work on vi bindings next...

We don't. We extract the content and present it as a single document.

Page anchors can be used for navigating between sections. We present a table of contents that makes this easy. For instance:

https://papertohtml.org/paper?id=6f9fc51102cf49bff4f4e2b3367...

Yup, we've tried a lot of different tools in combination. All of them have their own trade-offs and extraction errors.

This system uses GROBID and some extraction techniques of our own. We're working on a GROBID replacement too, which should help us make things better.

That's the idea!

If all goes well we won't need this software anymore. In a best case scenario the publishers start accepting HTML, and gone are the days of having to convert PDFs to something better...!

We don't retain the uploaded document. We cache the extracted content, as to make things more efficient.

See https://papertohtml.org/about:

What data do we keep? We cache a copy of the extracted content as well as the extracted images. This allows us to serve the results more quickly when a user uploads the same file again. We do not retain the uploaded files themselves. Cached content is never served to a user who has not provided the exact same document.

Also, we can delete the extracted data on request. Just send a note to accessibility@semanticscholar.org.

Sorry for the confusion!

Thanks, I'll pass this example along!

Thanks!

There's a lot of amazing people here, doing really great work. It's a really inspiring place to be. I feel really lucky to work with such great people on interesting, important problems.

Also, I should mention...we're hiring!

https://allenai.org/careers#current-openings

Yay, glad to hear it! If you end up viewing one of these on your Kindle, let us know how well (or not) things work.

We're not sure if it's something that we can distribute as OSS just yet. It relies on a few internal libraries that would also need be publicly released, so it's not as simple as adjusting a single repository's visibility.

all of the math and code parts were broken.

Yup, this is a known issue that we're working towards fixing.

But clearly it is a nice idea and I can't wait that such tools work better!

Glad to hear it!

Thanks for the feedback. There's two hard problems n' all that... :)

One comment is that the slowest page to load was the Gallery [0] as it loads an ungodly amount of PNG files from what appears to be a single IP (a GCP Compute instance?)

Yup. There's no CDN or anything like that right now. We kept things simple to get this out the door. But we definitely intend to make improvements like this as we improve the tool.

The more adoption we see, the more it motivates these types of fixes!

P.S. Also, the paper linked below [1] seems to have a few conversion problems -- I see "EQUATION (1): Not extracted; please refer to original document", and also some (formula? Greek?) characters that seem out of place after the words "and the next token is generated by sampling"

Thanks for the catch. As you noted there's still a fair number of extraction errors for us to correct!

Yup, right now we use GROBID, do some post processing and combine the output with other extraction techniques. For instance, we use a model to extract document figures[1], so that we can render them in the resulting HTML document.

Also, we're working hard on a new extraction mechanism that should allow us to replace GROBID [2].

There's a lot of really smart people at AI2 working on this, I'm excited to see the resulting improvements and the cool things (like this) that we build with the results!

[1] https://api.semanticscholar.org/CorpusID:4698432

[2] https://api.semanticscholar.org/CorpusID:235265639

Yup, we're definitely thinking about this.

Our focus right now is on providing a tool folks can run it on whatever papers they have access to. For instance, some researchers might have access to documents that aren't available to the public. We want them to be able to run this against those.

That said as we expand the effort I imagine we'll eventually pre-convert things that are publicly available, like those on ArXiv, etc.

Y'know, that's a good question. I'm not sure I know the answer.

My guess is it's largely for historical reasons. At the time most venues were organized PDF was probably the best (or only) mechanism for sharing documents for print distribution.

But we think it's time to change that :).