w01fe

This is incorrect. In the process of producing each token, activations are produced at each layer which are made available to future token production processes via the attention mechanism. The overall depth of computations that use this latent information without passing through output tokens is limited to the depth of the network, but there has been ample evidence that models can do limited "planning" and related capabilities purely in this latent space.

I think this ignores the fact that an agent can be meaningfully embodied in the internet, using APIs for sensors and actuators. OpenAI's training of large language models with reinforcement learning, recent retrieval augmented models and "chain of thought" reasoning are all meaningful steps in this direction in my opinion.

Microsoft (Semantic Machines) | Senior Software Engineer | Berkeley, CA or Boston, MA or Redmond, WA or fully REMOTE | Full Time

The Semantic Machines group is bringing next-generation natural language processing (NLP) technologies to products used by hundreds of millions of people worldwide. You can learn more about how Microsoft is using this technology to create entirely new kinds of user experiences here: http://semanticmachines.com/.

At the core of our platform is a new programming language, designed to support programs structured like human commands which are predicted by machine learning models. This language includes ideas from functional and logic/constraint programming, as well as novel features for meta-computation and introspection. We’re looking for engineers to work alongside product and research teams to help guide the evolution of this platform, including improvements to the core programming language, runtime, constraint system, and tooling.

The ideal candidate should be passionate about designing and evolving programming languages and/or practical formal reasoning systems, supporting users with high quality tools, and working on a rapidly evolving product-driven platform. No experience with machine learning or natural language processing is required – we’d love to work with people who are excited about the promise of these technologies and the opportunity to make them more accessible, regardless of their previous exposure to them.

Learn more: https://www.microsoft.com/en-us/research/group/msai/articles...

Apply: https://careers.microsoft.com/us/en/job/1215447/Senior-Softw...

We put a lot of work into this post to explain some of the deeper problems we are trying to address in our conversational AI framework:

https://www.microsoft.com/en-us/research/group/msai/articles...

At Semantic Machines [0] we rely heavily on probabilistic programming to build state-of-the-art dialogue systems. In particular, we use a library called PNP (probabilistic neural programming) on top of Dynet to allow us to express structured prediction problems in a simple and elegant form. If there are questions I am happy to elaborate to the extent I can. (Also, we are hiring! My email is jwolfe@.)

[0] http://www.semanticmachines.com/

Configurability was a major design goal. Check out the test for examples of how to insert your own leaf generators or wrappers:

https://github.com/Prismatic/schema/blob/master/test/clj/sch...

Does that seem like it will meet your needs?

Author here, would love to hear feedback, and more than happy to answer questions

Author here, would love feedback on this, and also happy to answer any questions.

Yep, that sounds about right. For us, API communication and some global events flow through channels into the app state, where it's distributed to components. For routing we've built up a library around secretary, where we write down a map of uris, handlers, and metadata (can be a modal, etc). Info about the current page and such goes into the app state and that drives the transitions between page components.

We're still figuring out exactly where the boundary lies, but our current approach is much like yours -- view state that's purely local to a component goes in local state, and everything else goes in app state. This feels more natural, but means you lose some of the nice benefits of things like replay/undo for this local state.

For client/server, we've started building up some nice abstractions around our API (like fetching more data in a generic paginated list stored in a cursor), but the write side is still mostly manual. We're trying to use the same API for iOS and web, which probably prevents us from doing some fancier things that continue the same architecture back into the server.

No, you are still free to organize your application state however you wish. For example, our app dynamically pulls in content as you browse from feed to feed and use infinite scroll. It's up to you to decide what to fetch when, and what to cache/discard as the user navigates around.

Engineer @ Prismatic here. I'm happy to answer any questions about the post or our experiences with the rewrite.

Also perhaps interesting for comparison, a Clojure library called Vertigo that emulates C structs with high performance on top of byte buffers:

https://github.com/ztellman/vertigo

I'm really excited about this library -- thanks Leon and Climate for releasing it!

Author here, interested in your comments and happy to answer any questions.

Author here; happy to answer any questions. Feedback and suggestions for improvement are very welcome as well.

I'd have to think about that a bit -- I'm hesitant to create names not provided by the user, but maybe there is another way.

Quite honestly that part of things (turning validation on and off, etc) isn't really done yet -- we have plans to make it much more flexible and powerful, but haven't got there yet.

Sure, we're happy to consider pull requests.

Schemas are already optional -- you can have a s/defn with no, some, or full schematization. You can also put the schemas in the meta if you don't like the :- syntax. I'm not quite sure what you mean by the other point though. My email is in my profile, let's talk :)

I've been really pumped about core.typed since I first heard about it, and I'm excited about the possibilities of combining (or eventually replacing) schema with core.typed.

I think core.typed has come a long way since last time I looked at it (when we started developing schema). Obviously it's a much bigger and more serious effort, and schema was about getting as much bang right away as we could for a few bucks.

I'd love to talk in more detail about how we can collaborate, or at least make sure we play nice together.

Nope, I missed that in the example -- the enum should be in square brackets.

Sorry, that didn't come across the way I wanted -- it certainly wasn't meant as a dig on the expressiveness or power of core.typed, which is a project I'm really excited about.

The main driver for Schema was to make annotating function inputs and outputs as simple and readable as possible. Personally, I find annotations directly on the function arguments easier to parse than separate function type declarations, but I suppose that's a matter of preference.

I think there are a few benefits.

First and foremost, schemas are simple, minimal, easy to read and write, and gracefully extend Clojure's existing type hints. This means that (in my biased opinion) they are significantly better for documentation, which was the primary motivation for developing them.

Second, schemas are data, so it's easy to do more with them beyond documentation. Runtime data validation is one such use, but we can also easily do things like generate core.typed annotations, generate model classes for clients, generate test data, and so on.

Nice! Really looking forward to the release!

You're welcome :) Please let us know if you have ideas for improving it

Thanks for the feedback!

Actually, schema can express arbitrary constraints. Your example translates to schema as:

  {String {(s/required-key "product") [String]
           (s/required-key "type") (s/enum "type1" "type2")
            s/Any s/Any}} ;; allow any other k-v pairs

We don't want write access to your Google account. Unfortunately (at least at the time we set this up) they didn't offer a read-only permissions option for contacts or Reader. We also offer a 'stealth' account with no external services, and you can always choose to attach accounts later once we gain your trust.

Huh, cool! I kinda assumed the JIT already took care of this sort of low-hanging fruit, we'll test this out and if it works include it in the next version of hiphip.

If your goal is more "Clojurey" syntax then just spend a day or two wrapping the functions you want over a tried and tested numerics implementation.

This is exactly what we're trying to do: provide some Clojure macros that give nicer syntax for interacting with Java arrays with high performance. We're explicitly not introducing a new vector type.

Most of the work here wasn't in the wrapping -- hiphip itself consists of very little code -- but in figuring out what's fast and what's not, documenting this, and making it easy to do things the fast way.

We did, and we've been talking to the developers about a potential future collaboration. Our goals are really complementary; hiphip is about getting your code into the inner loop of Java bytecode (not just a set of canned operations), whereas core.matrix is about abstractions for a fixed set of operations across different matrix types. There may eventually be overlap, if core.matrix gets into compiling expressions into new operation types, which sounds like something they're interested in.

Oops, thanks for letting us know! Fixed now.