Numpy Clone in Common Lisp

https://github.com/numcl/numcl

by lelf • 7 years ago

231 111 7 years ago

github.com

gumby2• 7 years ago

Nice work: not a skin over the python one but an actual, optimizable implementation.

varjag• 7 years ago

I concur, first rate work and good use of the language.

pmiller22• 7 years ago

I think this is great and interesting, but do CL compilers optimize numerical code anywhere near as well as Fortran compilers?

chunsj• 7 years ago

Maybe this will need C based highly optimized backend as well, however, the code looks very clean and beautiful to me. I hope someday differentiable programming becomes possible in common lisp.

darkpuma3• 7 years ago

BLAS/LAPACK is fortran, not C.

stephencanon4• 7 years ago

BLAS is a set of subroutine interfaces that originated in Fortran, but can be implemented in any language; most mainstream implementations use one or more of Fortran, C, C++, assembly, and DSLs.

jlarocco• 7 years ago

SBCL is actually really good at optimizing this type of code, and I think it can even do some vectorization.

If it's not not good enough, there are a few CL bindings and wrappers around BLAS and LAPACK.

A good one is Matlisp, which is available in QuickLisp.

http://quickdocs.org/matlisp/

fiddlerwoaroof2• 7 years ago

Sbcl has all the pieces in place for vectorization, but I don’t think anyone has actually hooked them up. However the great thing about lisps is that the compiler and optimizer are written in the language they compile, so an end user can do this sort of thing themselves: https://www.pvk.ca/Blog/2013/06/05/fresh-in-sbcl-1-dot-1-8-s...

pjmlp3• 7 years ago

Optimizing Lisp compilers are quite good.

Current generations tends to be unaware that once upon a time C compilers only generated turtle code, it was the investment into optimizing compilers and abuse of UB that made current compilers able to generate fast code.

jstimpfle4• 7 years ago

Another C hate post? It seems way overblown. When you compare trivial compilation of C code with optimized compilation, you would see maybe a factor of 5 in difference in code size.

On today's machines, I think you can expect to get more than 50% of performance from trivial compilation, compared to optimized code. Depending on workload you it can be 80-90%. (I have a small amount of experience, based on my own very shitty compiler, to back up that claim).

Even on machines from "once upon a time", where performance was more directly correlated with code size, it couldn't have been that bad, assuming the factor of 5. And also seeing the fact that C was used for systems development from the start.

And that's still ignoring (or at least contradicting) the common lore that "once upon a time", LISP was comparatively more inefficient than C than it is today!

And I don't believe that abuse of UB is important to generate fast code. After all, fast code doesn't necessarily contain UB.

yarrel2• 7 years ago

It's unlikely to need an optimized C backend :-) -

http://www.iaeng.org/IJCS/issues_v32/issue_4/IJCS_32_4_19.pd...

_0ffh• 7 years ago

It's a bit sad that Lush seems to be kind of dead since YLC has moved on...

eggy• 7 years ago

I agree. Lush 2 was very promising. On the bright side, now Lisp is back in what seems a good fit of research and practicality.

jxy2• 7 years ago

IBM doing Common Lisp. I wonder what this is actually used for inside IBM.

Nrsolis• 7 years ago

This might be a clue:

https://developer.ibm.com/recipes/tutorials/watson-iot-with-...

guicho2718285• 7 years ago

Author here. IBM is currently focusing on the combination of machine learning and reasoning, also called neural-symbolic AI [1], to combine the best of both worlds (noise robustness and explainability, etc.). I am from Tokyo and will join MIT-IBM lab soon to perform this line of research.

I have been working on Latplan [AAAI-2018, 2,3,4,5,6], a neural-symbolic classical planning system that learns symbolic representations of noisy images and perform efficient symbolic planning in the latent space. Later, several other groups have started working in this field, like [7].

The problem with python is the lack of sequential execution speed, which is necessary for solving symbolic tasks, e.g. SAT solver, classical planning, theorem prover. At the same time, traditional languages like C++, typically used for writing solvers, are too inflexible to accommodate the workflow in machine learning. For a neural-symbolic task, personally, Lisp is best in this space (I don't try to repeat why, because there are plenty of articles on it.), with Julia being a contender.

I am also interested in adding the compile time type/shape checking on the deep learning code, since python debugging is painful (just a personal opinion -- not intending to start a war here).

By the way, I made the first prototype of this lib in 3 days, and am working on improving it for 3 month (for other ML projects).

[1] https://medium.com/@MITIBMLab/the-shared-history-and-vision-... [2] https://github.com/guicho271828/latplan [3] AAAI18 presentation at : https://asai-research-presentation.github.io/2018-2-5-aaai/ [4] AAAI18 paper : Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary, https://arxiv.org/abs/1705.00154 [5] Follow-up papers (ICAPS2019) : https://arxiv.org/abs/1903.11277 [6] Follow-up papers (ICAPS2019), extension to first order logic : https://arxiv.org/abs/1902.08093 [7] Learning Plannable Representations with Causal InfoGAN, https://arxiv.org/abs/1807.09341v1

kazinator5• 7 years ago

Hi. Are there some openings over there? My entire career has been low-level development in C and C++ (embedded Linux in the last 15 years), with a passion for Lisp on the side (20 years of experience now). Since 2009 I've been developing the TXR language (http://www.nongnu.org/txr), which contains a Lisp dialect called TXR Lisp. My Japanese is okay-ish. I'm not a numerical researcher and have zero background in machine learning. I live in Vancouver, Canada.

jdc• 7 years ago

What are your thoughts on Clasp so far?

_ph_12• 7 years ago

Is there a specific reason you choose LGPL for the library? Are you aware of the issues to use that license with Common Lisp libraries? Common Lisp doesn't use shared library linking as C does, so some terms of the plain LGPL are not directly applicable to Common Lisp, Franz tries to compensate for this with the LLGPL, but it still more difficult to use correctly compared to a more permissive license.

mark_l_watson• 7 years ago

I hope you enjoy your new job! I am also interested in hybrid AI systems. I have been doing symbolic AI in Lisp (and sometimes Prolog) and also neural networks since the 1980s but I have never professionally combined disciplines (but some side of desk hacking at home). Glad to see the MIT-IBM lab peruse this with resources.

noobermin• 7 years ago

Sympathetic, but as long as lisp requires one to write math as function composition it won't catch on with computational types. Of course, I imagine that sort of thing could be a set of macros away...then that would be awesome, familiar infix mathematical expressions with common lisp in the rest of the code.

dreamcompiler2• 7 years ago

I've been doing serious math with common lisp for 30 years. For me, the convenience of having automatic bignums and complex numbers, n-ary functions, and true rationals means I can think about math rather than about the limits of computers. The prefix notation became a non-issue in about a day.

dwheeler• 7 years ago

"Prefix notation is a non-issue" is true for a tiny minority of software developers and mathematicians. The world has already spoken: infix is a minimum requirement for the vast majority of people doing math.

Bignums, true rationals, and many other constructs are already available in Python. Numpy is too. Most importantly, infix is available in Python.

The funny thing is that it's already possible to do infix in Common Lisp. My "readable Lisp" approach has been supporting infix in standard Common Lisp: https://readable.sourceforge.io/ It's already available in the QuickLisp libary as "readable". You can then use {1 + 2} with macros, quasiquoting, and so on.

reikonomusha• 7 years ago

You can use CMU-INFIX: https://github.com/rigetti/cmu-infix

linguae• 7 years ago

I'm not 100% sure about this. A counterexample is the popularity of RPN. HP's RPN calculators with postfix notation are well-loved among some engineers, scientists, and accountants, precisely because of the benefits of RPN over infix notation. I have a HP-48SX calculator, and I regularly use the `dc` command on Unix machines and in WSL whenever I need access to a calculator while I'm on a computer. Of course, I am proficient in prefix, infix, and postfix notation, but I like postfix notation the most.

Of course, though, the marketshare of TI's infix calculators is much higher than HP's RPN calculators, and I haven't heard of much recent developments regarding HP's calculator line (the last I heard was HP had a limited-edition re-release of the famous HP 15c model sometime back in 2011, and also earlier this decade HP released a graphing calculator with a color display). But nevertheless there are people who prefer RPN to infix.

darkpuma3• 7 years ago

Those TI calculators are real cool, I cut my teeth as a programmer on a TI-83+ learning z80 assembly, but the truth is they're considered by many to be devices for teenagers. HP calculators seem far more popular in industry.

mcabbott• 7 years ago

The thing about RPN is that it worked well as a sort-of "spoken not written" language. It's a clear and usable way to give instructions one-at-a-time to a calculator with a 1-line display.

But written out on paper, it's very hard to see what's going on at a glance, which is why nobody does this on paper.

Almost anyone programming in mathematical expressions of reasonable complexity today will spend much more time reading (and triple-checking!) these expressions than typing them. That's a strong reason to like infix notation, and in general notation close to what you would use on paper.

lgessler3• 7 years ago

What exactly do you mean by function composition? If you mean you can't have long chunks of purely imperative code in CL, I'm pretty sure that's not true: CL is multi-paradigm and supports imperative style. I think you might be referring to its S-expression syntax, which is prefix?

ken• 7 years ago

Have you seen numeric code (like SIMD intrinsics, BLAS, or LAPACK) in C or Fortran? It's all prefix function calls, anyway, even in those languages.

kyllo• 7 years ago

Right on, now we're only arguing about whether the open paren goes before or after the function name :)

vnorilo4• 7 years ago

I'm not saying you are wrong, but the oft-cited challenge of prefix is a meme. It's not really a big deal. It's trivially easy to add infix syntax to lisps via macros, but almost no one bothers. By the time you know enough to do it, you're also already used to prefix.

agambrahma• 7 years ago

Good to see someone fill in this missing link! Could become a viable competitor to Julia (uniform language stack, as opposed to the Python+C combination currently being widely used).

longemen3000• 7 years ago

A funny situation, the Julia parser is femtolisp, so in a funny way, Julia is already a kind of numpy in lisp

ddragon2• 7 years ago

I think it's more than a uniform language stack, after all C++ could be considered one as well but most people still use python to interface it as opposed to just using it (at least for exploratory data science). It's a combination of interactivity to experiment, conciseness to stay close to the pseudo-code/math, easy hackability since you need to stay beyond the cutting edge and performance so you're not restricted in what you can do.

Common Lisp (as does Clojure and Julia to a lesser extent) has unparalleled interactivity (besides maybe smalltalk), you can make it as concise as you want, ultimate hackability and really good performance. Research (not deployment) is the one area CL could probably surpass every other, but unfortunately it lacked a concentrated effort (by a company or community) in this current AI resurgence (maybe because of the Lisp Curse). Though there will always be other opportunities.

pmiller22• 7 years ago

From the point of view of a numpy user, the stack is 100% Python. You just import it and don’t worry about it.

zmmmmm• 7 years ago

Wonder how it compares to using something like ND4J with Clojure? That would (as I understand it) have the advantage of not sacrificing as much speed.

fartcannon• 7 years ago

This is an excellent project. Is there a developer blog? I'd like to try reimplementing numpy I'm various other languages (or part of it, since it seems like it's a huge task).

enriquto3• 7 years ago

You can re-implement a much better interface that numpy if your purpose is doing math comfortably. Numpy has a wonky interface to ease the transition from matlab/octave (while amazingly producing a much less comfortable system).

nemoniac• 7 years ago

Could you point to a "much better interface" to numerics?

nurettin• 7 years ago

I've used numpy without much trouble, but matplotlib really shows how creative and non-intuitive interfaces can get when developers are left to experiment with function and method parameters in a dynamic language.

Numpy Clone in Common Lisp

Related Stories