PyPy.js: A fast, compliant Python implementation for the web

461 134 11 years ago

raymondh• 11 years ago

This is pretty cool.

Besides the big download size, it does several things very "right": Full Python data model, access to the full javascript model (albeit slowly currently).

Also PyPy is the right choice as the basis for this (as opposed to CPython) because its ecosystem is generally more "pure" Python than CPython's. C Extensions would only get in the way.

fijal5• 11 years ago

it's systematically translating the PyPy interpreter to the Javascript, so it has to get semantics right. However I'm not sure if the drawbacks can be addressed in the current JS model, notably: very slow warmup, huge download size (that's primarily PyPy fault though), the lack of good JS access.

jychang• 11 years ago

However I'm not sure if the drawbacks can be addressed in the current JS model, notably: very slow warmup, huge download size

Only for the first time loading it.

It would be great if some big organization (looks at Google) hosts the "standard" version of the file, much like jquery, so that it would be cached and ready to be used on the local machine most of the time.

ggchappell• 11 years ago

... access to the full javascript model ....

How is the JS model accessed?

mkesper• 11 years ago

See nosir's comment.

jedberg2• 11 years ago

So does this mean that since I'm better at Python than Javascript, I can now use this to make my web frontends in Python? Because that would be pretty sweet.

riffraff• 11 years ago

weren't you already able to do it with pyjamas/pyjs?

jedberg2• 11 years ago

Quite possibly. To be honest I never looked into it.

raverbashing• 11 years ago

No, no, no!!!

Pyjamas was based on GWT, right? So, a Java Framework translated to Python. It's as nice as running a pack of deer on a road with a truck.

Awful

pyvpx• 11 years ago

I had the same thought. "this is cool. but what should I do with it?"

alexchamberlain• 11 years ago

Embed it in browsers as a migration path from JS to Python

strange_quark• 11 years ago

That would be great as one of the things that puts me off of front end development is lack of language choice. I adore Python as a language and it would certainly make a great choice for a front end language if supported. Maybe if this project takes off, browser vendors would ship a Python interpreter alongside the JS one. I'd bet that Apple, Google, Mozilla, etc. could make Python fast if they threw as much money at it as they do their JS interpreters. And PyPy is already pretty fast -- it's certainly way faster than JS engines were before V8 and JavaScriptCore hit the market.

This will probably never happen, but I can dream.

sgibat• 11 years ago

have you looked into CoffeeScript? it's slightly more pythonic than JavaScript.

est• 11 years ago

Yo know the most troubling aspect of browse side scripting is the DOM, js syntax has its issues but not the biggest issue.

brachi• 11 years ago

I watched the presentation in Pycon 2015[1], and I really liked how honest/aware they are about the challenges and tradeoffs (so far, the performance it's pretty bad compared to javascript, and the download size is still an issue). At the end is answered why python can't be directly included in Firefox or other browsers(the presenter/main developer works for Mozilla)

[1]https://www.youtube.com/watch?v=PiBfOFqDIAI

voltagex_• 11 years ago

I wonder if Python could be embedded as an addon.

RussianCow2• 11 years ago

Of course it could. But that would defeat the point, because support wouldn't be widespread enough for any real use (unless you're just using it for an intranet site or something).

vog2• 11 years ago

python can't be directly included in Firefox or other browsers

So Python is a bad candidate, but there are better candidates out there. For example, jsofocaml (using OCaml in the browser) looks quite promising.

In the end, we need a lightweight yet properly defined and highlevel language, and the ML languages as well as the LISP languages are great candidates for that.

maximilian• 11 years ago

Purescript is a Haskell-like language that is designed to output to javascript. It looks quite nice and has seen a lot of attention lately. It's even in GSoC.

stock_toaster• 11 years ago

In the end, we need a lightweight yet properly defined and highlevel language....

Even lua would be a joy!

Macha• 11 years ago

I believe the issue is politics. I.e. browser vendor A (any vendor) can't unilaterally introduce a programming language and expect others to follow. (MS tried with VBScript, Google gave up on doing so with Dart, for example)

This applies equally to any language.

vog2• 11 years ago

Not sure why this was downvoted. Does anyone care to elaborate?

nailer• 11 years ago

At the end is answered why python can't be directly included in Firefox or other browsers(the presenter/main developer works for Mozilla)

Direct link to answer: https://youtu.be/PiBfOFqDIAI?t=1611

jdnier• 11 years ago

"If we ship Python inside of Firefox, no one is going to use it because it's not available in Chrome, it's not available in Internet Explorer." Oh yeah? Try me.

itsnotlupus• 11 years ago

For historical context, early IE browsers weren't hardcoded to use JavaScript. We know about VBScript, but really there was a generic framework called Active Scripting that would allow any other language to be plugged in.

Python was one of those languages. You could download an ActivePython distribution and start being able to run python in your web page with a <script language="PythonScript"> tag.

I'm not sure if this still works in current IE versions. It might.

btown• 11 years ago

This is glorious. From https://github.com/rfk/pypyjs/blob/master/CONTRIBUTING.rst :

We have the following major components in the PyPy repo:
> An "emscripten" build platform definition, which teaches pypy's rpython toolchain how to compile things with emscripten: ./deps/pypy/rpython/translator/platform/emscripten_platform/.
> An rpython JIT backend that emits asmjs at runtime: ./deps/pypy/rpython/jit/backend/asmjs/.
> A "js" builtin module for the resulting interperter, to allow interaction with the host javascript environment: ./deps/pypy/pypy/module/js/.
Along with these wrappers to help working with the resulting interpreter:
> A wrapper to load up the compiled VM and expose it via a nice javascript API: ./lib/pypy.js.
> A script for bundling python modules into an indexed format that can be easily loaded into the browser: ./tools/module_bundler.py.

In https://github.com/rfk/pypy/blob/master/rpython/jit/backend/... alone, there are 2887 lines of custom Python code to emit asm.js for function and block JITing.

And at 3.26 seconds to initialize from cache on a 4-year-old machine, 5MB gzipped (not too much worse than your average high-quality gif) and only 1.5x slower than CPython... this is actually approaching viable if you needed to run Python business logic in the browser. Which I never thought I would see.

mutagen• 11 years ago

Glorious indeed. At least three levels of abstraction, two JITs, and a host of supporting code. Can we add some more layers like an emulator or get recursive by getting something like Pyasm running inside it?

madeofpalk• 11 years ago

Have you seen Gary Bernhardt's THE BIRTH & DEATH OF JAVASCRIPT?

https://www.destroyallsoftware.com/talks/the-birth-and-death...

nudpiedo• 11 years ago

I work on a machine that has two hypervisors, one third in plan, and more abstraction layers than a lasagna software.

It is still very very efficient.

EpicEng2• 11 years ago

But... Hardware is cheap, right? Hardware gets faster all the time, right? Sure, the end user rarely sees any noticeable improvement because we just use the speed increase as an excuse to write ever more inefficient code, but at least you get to use your favorite language in the browser.

StavrosK3• 11 years ago

Since PyPy is written in (R)Python, you can just run PyPy inside the resulting interpreter, for a 1000x slowdown.

callahad2• 11 years ago

Potentially just one JIT, right? asm.js is AOT compiled on Firefox.

pjc50• 11 years ago

It's compilers all the way down.

throwaway415973• 11 years ago

Since PyPy compiles RPython to C, couldn't one write a web app in RPython, compile it to C, then to JS (with asm.js) and enjoy a quick page load and smaller file size?

By the way, is RPython a nice language to program in?

icebraining• 11 years ago

The differences are described here: https://rpython.readthedocs.org/en/latest/rpython.html

Veedrac3• 11 years ago

is RPython a nice language to program in?

Not really, if only for the insane compile times. The PyPy authors also heavily discourage using it except for interpreters.

masklinn• 11 years ago

By the way, is RPython a nice language to program in?

It's a nice language to develop VMs in, it's a terrible language to do general-purpose programming in.

throwaway415973• 11 years ago

correction: the C to JS step would be done with Emscripten (asm.js is a subset of JS not a compiler)

Thank you for the answers

aidanhs• 11 years ago

A little frustrating that github's down, as I'd be interested to see their build process - with some trickery I was able to get empython [0] down to 2.5MB gzipped from 7MB, though I'm aware pypy as a binary is quite a bit larger anyway.

I also wish github was up so I could look at the benchmark code - startup looks much faster with empython, but I'd like to know how much slower interpreting would be.

[0] http://aidanhs.github.io/empython/

nighthawk454• 11 years ago

Sadly, "import antigravity" doesn't work

reuven• 11 years ago

Yes, but "import this" does!

(However, it took about 10 seconds to execute. Maybe that's the network I'm on, but even so...)

e12e4• 11 years ago

I was more disappointed (but not surprised!) that "import pip" didn't work.

If it did (and it has big ramifications for how much of "python" (for some definition of "python" that includes most of the standard library) works. Eg sockets, interacting with some sort of "local" "file-system" etc) one could (this is handy if working on windows, and running python via cmd-r python.exe):

  import pip
  # pip.main expects a list of arguments, hence .split() 
  pip.main("install ipython".split())
     # Only needed first
     # time, as it installs ipython under site-packages.
     # Only works if the user has write access, obviously

  import IPython
  IPython.start_ipython()

toyg4• 11 years ago

That would be a total security nightmare and hopefully it's never going to happen.

I'd be happy enough with serving .py files with the exact same restrictions and privileges that JS libraries have now.

retrogradeorbit• 11 years ago

6.6 Megs of binary blob download (pypy.vm.js.mem). Plus 593K of index.json. Yuck.

Curious. Interesting tech. But hardly practical.

nosir33• 11 years ago

Sample code to execute some Javascript code inside PyPy.js. I couldn't find any examples.

    import js
    js.eval("console.log(\"hi!\")")

dag11• 11 years ago

Or this:

    import js
    js.globals.console.log('hi!')

catlover99• 11 years ago

I'm loving this, my head is reeling with the glint of possibilities

    import js
    js.globals.console.log('hi!'*5)

fragmede6• 11 years ago

It's a bit messy, but:

    js.eval('var div = document.createElement("div"); div.style.width = "100px"; div.style.height = "100px"; div.style.background = "red"; div.style.color = "white"; div.innerHTML = "Hello"; document.body.appendChild(div);');

manages to append a div.

Directly modifying document.body.innerHTML in any way seems to hang it. Both:

    js.eval('document.body.innerHTML += "<div></div>";');
    js.globals.document.body.innerHTML += "<div></div>"

Cause it to hang, though the javascript console shows it was added (or using an img tag instead).

(ChromeOS v43 beta)

batbomb• 11 years ago

It's more fun doing this, since jQuery is already on the page:

    import js
    jq = js.eval("$")
    first_line = jq(jq(".container p")[0])
    first_line.text("jQuery was here")

Retr0spectrum2• 11 years ago

Just playing around:

    import js
    c = js.globals.document.createElement("canvas")
    js.globals.document.body.appendChild(c)
    ctx = c.getContext("2d")
    ctx.fillRect(10, 10, 10, 10)

Edit: Here is a simple animation:

    import js
    import math
    import time
    c = js.globals.document.createElement("canvas")
    js.globals.document.body.appendChild(c)
    ctx = c.getContext("2d")
    
    def render():
      ctx.clearRect(0, 0, c.width, c.height)
      t = time.time()
      ctx.fillRect(20+math.sin(t*10)*10, 20+math.cos(t*10)*10, 10, 10)
    
    js.globals.window.setInterval(render, 20)

Here's a version you can paste in properly: http://pastebin.com/raw.php?i=67GadSUV

fragmede6• 11 years ago

Sadly, if I paste that in, I get:

    >>> import js 
    ... c = js.globals.document.createElement("canvas") 
    ... 
    debug: OperationError:
    debug:  operror-type: SyntaxError
    debug:  operror-value: ('EOL while scanning string literal', ('c callback', 1, 12, "r = c.push('import js\n", 0))
    debug: OperationError:
    debug:  operror-type: KeyError
    debug:  operror-value: 'r'

And then the input window hangs. Strangely enough, if I type it in instead, it's okay.

(ChromeOS v43 beta-channel)

stuaxo• 11 years ago

It would be good if they could support a common api with brython, which has "import web" and some other bits...

I guess these things will take a while to shake out, but I'd imagine a python-in-browser-api PEP at some point.

estomagordo• 11 years ago

Unfortunately, this is of little use to me. Why? Well, with my Swedish keyboard layout, some alt-combinations are crucial (mainly square brackets, []). And these keypresses do not seem to work.

Veedrac3• 11 years ago

You can try `vm.eval` from the JS console.

acron0• 11 years ago

Obligatory link to Atwood's Law: http://blog.codinghorror.com/the-principle-of-least-power/

Although I appreciate that this isn't technically "written in JavaScript" but it follows the principle.

morekozhambu• 11 years ago

This is super awesome.

dlitz• 11 years ago

Is there a Python 3 version of this?

venti2• 11 years ago

Yes, in his talk at https://www.youtube.com/watch?v=PiBfOFqDIAI&t=1355 Ryan Kelly already showed a Python 3 version of pypy.js

techdragon2• 11 years ago

I was talking with some contributors at PyCon last month and there was a consensus that Python 3 is actually preferable. There's issues to be resolved that are more important than switching the version to 3 so the work is focused on these for now.

It boils down to the argument that "if there is no existing Python code relying on Python 2 semantics, why even bother making a Python 2 version, why not skip straight to Python 3 and not introduce the switching problem later?" There's a small matter of the Python 3 version not being the prime target of PyPy but they are updating it and progressing it forward so it's not any kind of roadblock just a little less than perfect.

bootload4• 11 years ago

If software is eating the world, JavaScript is eating the languages.

zo12• 11 years ago

Only because the browser vendors are too scared (for whatever reason, legitimate or otherwise) to add extra languages to the mix, like python.

bootload4• 11 years ago

"vendors are too scared (for whatever reason, legitimate or otherwise) to add extra languages to the mix, like python."

Tried python server-side, mod_python and got lots of issues with formatting. Not syntax or mixed tab/spaces but one-off spacing errors. I realised then, Python used this way has problems. Could be fixed though.

nirai• 11 years ago

Doesn't seem to work well on chrome on a modern tablet.

henryaj• 11 years ago

Please humour my lack of knowledge: is there much difference between something like this and Opal[0]?

[0] http://opalrb.org

Veedrac3• 11 years ago

PyPy.js is not a transpiler, it is a JITted interpreter. This has correctness advantages since the major work is upfront, and potential throughput advantages, but is results in huge upfront download and warmup costs.

acbart• 11 years ago

I use Skulpt for a lot of my research - I'd love to see a comparison against it in terms of speed, accuracy, etc.

xiaq• 11 years ago

The next level is to run a Python implementation of JavaScript inside that...

gdw22• 11 years ago

Can the client-side python access the DOM?

venti2• 11 years ago

yes it can: import js js.globals.document.getDocumentById("...")

Nitramp• 11 years ago

I think there is no point in trying to emulate different language semantics on top of JavaScript.

asm.js is a weird hack that lives in a totally different world than actual browser APIs (e.g. the DOM), and you have to emulate your entire runtime and environment to get anything useful, which means your binary is going to be huge (as in this example, but similar things apply).

Emulating better semantics than JS based on JS usually leaves you in this uncanny valley where you either trade off performance and size for nicer semantics (e.g. killing the null/undefined dichotomy, or 64 bit integer math), or you end up with odd semantics that don't quite fit the language you're compiling from, or a mix of both, which is a usability disaster.

So the only real chance to improve on the state of JS is being a syntactical shim on top of JS and help users with better syntax, static analysis, and compile time tooling. That'd be TypeScript.

fragmede6• 11 years ago

That'd be TypeScript.

Or CoffeeScript. Or Dart. Or this. In fact, here's a whole big long list of other languages. Most of which don't use the asm.js "weird hack".

https://github.com/jashkenas/coffeescript/wiki/List-of-langu...

alexweber• 11 years ago

one weird hack to run assembler-like code in the browser!

developers hate him!

Tossrock2• 11 years ago

Looks like there are some issues to iron out...

  >>> from datetime import datetime
  >>> datetime.now()
  Traceback (most recent call last):
    File "<console>", line 1, in <module>
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1548, in now
      return cls.fromtimestamp(t, tz)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1522, in fromtimestamp
      result = cls(y, m, d, hh, mm, ss, us, tz)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1459, in __new__
      hour, minute, second, microsecond)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 312, in _check_time_fields
      raise ValueError('microsecond must be in 0..999999', microsecond)
  ValueError: ('microsecond must be in 0..999999', 1716000)

caryhartline• 11 years ago

That works now.

Tossrock2• 11 years ago

Does it?

  >>> from datetime import datetime
  >>> datetime.now()
  Traceback (most recent call last):
    File "<console>", line 1, in <module>
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1548, in now
      return cls.fromtimestamp(t, tz)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 1511, in fromtimestamp
      us = _round(frac * 1e6)
    File "/lib/pypyjs/lib_pypy/datetime.py", line 28, in _round
      return int(_math.floor(x + 0.5) if x >= 0.0 else _math.ceil(x - 0.5))
  ValueError: cannot convert float NaN to integer

MBlume• 11 years ago

I'm curious -- since the custom emitter is tied to the RPython JIT and not to the Python interpreter, could one easily plug in the existing Ruby or PHP interpreters written in RPython and get similar performance?

pc2g4d• 11 years ago

Not working for me:

"Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://10.30.229.27:15871/cgi-bin/blockpage.cgi?ws-session=1.... This can be fixed by moving the resource to the same domain or enabling CORS."

Firefox 37.0.1

hurin• 11 years ago

>> def foo(x,y):

... return x,y

debug: OperationError:

debug: operror-type: SyntaxError

debug: operror-value: ('EOL while scanning string literal', ('c callback', 1, 12, "r = c.push('def foo(x,y):\n", 0))

>>

forgottenuser• 11 years ago

How is this different than Brython or Skulpt?

Also python implementations that run in the browser.

[1] skulpt.org [2] brython.info

dulvac• 11 years ago

I can imagine a way to include language extensions in the browser and have them loaded up lazily in a sandbox. And a checkbox that says "Allow websites to run code" or something. Then you could develop an extension for your favourite language and either install them manually or have some popular ones bundled with the browser.

If we're going to run everything in the browser, why use javascript as the low-level language that stuff compiles to?

I know portability and existing standards are a limitation, but those can be influenced by the 3 (?) major browsers if needed.

I think asm.js is an awesome cute little thing, but the fact that it's taken seriously is worrisome to me.

Am I completely crazy here?

PyPy.js: A fast, compliant Python implementation for the web

Related Stories