Use JSON files as if they are Python modules

https://github.com/kragniz/json-sempai/tree/master

by bane • 10 years ago

267 65 10 years ago

github.com

hobarrera• 10 years ago

Disclaimer: Only do this if you hate yourself and the rest of the world.

I'm curious why the author would do this. Just for the fun in it?

jzelinskie• 10 years ago

Likely for the fun and it might just save time when writing one off scripts or using the interpreter.

I nearly spit my drink out at the name, though.

kragniz2• 10 years ago

Yes, it was just for fun.

karolmajta• 10 years ago

Fun is the only thing that can justify this blasphemy :)

Why do this instead of:

   myvar = read_json_file_as_object('path/to/file')

Ok, you can argue that function would hit the hard drive many times when called from multiple modules (while imports would not), but this can be fixed with simple memoization.

moonpompey• 10 years ago

Ah. kragniz must be the Damien Conway of Python.

nemothekid• 10 years ago

Really love the fact the import is named `from jsonsempai import magic`.

If this were to be used seriously, it would make sense to other devs that the import named magic is doing some crazy stuff.

BinaryIdiot2• 10 years ago

As a JavaScript developer for the past 5 years or so, when jumping into Python I certainly miss the simple require(<json file>) which this seems to replicate into Python pretty well!

Having said that I don't think that I would use it for serious things since this isn't really the Python way and I would like my code to be most understood by others. Neat though!

aikah• 10 years ago

JSON stands for JavaScript Object Notation , not Python Object Notation. Furthermore "require" is "proprietary" to nodejs, ES2015 has a different module system and it will be the official js module system.

BinaryIdiot2• 10 years ago

Yeah, and? I'm not sure what you were trying to get at but I never said anything to the contrary to this and I mentioned I've done lots of JavaScript dev so this isn't anything new.

russjr08• 10 years ago

JSON stands for JavaScript Object Notation , not Python Object Notation.

That does not mean it shouldn't be used anywhere else.

alexwebb2• 10 years ago

The require() function was actually part of the old CommonJS standard: http://wiki.commonjs.org/wiki/Modules/1.1

node was originally an implementation of it before eclipsing it. Between the original advocacy by CommonJS and the rising popularity of node, browserify, and Webpack, things like require() leaked out to become the pre-ES2015 de facto standard for importing modules.

require() is so widespread now, and the transform to the ES2015 syntax so trivial, that it's not going away anytime soon.

arthurcolle• 10 years ago

Why do all the comments act like this is bad? Seems fine to me. Explanation would be appreciated :)

ben336• 10 years ago

You're changing the behavior of import. So if somebody else were to remove the seemingly unused `from jsonsempai import magic` the file would stop working as expected.

Non obvious "magic" code like that tends to be frowned on in general, and especially in the python community

fbukevin• 10 years ago

Is it be useful? Or there are some alternatives to import JSON as a module as easy as this one does?

onnoonno• 10 years ago

Cython does the same thing for ".pyx" files and I have to say it is very handy and works well.

ngoldbaum• 10 years ago

Because it monkey patches the way imports work. The context manager isn't bad really, although there are probably more natural ways to create a python object with the same structure as a JSON file without abusing the import system.

pmiller2• 10 years ago

Right. JSON is almost valid Python to begin with. Off the top of my head, the main differences are that JSON true needs to be True in Python (likewise with false/False, null/None). It's not that difficult.

jstoiko• 10 years ago

My favorite is this issue: https://github.com/kragniz/json-sempai/issues/7

Ever since I saw this I have been unable to sleep. Please fix.

sarciszewski• 10 years ago

Works as intended. adds the 'wontfix' label

This is gold.

cjslep• 10 years ago

The 'wontfix' label looks more like a yellow than a gold to me.

PaulHoule• 10 years ago

... and I was writing SPARQL queries against my Java source code yesterday

hyperion2010• 10 years ago

IIRC there is a moment of sheer terror in this talk by David Beazley [0] where he shows how to import an xml file as a python module.

0. https://www.youtube.com/watch?v=0oTh1CXRaQ0

latenightcoding2• 10 years ago

I did the same with Perl in 10 minutes: https://git.io/vzKDs

it was fun I will check it tomorrow when I'm back from the pub

patcon• 10 years ago

A man from Toronto writing a 10-minute perl module from a pub? This sounds like a Wheat Sheaf sort of affair.

viraptor• 10 years ago

Is there any practical difference between that and just a normal new() function returning a hash based on that json? Does bless change anything?

I mean, for "the same" I'd expect something like:

    ....
    use test
    say test->hello;
    say test->this->could->be->a->bad;

latenightcoding2• 10 years ago

Ok, back from the pub and fixed it you can now use it like this:

use BlessJSON qw(test.json);

say test->hello;

say test->this->could->be->a->bad;

close enough

jamesdutc2• 10 years ago

The mechanism is called "import hooks":

https://docs.python.org/3/reference/import.html#import-hooks

https://www.python.org/dev/peps/pep-0302/

You can see the import hook added here:

https://github.com/kragniz/json-sempai/blob/master/jsonsempa...

Import hooks are a great feature in Python.

Take the `import` mechanism in the language and reduce it to its barest theoretical formulation - what does it really do? (Name binding.) Think of all the other features in your application that can be reduced to this. (e.g., configuration management) Use the constraints of the import mechanism to guide the design of this feature, and use import hooks to implement it. You'll end up with an implementation that is very "close" to the core language. I would assert that this closeness strongly suggests correctness and composability.

Less philosophically, think of the Python virtual machine as a system with lots of "safety valves" and "escape hatches." Import hooks are one such safety valve. Think about all the things you could do easily by hooking into module importing? (Real use-cases: loading code from bitemporal object databases, deprecation of modules, relocation of modules, configuration management, &c.)

Of course, there are languages that are much more flexible than Python in this respect. Python aims for a practical flexibility, and I find that, in practice, Python strikes a nice balance.

jamesdutc2• 10 years ago

A simple example of how import hooks use for module relocation/deprecation:

https://gist.github.com/dutc/0f7498451d98e3114268

orf• 10 years ago

Import hooks are awesome, one of the cool things you can do is re-write the code as you load it. I made a toy loader that automagically inlines (some) Python function calls[1]. There are also cool projects like MacroPy[2] that do much more extensive things.

1. https://tomforb.es/automatically-inline-python-function-call...

2. https://github.com/lihaoyi/macropy

finyeates• 10 years ago

Awesome, I saw your post a while back and it triggered me to write a full inliner for python, also added some IFDEF macros using context managers e.g.

with IFDEF("DEBUG"):

I'll have to dig around and find my code, your inline project was a great help in understanding the import hooks and walking the ast.

dr_zoidberg• 10 years ago

I read your experiment on inlining and it looks quite interesting. Did you do a benchmark on how it affects (improves?) performance? In my line of work I've found ocasionally places where inlining would've helped (functions that do masking with 64 bit masks, for example, are a mess to "inline by hand" and kill a lot of readability and clarity). Even with the current limitations of your "toy" implementation, it seems like it would help to avoid the costly function calls.

jimbokun• 10 years ago

Sounds a bit like the Ruby-fication of Python.

inglor• 10 years ago

For what it's worth - modern JavaScript does it too, it separates the syntax (import/export) and semantics of binding the reference from the actual loading mechanism. Not only can you "hook" into imports, you can replace them altogether if you'd like.

Grue3• 10 years ago

This seems like a huge case of "implicit over explicit". Ideally if you import package foo, it shouldn't affect anything but foo.bar

icebraining2• 10 years ago

Python itself never enforced that, though. It's the libraries' responsibility to have a "register_import_hook()" function or similar instead of just doing so unannounced.

bdarnell• 10 years ago

The "loader protocol" (PEP 302) is a related trick that can be used locally, so it only affects imports from packages that opt in. I use this in https://github.com/bdarnell/codegenloader to automagically process protobuf/thrift files and import the generated code.

_xhok• 10 years ago

Some people are saying JSON ≠ Python Object Notation. Here we go! https://github.com/brhs/pon

(I used OP's work as a reference. Amazing you can do this in so few lines of code.)

w_t_payne• 10 years ago

It's official. PON is now a thing.

vog• 10 years ago

From the examples:

    >>> from jsonsempai import magic
    >>> from python_package import file
    >>> file
    <module 'python_package.file' from 'python_package/file.json'>

I believe it is not a good idea to teach people to import something named "file", as that overrides Python's builtin class "file".

(On the other hand, that class is usually instantiated via calls to "open", so the class name "file" is unused is most programs dealing with files.)

aleri• 10 years ago

Also, 'file' doesn't exist in Python 3.

kragniz2• 10 years ago

Feel free to send a pull request changing it to a name that makes things clearer. Calling it 'file' probably isn't the best idea, but I think it makes it clear you're loading from some arbitrary json file.

kelvin0• 10 years ago

Awesome, although I don't quite have a use for it right now. Next we could: * Import HTML/CSS/JS/DOM via a URL: https://news.ycombinator.com

* Import XML? * Import any_structured_data!

amelius• 10 years ago

The enormous downside to this approach is that once you want to load a json file dynamically, e.g. with a filename that comes from a dynamic string, you're stuck and you have to come up with a completely different approach and rewrite your code.

njharman• 10 years ago

Not correct.

Like many things in Python import is actually just syntactic sugar for a dunder function.

https://docs.python.org/3/library/functions.html#__import__

jm666• 10 years ago

These jokes are getting rather sophisticated. For a few (head-scratching) minutes, I thought this was meant seriously.

StavrosK3• 10 years ago

Excepting the cringemarsh that is "import <jsonfile>", this is actually very useful. This looks very unpythonic:

my_json["this"]["can"]["be"] == "nested"

and is impossible to write defensively and cleanly after the first call:

my_json.get("this", {}).get("no way to not crash here")

Something that could do:

my_json.this.can.be == "nested"

With a default value would be very useful. Does anyone know something like it, or should I whip a module up in an hour?

dadrian• 10 years ago

I've taken a crack at something similar, I'd love to see what you came up with. I ended up with this super-meh solution: https://github.com/zmap/ztag/blob/master/ztag/transform.py#L...

StavrosK3• 10 years ago

What I had in mind sounds pretty close to yours, really. I was thinking of adding a default somewhere along the line (possibly the end), so you could do:

json.structure.that.i.want.to.access.default(True)

Or something like that. I don't really like the default item API I've come up... I'll comment here if I do give it a go, thanks for the snippet!

StavrosK3• 10 years ago

My thing turned out to be pretty much the same as yours, also equally meh:

https://github.com/skorokithakis/jsane

There's no code there, but the underlying class is pretty much spot on yours. I'm still trying to improve the API, but it's not looking great.

zippoxer• 10 years ago

Have you ever been kept awake at night, desperately feeling a burning desire to do nothing else but directly import JSON files as if they were python modules

I almost died laughing at this. Feels like the author is aware that this scenario is probably the only one where such library can be mission critical.

erikb• 10 years ago

Funny cause it's true. It's really something I desperately seeked a few years ago

jason_s• 10 years ago

... ??? why? What's wrong with json.loads()?

bsg75• 10 years ago

I'm not sure which is the better/worse idea:

"JSONx is an IBM® standard format to represent JSON as XML. The appliance converts JSON messages that are specified as JSON message type to JSONx. The appliance provides a style sheet that you can use to convert JSONx to JSON."

https://www-01.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/...

Use JSON files as if they are Python modules

Related Stories