Show HN: Addict – a Python dict whos values can be get and set using attributes

by mewwts • 12 years ago

80 72 12 years ago

Cieplak3• 12 years ago

Maybe consider creating a pull request against the standard python library. This is really nice. This feels like it belongs in the collections module alongside defaultdict.

the_mitsuhiko2• 12 years ago

No it really shouldn't. The usage is highly problematic because of naming conflicts and the implicit default behavior. This causes more problems than it's worth.

Cieplak3• 12 years ago

I'm not suggesting that it override the default dictionary behavior, but that it would be nice to have the option of using javascript-like object syntax by importing some special dict class. This library is really young, but I've always thought this sort of feature would be nice syntactic sugar, particularly for configuration objects. I find myself using namedtuples for this sort of thing, but it would be nice to have the option of using an ad-hoc data structure like this that's inherently trivial to serialize.

mewwts17• 12 years ago

Hi, the naming conflicts should now be fixed :-)

rcarmo4• 12 years ago

I've been using a similar thing I call Struct for ages, as part of my utils package:

https://github.com/rcarmo/python-utils/blob/master/core.py

Can't really see this as newsworthy, sorry.

Edited to add: didn't mean to sound mean (if you'll pardon the pun), just factual. There are ActiveState recipes out there for this sort of thing since the beginning of time... Also, check my other comments for thoughts on syntax and immutability.

lmartel• 12 years ago

Does that last sentence contribute anything?

"Neat, I did something similar here: [link]" communicates the same information. If it's not newsworthy it'll fall off the front page and that'll be that.

I'm sure you didn't mean to be hurtful, but `Show HN` posts almost always end up with a broad, critical top comment. Constructive criticism is great but "this is trivial" just seems discouraging.

rcarmo4• 12 years ago

Didn't mean to come across as mean, merely factual.

There are plenty of similar implementations around, in fact, and this is the kind of thing I've also seen as part of ActiveState recipes.

cjbprime• 12 years ago

Hey, this looks super useful. Thanks for posting!

mewwts17• 12 years ago

Thanks man, glad you like it.

Argorak2• 12 years ago

A pleasure to see the Python community adopting Rubys drug-related naming schemes.

rmc3• 12 years ago

Yeah, something professional, maybe?

rankam3• 12 years ago

Can I ask why you would want it to be "professional" and who deemed this not "professional"? I mean, the language's name is derived from Monty Python - I don't think anyone would consider that "professional". Hackers are notoriously clever and comical - shouldn't our work reflect our culture?

"Professional", to me, is boring - it's what the pointy haired boss wants you to be because anything remotely outside of the box doesn't conform to their risk-averse norms.

rooodini• 12 years ago

I’m afraid I find it thoroughly unpleasant :(

mewwts17• 12 years ago

Haha :-)

ecma2• 12 years ago

This doesn't even come close to correctly implementing the MutableMapping interface in a way that makes sense for the expected behaviour. The biggest problem I see is that it's not at all clear how methods like __iter__, key/value (and the view variants) and has_key should behave. Honestly, I'd homebrew a nested defaultdict(defaultdict) every time because at least I know what it'll do.

mewwts17• 12 years ago

Valid input, but a nested defaultdict won't let me recursively define items: a.b.c.d.e = 2 as far as I can tell?

adambard2• 12 years ago

It will if you recursively define it:

    from collections import defaultdict
    def dd(): return defaultdict(dd)
    d = dd()
    d['a']['b']['c']  # => defaultdict(dd, {})
    d['a']['b']['c'] = 'test'
    d['a']['b']['c']  # => 'test'

ecma2• 12 years ago

It does recursive access but not attribute access. The problem I see is that attribute access is only part of the problem you're attempting to solve. That's a nice feature but a pretty well known problem (check out the self.__dict__ = self trick) with fairly clear behaviour mimicking the usual dict.

The surprising bit that would stop me dropping this in any real code is that as an end user, it's not immediately clear how the mapping interface should apply to this class. It's all implementation defined but it's not actually defined in the implementation. e.g., Does my Dict() have the key 'a.b.c'? Nope, it only has 'a' and the other keys are managed through some magic. Same.applies to iteration, it might be reasonable to assume that'd be a depth first terminating node iterator but it would actually just iterate the top level of the main Dict instance.

mewwts17• 12 years ago

Hey guys, author of addict here. Would appreciate immensely if you would file issues of problems you find. It's a very new project, that I cooked up after work hours this week, and any input is good input! Thanks.

thruflo• 12 years ago

I like .attribute syntax as much as the next developer but the mutate-on-get behaviour strikes me as... undesirable.

czam• 12 years ago

Yes, .attribute syntax is much nicer than brackets, especially on German keyboard layouts (needs "Alt Graph"). Why can't dicts just support both? Because of the Zen? Also this is an old issue, see Alex Martelli's bunch class recipe on ActiveState http://code.activestate.com/recipes/52308-the-simple-but-han...

The mutate-on-get makes it very similar to how Matlab does its "structs", I'm mostly undecided on that but at times it can be nice.

ekimekim• 12 years ago

He is explicitly copying the behaviour of the standard library defaultdict here. One clear motivation is thus:

Suppose I have an empty defaultdict(list). I do the following:

    d = defaultdict(list)
    l = d['foo']

According to no-mutate-on-get semantics, we now have an empty list, and d is still empty. So what happens if I do:

    l.append('bar')

You might expect the dict to now be populated with {'foo': ['bar']}. However, the dict has no way of knowing (without a large amount of additional tricks) whether the new list that it previously returned had been later modified. So what would actually happen would be nothing, d would remain empty. Worse, now your behaviour is different depending on whether or not the 'foo' key was already present (if it was, you would've gotten a reference to the actual value and mutated it, and d would have changed as a result).

As I see it, there are only two sane solutions to this problem: Either immediately store every generated value so any subsequent mutation is saved (the solution used here)...or require that your default values be immutable.

wshac• 12 years ago

How does this compare to AttrDict and treedict?

mewwts17• 12 years ago

AttrDict only lets you get items using attributes. addict lets you set them as well!

zb3• 12 years ago

I haven't tried it but here's some issues from a superficial code review:

This uses twice as much memory as it needs to (and opens the door to potential inconsistency) by storing everything redundantly in both the attributes dictionary and the underlying dict.

Setting an item with a non-string key (e.g. An int or even unicode) is silently ignored.

    myDict[u'foo'] = 'bar'  # <- this ought to do something

Setting or deleting an item with the same name as an existing attribute (e.g. the name of a method) will replace or delete that attribute.

    myDict['get'] = 'foo'  # <- should not replace the method myDict.get

This can't be initialised with kwargs or an iterator like an ordinary dict can.

mewwts17• 12 years ago

Thanks so much for your input. All of this is now fixed!

personjerry• 12 years ago

I thought AdDict would be a dictionary whose values automap to Google Adsense ;)

michaelmior• 12 years ago

*whose values

mewwts17• 12 years ago

thanks :-)

tyrion2• 12 years ago

I think you could implement it like this:

  from collections import defaultdict
  
  class D(defaultdict):
      
      def __init__(self):
          super(D, self).__init__(D)
      
      __getattr__ = defaultdict.__getitem__
      __setattr__ = defaultdict.__setitem__
      __repr__ = dict.__repr__

BerislavLopac• 12 years ago

You think right.

zb3• 12 years ago

Almost, he is also converting ordinary dicts that get added as items to his own Dict class to enable you to keep using this syntax further down the tree, so you'd need a small wrapper around __setitem__.

But yes, the idea of storing everything twice (once as an item and once as an attribute) just so you can unconditionally add defaults in __getattr__ is… bizarre and will break things horribly as soon as you accidentally do something like:

    myDict['get'] = 'foo'

tyrion2• 12 years ago

Almost, he is also converting ordinary dicts that get added as items to his own Dict class

You are right I dind't notice that before. But looking at the code it seems that the only permitted values are dict and str?!

But yes, the idea of storing everything twice (once as an item and once as an attribute) I don't get that, you don't store things twice and nothing breaks, at least with the example code I wrote above. Try:

  >>> a=D()
  >>> a.lol = 2
  >>> a['get'] = 'foo'
  >>> a
  {'get': 'foo', 'lol': 1}
  >>> vars(a)
  {}

hueving2• 12 years ago

But if you do it like that, it's almost not worth packaging into a library and posting to hackernews. :)

ntoshev• 12 years ago

This doesn't trigger autocompletion of attributes in dreampie / bpython / ipython - the original implementation does.

mewwts17• 12 years ago

This is true!

mewwts17• 12 years ago

This is neat! Super close to what I want in much less code :-)

nvader• 12 years ago

One of the best things about the Python ecosystem is the acceptance and idealization of "Pythonic" solutions. While I acknowledge and praise the authors effort in making this complete and correct, the aim itself is misguided because it is grossly unpythonic. I reject it and hope it does not see significant use.

adambard2• 12 years ago

The problem with "Pythonic" is that it's poorly-defined. There seems to be a consensus on some things, and a wide disagreement on others.

You're welcome to your opinion, but I don't think it's productive to wield the phrases "pythonic" and "unpythonic" as if they represent an objective truth. If you have a problem with the library (and there are a few angles you could take), say that instead of denouncing the library as blasphemy.

gbog• 12 years ago

I have never seen anyone saying that pythonic refers to some objective truth. It refers to subjective but real concepts as clarity, simplicity, explicit-ness, etc. There is also one thing that is not really pythonic but still important, it's reliability. Here addict is not reliable because it is unclear what happens when you call "addict.get" or other dict built-in methods like keys, items, etc. Something is reliable when it works equally for all values in the value space. dict['x'] is reliable because it works the same even if you replace 'x' by 'get' (or any other immutable value).

hueving2• 12 years ago

Implementing an entire 3rd party python library with behavior almost the same as a defaultdict is unpythonic because a reader of the code base now has to learn an entirely new library for functionality that could be represented in a 10 line extension to defaultdict.

RubyPinch• 12 years ago

The whole "it must be pythonic" thing bothers me so much sometimes.

Its a shortcut to dismiss someone's views. Having an opinion rejected with a "import this, line N" is... such an annoyance. Some people treat the damn thing like a strict bible

sswaner• 12 years ago

I disagree. "Pythonic" and "Unpythonic" are inherently subjective and are always used as an expression of opinion regarding the consistency of some code, routine or library in relation to the larger body of Python.

Unless it is written PEP8 or by Guido, I would never take a "pythonicism" to be nothing more than short-hand for "I think this is (in)consistent with the larger patterns and traditions of Python as I understand them."

phazelift• 12 years ago

That's funny, I'm working on something similar for Javascript: https://www.npmjs.com/package/xs.js

sysk• 12 years ago

What if a key has a space?

PythonicAlpha• 12 years ago

I use some similar classes in my projects.

It always has frustrated me to see that attribute- and dictionary access is so similar but still different. Of course in most cases, there are some good reasons to have it different -- but sometimes there are also good reasons to generalize access to some important values.

So, we have duck typing and car typing. And both together we have a duck-car, because it quacks like a duck and runs like a car.

Demiurge• 12 years ago

This looks like something I really wish was just baked into the language. I think there is a reason it isn't, like in JavaScript, and that is because Python isn't as weakly typed, and encourages explicitness. So, I wouldn't use it and would discourage everyone working with me from using this.

pjkundert• 12 years ago

https://github.com/pjkundert/cpppo/blob/master/dotdict.py

Quite complete attribute accessible dict, with iteration by full-depth keys. Very useful for human readable output.

lifeisadance• 12 years ago

Aaron Swartz did something like this years ago, except it's ultra simple and doesn't include any of the weird recursive stuff, in his web.py: the Storage class.

https://github.com/webpy/webpy/blob/master/web/utils.py#L52

mewwts17• 12 years ago

Cool! I need the recursive stuff, though!

dgemm• 12 years ago

Simpler:

  class Storage:
    pass

anentropic• 12 years ago

why do people keep insisting on inventing this ugly and pointless 'solution'? too much Javascript?

just use `mydict.setdefault()` and `defaultdict` and give up on the whole attr access thing. it's a dict

mewwts17• 12 years ago

I think the example on the README should make it pretty clear.

mewwts17• 12 years ago

So I've made some changes over the day. Some issues still stand, but as some people pointed out it was unnecessary to store both attributes and items. Now the get and set attribute simply calls get and set items. If you find the time, feel free to add an issue along with you HN comment :-)

RayVR2• 12 years ago

this is an idea that I've seen used many times in the past. One of the first refs I saw to this mashup of the keys of a dict and the values in __dict__ was on Activestate in 2006 or so. There are also multiple versions on Stackoverflow.

I'm reminded of the web designer example here (https://gist.github.com/fmeyer/289467) line 89.

I know that people use github for many things, including tracking their config files, etc. However, this seems to be a post by the owner of the repo. Would you ever, I mean _ever_ be happy if someone required this sort of 'library' as a dependency in a project? I know I would not.

RayVR2• 12 years ago

An example that you should try: d = addict() d['__dict__'] = None

Show HN: Addict – a Python dict whos values can be get and set using attributes

Related Stories