Python: copying a list the right way

http://henry.precheur.org/2009/2/11/Python%3A_copying_a_list_the_right_way.html

by henryprecheur • 17 years ago

71 47 17 years ago

henry.precheur.org

mjtokelly• 17 years ago

Nice, clear explanation of something that drives Python beginners crazy--especially if it's their first programming language.

It would have been worth mentioning the 'copy' module. 'copy.copy' for shallow copies of any object, 'copy.deepcopy' for recursive copies.

JabavuAdams2• 17 years ago

I'm guessing, but a function that can copy any object is unlikely to be as efficient as one that works with known types.

Of course, even if true, that may not matter.

See, I'm all about the weasel words today.

cstejerean• 17 years ago

It would be a little faster. The first part of copy.copy looks like

  def copy(x):
    """Shallow copy operation on arbitrary Python objects.

    See the module's __doc__ string for more info.
    """

    cls = type(x)

    copier = _copy_dispatch.get(cls)
    if copier:
        return copier(x)

    # more after this point but it's not relevant for lists

So calling copy.copy on a list over using list() will check for the type, look up the type in a dictionary after which it will proceed as if you called list() yourself.

palish3• 17 years ago

I'm not sure either... A hand-written .clone() method would always beat the generic solution of course, but it seems like the tradeoff is that you spend less time coding, which is a win.

newt0311• 17 years ago

I think addresses and pass-by-reference drive people crazy when they start

ggrot• 17 years ago

Agreed. The deep copy is much much better. Consider this code:

>>> a = [1, [2, 3]] >>> b = list(a) # or a[:], they are identical >>> a[1].append(4) >>> a.append(5) >>> a [1, [2, 3, 4], 5] >>> b [1, [2, 3, 4]]

If you had used copy.deepcopy, b would now be [1, [2, 3]] as intended.

diN0bot2• 17 years ago

these are good points.

of course, what with classes i very rarely have deep lists anyway. in fact, copying lists is seldom an issue i run into (though i always run a few tests at the prompt to make sure my understanding is right because mutability bugs can be hard to track down later).

anuraggoel• 17 years ago

DO NOT replace [:] with list() next time you see it.

While the article explains some concepts well, [:] is no less pythonic than list(), though the latter might be more readable for people completely new to the language. In fact, it would be helpful to become very comfortable with python slices early on, because they allow you to easily manipulate any sequence, not just lists. For example, you can reverse a string using slices in one line:

  reversed_string = orig_string[::-1]

Staying away from slices, you're leaving that power behind.

marcus• 17 years ago

Another neat trick with slices is that you can do a partial update on a list.

a = range(6) a[::2] = range(10,13) -> generates a with [10,1,11,3,12,5]

I saw a very elegant implementation of Eratosthenes sieve based on that trick

d0mine4• 17 years ago

http://www.rosettacode.org/wiki/Sieve_of_Eratosthenes#Using_...

  from numpy import bool_, nonzero, ones

  def primes_upto(limit):
      is_prime = ones(limit, dtype=bool_)
      for n in xrange(2, int(limit**0.5) + 1): 
          if is_prime[n]:
             is_prime[n*n::n] = 0
      return nonzero(is_prime)[0][2:]

palish3• 17 years ago

Programming shouldn't contain 'neat tricks'. (Incidentally, one of Python's guiding principles is "explicit is better than implicit": http://www.python.org/dev/peps/pep-0020/)

Every single instance of "reversed_string = orig_string[::-1]" should be preceded with a comment saying "reverse the string", because it's not immediately obvious what [::-1] does. You can learn what it does, but it's not obvious, the same way a cryptic regex isn't obvious. So why not write reversed_string = orig_string.reverse()? (Or reverse( orig_string ) if that's more Pythonic.)

dood2• 17 years ago

But [:] and [::-1] are not neat tricks, they're basic features of the language, and are immediately understandable once you've learnt about slices. Practically the opposite of a cryptic regex.

The purpose of comments should not be teaching the reader the language, that just makes the code noisy.

Daniel_Newby• 17 years ago

"So why not write reversed_string = orig_string.reverse()"

reverse() does an in-place reversal of the original list and does not return the result.

henryprecheur2• 17 years ago

reversed_list = list(reversed(orig_list))

This works too, and IS more readable. But this doesn't work for a string for example.

I didn't stay to replace ALL slices with list(). Of course this wont work all the time. For instance [:] has nice properties like conserving the sequence type:

>>> (1, 2, 3)[:] (1, 2, 3)

>>> [1, 2, 3][:] [1, 2, 3]

This is not possible if you call list(), or tuple() directly.

Slice is a feature that requires more "cleverness" from the programmer and more brain power from the guy reading the code.

To cite Dijkstra: "[The competent programmer] avoids clever tricks like the plague."

If list() make your code clearer, use it. The goal is to write a working program with clear and readable code, not to show off with you mastery of slices.

anewaccountname• 17 years ago

Give me one example where you would ever want a copy of a tuple rather than another reference to it.

d0mine4• 17 years ago

numpy arrays do not copy on `[:]` therefore an explicit `.copy()` is required.

`copy.copy()` makes a shallow copy of both a ordinary list and numpy arrays.

snprbob86• 17 years ago

Is this more immediately obvious? I would have expected list to be defined as list(*elements) and called like this list(1, 2, 3)

I guess that this allows me to type help(list) and figure it out, but I would have done that anyway to identify an operator.

Clearly the answer is a .clone() or .copy() method...

ewiethoff3• 17 years ago

> Clearly the answer is a .clone() or .copy() method...

if you want Python to smell like Java or Ruby. :->

latortuga2• 17 years ago

http://docs.python.org/library/copy.html

Seems to disagree with you.

nihilocrat• 17 years ago

I think doing list(foo) to create a list copy is a pretty obvious way of getting around the 'variables are just references' thing.

I honestly have never seen someone do foo[:] just to copy a list, this is all news to me.

jodrellblank• 17 years ago

This is confusing for beginners and should be avoided.

That doesn't follow.

Beginners need to learn list slicing to get anywhere with Python, and by the time they've got through [1], [0:10], [2:], [:5], [:-1], [0:10:2] and so on then [:] is just another use in the same pattern.

quantumhobbit• 17 years ago

I have to admit that slices were the hardest part of learning python for me. Mostly because it took me awhile to find as clear an explanation of their "pass by value" behavior, to borrow C terminology. Slicing incredibly powerful, though and should be given greater attention in intro materials.

palish3• 17 years ago

Drat, the only one I don't know is [0:10:2]. I guess it's back to The Python Tutorial for me. http://docs.python.org/tutorial/

Followup: The tutorial didn't seem to mention what [x:y:z] does, so I checked:

  >>> range(15)
  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]

  >>> range(15)[0:10]
  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

  >>> range(15)[0:10:2]
  [0, 2, 4, 6, 8]

  >>> range(15)[0:10:3]
  [0, 3, 6, 9]

So the third argument is the number of elements to skip (minus one) between each item in the new sequence. Neat.

nadim• 17 years ago

I also found this: http://www.python.org/doc/2.3.5/whatsnew/section-slices.html

These are "extended slices" and the third argument is the step. This link also explores deletion and the __getitem__ method:

One can also now pass slice objects to the __getitem__ methods of the built-in sequences:

>>> range(10).__getitem__(slice(0, 5, 2))

[0, 2, 4]

Or use slice objects directly in subscripts:

>>> range(10)[slice(0, 5, 2)]

[0, 2, 4]

limmeau• 17 years ago

And while we're at confusion and beginners: at least

them = things[:]

looks different from

them = (list) things;

which in C-like languages usually does not make a copy of things.

Using slices may avoid fallacies like "I know that int(x) is something like a cast of x to int, so list(x) is like a cast of x to list, which does nothing when x is a list".

njharman2• 17 years ago

Huh, I've almost never seen [:], would never think of it. Do people really use that? I mean it takes a lot of work to make python cryptic but I guess if you're determined anything is possible.

use deepcopy or list().

I don't even use [] or {} to create empties anymore. I much prefer explicit esp since there is proliferation of container types and it's silly, inconsistant, and confusing that dicts and lists have syntactic exceptions.

    newlist = list()
    newdict = dict()
    newdict = defaultdict(str)

etc.

And to anuraggoel. not using [:] is not avoiding slices. just as not using string += "ext" (esp in loop) is not avoiding strings.

nostrademons• 17 years ago

I always used [:], although now I think I'll switch to copy.copy or list() after reading this thread. (Actually, now that I work for BigCo, I'll have to check what their coding standards say about this.)

thorax• 17 years ago

I've also used list() because I find it just generally more readable.

ivank• 17 years ago

Creating empties with {} takes just 58% of the time as dict() [0.25 us, 0.39 us] because there's no need to LOAD_GLOBAL and CALL_FUNCTION.

'not not variable' is similarly faster than bool(variable).

(not that I've run into this often in Python.)

njharman2• 17 years ago

> takes just 58% of the time

Whoopee fucking doo, really.

yesimahuman• 17 years ago

I thought this was going to be a talk on performance issues. I think most python people dealing with production code would understand either.

pkrumins• 17 years ago

<moody> "a[:] feels a bit too much like Perl" - doesn't feel like Perl at all!

Python: copying a list the right way

Related Stories