Boto3 – AWS SDK for Python

https://aws.amazon.com/blogs/aws/now-available-aws-sdk-for-python-3-boto3/

by jeffbarr • 11 years ago

113 25 11 years ago

aws.amazon.com

danso2• 11 years ago

Wow, what a coincidence, I literally just installed this yesterday because I wanted to write my own command-line S3 uploader...yes I know there's an official CLI [1] but I wanted some customized behavior and options (such as simultaneously backing up to s3 and glacier) and thought it'd be a good chance to better understand AWS in general. I don't know what the previous boto was like but boto3 has been pretty pain-free. Many of the error messages I've run into have been very actively discussed/patched/hacked-around in the repo issues.

[1] http://aws.amazon.com/cli/

samstave• 11 years ago

https://github.com/bloomreach/s4cmd

vosper2• 11 years ago

Sorry to go off topic, but why would you want to simultaneously backup to S3 and Glacier? Unless you're using S3 with Reduced Redundancy then you're not going to lose any files, so Glacier seems pointless. Also, working with S3 is way easier than working with Glacier, so if you're using S3 already I would just stick with that.

Psyonic• 11 years ago

Redundancy is not the same as backup. You may not lose files due to machine failure, but a code error, some data scientist deleting the wrong folder, etc, could easily delete the files. Having a backup that isn't automatically tied to the s3 bucket can help prevent catastrophe in these cases.

danso2• 11 years ago

Yeah that's true. I think I meant more toward understanding the granularity of operations common to AWS, with the assumption that the logistics of moving data onto S3 to Glacier would be the same. Since you can control expire-times on S3 files, there might be situations in which you want files on S3 for a short period, maybe 30 days, because you want a window for quick access just in case you were prematurely archiving, while making sure they're also on Glacier...I frequently find myself hoarding files, keeping them on S3 (or worse, USB keys) for longer than I had intended...

pdelgallego• 11 years ago

such as simultaneously backing up to s3 and glacier

I you want that for every object in the bucket (or specific files with a prefix), you are be able achieve that by adding a lifecycle rule:

1 Select the bucket 2 Add a new lifecycle rule 3 Action = Archive only 4 Archive to the Glacier Storage Class 0 after the object's creation date. (Enter '0' for same-day archival) 5 Save rule

You can automate that using aws cli

arthursilva• 11 years ago

Breaking backwards compatibility is dangerous in any major project. Hopefully the community will migrate reasonably fast and this don't turn up the same as Python 3.

jessaustin• 11 years ago

This is even more ridiculous than the increasingly desperate FUD we hear about py3. If you want to keep using boto, you can. If you want to use boto3 on py2, you can. If you want to use boto3 on py3, you can. As noted in TFA, if you want to use the old boto on py3, you can, and some of us have since last year.

rectangletangle• 11 years ago

I've been using boto's S3 client in Python 3 for quite some time, and have had no issues. Python 3 isn't PERL 6, it takes time to supplant something as ubiquitous as Python 2; look how many banks are still running COBOL...

pbreit• 11 years ago

Wow: 100s of 1000s of weekly downloads!

vanessa98• 11 years ago

http://stackoverflow.com/questions/9648015/pypi-download-cou...

sciurus• 11 years ago

The title is slightly misleading; boto3 is a new major version of boto that works on Python 2 and 3.

anezvigin• 11 years ago

It includes more functionality as well. I've been using the Import/Export API [1] for example. From what I've seen it's a major improvement over boto2.

[1] http://boto3.readthedocs.org/en/latest/reference/services/im...

omni• 11 years ago

I find it really hard to be excited about anything Boto-related when whoever's in charge of the main Boto project has allowed it to become an unmaintained disaster. Would much rather see them support their existing project. https://github.com/boto/boto/pulls

hoodoof2• 11 years ago

Anyone here from Amazon to address this? Is this incorrect?

nhumrich2• 11 years ago

This is correct. Boto3 (and its child dependency botocore) have been in the process for quite some time now. I'm not on the team that works on botocore, but I do interface with them (I work on the EB CLI which uses botocore).

zimbatm• 11 years ago

I am not intimately familiar with the project but the boto3 rewrite has been in the doing for a long time, all while boto2 was still getting new services and features added to it. Just the single lib is a huge project in itself, it must have been a long grind doing both at the same time.

pekk• 11 years ago

Boto3 as a successor addresses the problem of maintenance, doesn't it?

andrus• 11 years ago

I am glad the docs have improved. For example, RoutingRules: you used to have to go to the code to figure out how to use them.

cbaleanu• 11 years ago

boto currently has 3 APIs, depending on the module's version. Having both Table.query and Table.query_2 is confusing for newcomers, especially with the poor documentation available.

Rant over, this release looks cool, love the .all() chaining.

Boto3 – AWS SDK for Python

Related Stories