Show HN: Given an API, Generate client libraries in Node, Python, PHP, Ruby

by sunkarapk • 13 years ago

268 73 13 years ago

PS: Assuming a REST architecture.

sunkarapk10• 13 years ago

Author here. I covered almost everything in the documentation. And there is also a small example which is hosted at https://github.com/alpaca-api.

Please ask if you have any questions. Thanks

htuao• 13 years ago

How do you differ from Swagger, Blueprint, I/O Docs, RAML and all these API endpoint descriptions?

sunkarapk10• 13 years ago

I tried searching for a program which generates client libraries in different programming languages and failed.

Could you please link if something does it? From what I gather, the above mentioned sites only provide methods of describing the API and/or Automating the API on the server side. Nothing is said about client libraries.

Thanks

rybosome• 13 years ago

Hey, I noticed you mentioned that you were interested in having client library generation in Obj-C, Java, and Scala, yet all of the existing client lib support is for dynamic languages. I think you may find that some fundamental changes need to be made to support this, unless every datatype is just going to be a string.

sunkarapk10• 13 years ago

If you are talking about API definition, there need not be many changes since api.json do not define any objects.

If you are talking about the code of the program, they wont be changed because, the program just compiles and executes the templates.

If I didnt answer your question properly, could you please elaborate?

johnnyio2• 13 years ago

Describing an API is not hard, but the API authentication method is. How do you think you will do it?

Edit : If you don't make oauth consumption simpler, you don't really solve the problem

sunkarapk10• 13 years ago

Currently, I am supporting 3 authentication strategies. Basic, Token in header and OAUTH.

These will cover most of the APIs. And I am open to include other kind of authentication strategies too.

https://github.com/pksunkara/alpaca#authorization-strategies

EDIT: OAUTH Consumption is not the main priority here. The problem I intended to solve is automating the development and maintaing of API client libraries in several different languages.

That said, I intend to add support for OAUTH consumption too.

pjmlp2• 13 years ago

Can we please stop calling Web Services APIs?

AznHisoka• 13 years ago

Why? what else do you call APIs?

thu• 13 years ago

Libraries have APIs too.

johnnyio2• 13 years ago

Win32 API for example

pjmlp2• 13 years ago

Libraries and operating systems have APIs.

Web sites have Web Services using REST, XML-RPC, SOAP, whatever as communication protocol.

alok-g• 13 years ago

Or probably put "Web Services API" in the title: "Given a Web Services API, Generate ...".

rhizome313• 13 years ago

Or just "Web API", short but specific.

mikekekeke2• 13 years ago

Am I right in thinking about this like a WSDL, but based on JSON?

sunkarapk10• 13 years ago

Being a WSDL is not the aim here. Generating client libraries in multiple programming languages is.

But, to do that I need a format using which people can define their API. I used JSON and only supported the elements that I needed to generate the client libraries.

mikekekeke2• 13 years ago

Right, the format affords generic description documents from which to generate client code. This seems very similar in spirit to WSDL. No?

MichaelGG• 13 years ago

It looks like a WSDL-like spec (although not as complete) with a code generator as well.

I wonder if we'll see JSON versions of DISCO and whatnot as well.

squar1sm• 13 years ago

I think this is fantastic. @sunkarapk, great job. That it's written in Go makes using it so much easier. Wonderful.

jxf2• 13 years ago

Just tried this out and Alpaca is an awesome tool. However, I'd never want to release this in production without tests. Alpaca doesn't generate tests, so you're back to maintaining the tests for your N different platforms/languages. But you'd have the same problem with its competitors, too; Thrift [0], for instance, doesn't generate tests either.

Overall, I'm not sure that the time savings is as big as it first appears, but I think it's great for quick projects.

[0] http://thrift.apache.org/

sunkarapk10• 13 years ago

Tests is one of the main priority and I am planning to do them soon.

This current program just a small step in the right direction. :)

EDIT: Currently, I have a test suite at https://github.com/pksunkara/alpaca/tree/testing which tests the generated client libraries of an example API with the respective server.

X4• 13 years ago

That sounds cool! Is there a way to automatically generate BDD style Tests? Have a look at http://funkload.nuxeo.org/intro.html

Robin_Message2• 13 years ago

I'm confused by this comment. What would the tests test?

My thinking is that, if trusted to output a library that matches the API spec, there is no need to test that API. However, that does require that Alpaca is well-tested enough to be trusted - would you be happy with tests in Alpaca itself, or should it generate tests too?

jxf2• 13 years ago

I guess it depends on whether you view Alpaca as (a) "it's a starting point for generating your API", or (b) "it's the authoritative source and the only way to generate your API". If you use it like (b) you don't need tests, but Alpaca does. If you use it like (a) Alpaca still needs tests, but the buck stops with you.

dsl2• 13 years ago

If you have programmatically generated code, generating tests is almost completely useless (especially if the same code is generating the tests as well).

A bunch of passing tests shouldn't make you feel comfortable able releasing something to production. Proper code review and monitoring of client failures on your servers should make you confident.

acbart• 13 years ago

So, this is markedly similar to the project I've been working on in grad school, only without static typing. http://research.cs.vt.edu/vtspaces/realtimeweb/ Also, mine is explicitly geared towards education purposes. I'm about one third of the way thru version two, but I wonder if we can cross pollinate our code bases to get something even better.

bashtian• 13 years ago

I did something similar for Go and Java. It's simpler if you don't need the whole API, but of course not as powerful. https://github.com/bashtian/jsonutils

endeavour• 13 years ago

Does this use the Json Schema spec or have you reinvented the wheel?

sunkarapk10• 13 years ago

JSON Schema is too complex for this project. I just chose the fields which needs to be populated so that I can generate the code.

sethrin• 13 years ago

Not that I would defend JSON Schema, but yes, you did just reinvent the wheel, and it was also not a new idea. I thought that despite the verbosity of XML, WSDL and associated technologies solved this problem rather better. I have the misfortune to develop in PHP, so pardon the example, but the client-side code went something like this:

$PayPal = new SoapClient($WSDLLocation); $PayPal->SomeMethod();

Yes, you can get away with having a less verbose description language if your needs are simpler, but that to me just ends up moving the problem when your needs become less simple. It's not like you can avoid validating your requests in some way, and required/not required is where that starts -- but, I maintain, not sufficient. I most definitely fail to see client code generation as being a necessary step.

You could say I've made this mistake myself; the thought process that leads to code like this must not be unique. Web services need documentation if they are to be used, and since that code will only ever talk to other code, it makes sense to have a machine-parseable description. Language agnosticism follows as a matter of course. I'm willing to entertain that XML was a bad idea and JSON Schema is not an improvement, but I still feel that if one must reinvent the concept of a language-agnostic machine-parseable web services description language, one should thoroughly understand the prior art. It may be complex for a reason, and like it or not all that SOAP stuff actually tends to work pretty well. I'm sure I have every reason for wanting to see a better technology suite which is JSON based, and I wish this were it. At the moment I don't think you're headed in quite the right direction.

gamache5• 13 years ago

Wouldn't it be wiser to choose a hypermedia format for the API, and then use a generic hypermedia client on whichever platform you like? Then you write just as many client libraries (zero), but the problem of pushing updates to your clients is solved as well.

Full disclosure: I wrote such a library for Ruby, called HyperResource. https://github.com/gamache/hyperresource

sunkarapk10• 13 years ago

I agree its nice to do it that way. But I found that not all APIs use hypermedia format. So, I did it this way. :)

gamache5• 13 years ago

Ah, that makes more sense. I was under the impression that Alpaca was targeted at people writing the APIs.

Neat tool!

scarmig• 13 years ago

At risk of writing a vacuous comment but as a supporter of more HN positivity...

Damn, HyperResource looks sexy.

gamache5• 13 years ago

I appreciate it -- that was kind of the point. :)

rhizome313• 13 years ago

I've read most of the book RESTful Web APIs, which advocates Hypermedia as a way to stop having to create client-specific libraries for each and every Web API. But the book doesn't say much about creating such clients.

Is there such a generic Hypermedia client for Python?

How does this affects performance? Navigating through the maze of a Hypermedia API requires many more requests than just hitting a known endpoint, doesn't it? Do hypermedia clients provide client-side caching?

My other concern is that those APIs might be harder to learn than ad hoc fiat standard. The reason why RESTful-ish APIs have been successful is because they don't add any overhead on top of the things they're representing. At first glance, stuff like JSON-LD, Siren, HAL, etc. seem to be bringing back the complexity of things like SOAP that people have fled from.

steveklabnik• 13 years ago

Yup, this is largely the year of the client.

Python?

There are some libraries, expect more to spring up soon.

How does this affect performance?

This kind of question is incredibly broad. In ways it's more efficient, in ways it's less. It Depends.

That said, as you alluded to, caching should be very prevalent, so that helps.

Also, it's not like you have to make 5 requests any time you want to do anything: the point is that you follow the application's state along. Just one request per transition. A maze is a pretty decent analogy, actually...

harder to learn than an ad hoc fiat standard

They may _seem_ harder to learn, but you have to re-learn every single ad-hoc standard over and over and over and over. If you learn HAL, you can speak to any number of APIs that use HAL. Plus, you say 'complexity,' I say, 'no surprises.' Everything is actually enumerated for you, so it should be easier to learn. No hidden assumptions.

Furthermore, as you're more familiar with the format, the details fade into the background. I'm sure you don't read RFC 4627 every time you want to deal with a JSON-based API, either.

gamache5• 13 years ago

The client space is slowly filling in. There are a bunch of clients which still leave the plumbing exposed (a couple good ones are HyperClient.rb[1] and HyperAgent.js[2]) -- it still feels like you're making HTTP requests.

One of the things I like most about hypermedia is that the hyperlinks can represent a complete set of functions which can be applied to an object. In other words, each object contains its own method list. This fits well in languages which get to implement catch-all methods, like Ruby, and I couldn't resist coding up a client that worked that way.

(There's room for this sort of trickery in near-future ECMAScript too, with Proxy[3]. I would really like someone to do this and I would kind of like it to not be me.)

When you lay out your API according to that philosophy, and cache a few "stepping-stone" objects you'll be traversing often, hypermedia APIs don't seem so inefficient at all.

As to your last point: take a look at HAL[4]. It sits alongside a "traditional" API layout very nicely, essentially adding "_links" and "_embedded" which can be safely ignored by non-hypermedia clients. The HAL spec is extremely sane.

[1] https://github.com/codegram/hyperclient

[2] http://weluse.github.io/hyperagent/

[3] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

[4] http://stateless.co/hal_specification.html

mjs• 13 years ago

Regarding bug reports:

Guaranteed reply within a day.

That seems difficult to achieve--wonder how they're doing that? (Also, why??)

sunkarapk10• 13 years ago

I am just a single guy. I always make a point to reply back to issues/pull-requests on my open source projects within 24hrs.

Yes, sometimes I fail to make it. But it's a rule I want to live by.

lucaspiller• 13 years ago

Good luck. I found a six month old PR on one of my projects the other day....

OGC• 13 years ago

"Thanks for your report. I will read it later."

Done.

memset• 13 years ago

This is cool. I would suggest that it would be very useful to have this kind of thing for JSON Schema [1], which is what I use with Python code to validate incoming JSON. (I was originally hesitant to using that, but since getting into it, I have yet to run into a use case which it cannot handle.)

There is also an RFC for "JSON Hyper Schema" which is intended to describe REST APIs. It doesn't have much library support in much of everything, but I am surprised that it hasn't taken off!

I like that this library is fairly opinionated (options for how to authenticate, supported formats, etc.) Though I worry that that creates a bit of inflexibility - for what exactly does "oauth" actually mean, there are always vagaries.

Neato!

[1] http://json-schema.org/

egonschiele• 13 years ago

Useful, well-documented, TODOs right in the readme, and fast response for pull requests. I wish every open source project was like this.

flippyhead• 13 years ago

I've been enjoying http://apiblueprint.org

cvburgess• 13 years ago

The web page was very pretty, but I still have no idea what this is/does. Help?

elwell• 13 years ago

Great idea! Including Obj-C would be very helpful.

codereflection• 13 years ago

I have to admit, this is very reminiscent of "Add Service Reference" in Visual Studio, a capability to which I have grown to despise over the years. The code was almost always incomprehensible. I cannot tell you how much I loath seeing the comment at the top of a file "This was generated by a tool".

Having said that, this tool does look interesting. I hope that a goal is to always make sure that the generated code is as readable, and maintainable, as possible. Also, as mentioned by others, adding generated tests to the generated client libraries is extremely important.

sunkarapk10• 13 years ago

You can go through the generated code examples at https://github.com/alpaca-api.

I tried to make them a lot more readable. :)

abengoam• 13 years ago

That looks fantastic, and seems to be in alignment with some things I have been doing lately (generating the server side controller of the API in Clojure + a set of documentation, from a set of definitions of API methods). Good job!

notastartup• 13 years ago

oh man you just killed a major feature of mashape.com

anonfunction• 13 years ago

Client library generation is something that mashape supports, but currently they generate libraries for:

Java, Node, PHP, Python, Objective-C, Ruby, and .NET

It looks like alpaca supports:

Java, Go, Perl, Clojure, Scala, Obj-C

mmccaff• 13 years ago

Well done!

Know what I think would be really neat? If it could be pointed at an instance of Swagger-UI, or use the same discoverUrl that Swagger-UI would use, and spit out the libraries from that.

If you're not familiar.. https://github.com/wordnik/swagger-ui

kimmel• 13 years ago

I like the SPORE (Specification to a POrtable Rest Environment) approach better. You create a description file in JSON and each native language client can use that file to access the HTTP API. https://github.com/SPORE/specifications

SPORE already has clients for Clojure, Javascript, Lua, NodeJS, Perl, Python, and Ruby. I have used SPORE in a few projects and I was not disappointed. Another approach to solving the cross language library problem.

CCs• 13 years ago

Looks promising.

Too bad the last commit was 2 years ago...

dsl2• 13 years ago

Maybe it is finished?

reklaklislaw• 13 years ago

fwiw, I experimented along this line, dynamically generating python wrappers from yaml: https://github.com/reklaklislaw/rest_easy

It lacks documentation, a bunch of features, and parts smell pretty bad, but since the topic came up I thought maybe someone would find it interesting, if only vaguely.

jodoglevy• 13 years ago

Very cool, but why come up with a new API schema rather than use an open standard like OData (http://www.odata.org/)? Then Alpaca would be compatible with a bunch of APIs that already exist today. In fact, something like this (generating client libraries from APIs) may exist for OData already, but if it does, I've only seen it for .NET and OData (Visual Studio 'Add Service Reference').

This is actually pretty similar to a side project I've been working on called Gargl (Generic API Recorder and Generator Lite) (https://github.com/jodoglevy/gargl). Haven't gotten around to doing a Show HN post yet, but would love any feedback or to combine efforts. Basically it lets you generate an API for websites that don't have APIs publically available, by looking at how a web page / form submission of that web site interacts with the web server. You record web requests you make while normally browsing a website via a chrome extension, parameterize them as needed, and then output your "API" to a template file. Then this template file can be converted to a client library in a programming language of your choosing.

Show HN: Given an API, Generate client libraries in Node, Python, PHP, Ruby

Related Stories