Django Rest Framework general model serializer

http://blog.hipwerk.com/django-rest-framework-general-model-serializer/

by norbert_mate • 11 years ago

64 25 11 years ago

blog.hipwerk.com

kayoone• 11 years ago

I am quite amazed this has to be done manually in Django. I work a lot with Symfony2 in PHP and that kind of Object serialization thing is pretty much built in. I just return my model from the controller and it gets serialized to JSON including child objects with pretty fine grained controls about what to serialize.

Besides, after working a lot with REST webservices and AngularJS, i don't know if serializing child objects is really a good idea vs just getting the id and then using the api endpoint for the specific children separately. Having child objects serialized creates problems when doing PUT/UPDATE requests and you can also end up with fairly large sets of JSON. Of course doing dozens of requests instead (especially for lists) is also not the best of ideas, so i am not really sure how others handle this ?

andybak2• 11 years ago

Django does this out of the box: https://docs.djangoproject.com/en/1.7/topics/serialization/

But DRF adds lots of extra functionality. Even with DRF it's as simple as adding:

    class UserSerializer(serializers.ModelSerializer):
        class Meta:
            model = User

to get full automatic serialization of a model.

I don't know if serializing child objects is really a good idea

Depends on the specific case. For internal APIs I tend to create endpoints that give a view what it needs in a single request.

norbert_mate6• 11 years ago

Correct. It's very easy to create a simple model serializer and this is what the general model serializer does. It's no big deal just saves a few line of code.

whalesalad• 11 years ago

You say "out of the box" but it's not that simple... 'blah blah DateTime is not JSON serializable'

norbert_mate6• 11 years ago

In Django Rest Framework (DRF) you need serializer class. Maybe this is because DRF is not tightly coupled with the models from Django framework.

As far as I know DRF does not support PUT/UPDATE operations for nested serializers. But there are cases when you just need to read. Like getting an activity feed which can't be modified directly by the users. I think if you have a good caching system it's better to return all the data for one request instead of creating new requests to get details about each entity.

numlocked2• 11 years ago

We tend to load things (GET) in one large request that pulls a big hunks of JSON for everything certain parts of the app need. These have lots of nested serialization going on.

We then have separate, traditional REST endpoints for updates/creates and incremental updates of the larger object.

adambratt• 11 years ago

No one has answered your question properly yet.

This is an edge case for something called Django Content Types. It's basically a way to handle some sort of polymorphism in a table.

Let's say you have a table called activity feed like in this example. Whether the items in the feed are images, blog posts, or status updates they're all going to have something in common and you'll have endpoints that should link any relevant tables and combine them. So you'd have an activity table and it would have a column for content_type and then content_object_id. Content type is basically just a reference to another DB table (blog or status update) with more info on this activity and content_object_id is the actual id of the row in that table.

I've used Symfony, Laravel, and pretty much every other trendy PHP framework and nothing comes close to the ease of building APIs that is inherent in Django Rest Framework.

This is just a very specific use case that you normally have to handle using it's own custom class. This just shows a way you can have DRF serialize it like a normal object without issue.

lazerscience• 11 years ago

There's not a general solution to serializing child objects, it totally depends on the use case and the number of children. So for example if you have a lot of child objects it might not even make sense to include theirs ids, but have a seperate (maybe nested) resource to retrieve them. On the other hand if there are only few children you might want to include them in their parent resource.

skatenerd3• 11 years ago

I really have a problem with the concept of the Serializer in DRF. It handles so much that has nothing to do with serialization. It performs something like de-serialization, where it takes an incoming dictionary and turns it into an ORM object. The docs even recommend using its ".save()" method. Save? on a Serializer?

I think it's a really cool concept if you think of it as more of a Lens. The so-called serializer lets you declare properties of "fields", and these properties will influence both the native-to-ORM conversion, as well as the ORM-to-native conversion.

norbert_mate6• 11 years ago

I agree with you. It is a bit weird that a serializer can interact with the database but what it basically does: hides the database interaction.

skatenerd3• 11 years ago

out of curiosity, how often do you find yourself reading the DRF source code to figure out whats happening?

tomchristie• 11 years ago

The model / model manager layer is the right place at which to design your state changing API. REST framework absolutely you to work with the grain there. Some good practice I'd recommend...

* Write the `create()` and/or `update()` methods explicitly on the serializer class.

* Push logic into the model and model manager where possible and only have the serializer `.save()` as a thin layer on top of that.

That way a serializer class still has all the behavior it needs to map both ways between persisted objects and their corresponding native python representations, but you still have a well separated model API.

morphyn• 11 years ago

Their API/role is very similar to the `forms` in standard Django. I agree with you that they handle too much stuff and that saving to the database has nothing to do in a serializer, just like it has nothing to do in a form. But I also think it's a very good choice to follow common Django practices, so that codebases stay somewhat consistent, and new users are able to understand the concepts very quickly.

ComNik• 11 years ago

I was thinking the exact same thing when I first read about DRF's Serializers. Also I don't quite get why serialization is always expected to work automatically, at least not in the context of mapping a business layer to an API layer (e.g. for JSON consumption), as those domains don't have to map directly onto each other.

Therefore, for an ongoing project I've implemented something along the lines of your idea (if I got that correctly), by describing API-Layer models in terms of properties, which are just named sets of pure lambdas. The most general property only requires a single one to compute the property value from an input business-model.

This allows for some very nice things, especially with regards to API-level functionality like support for nested ?expand=, ?only=, and such niceties.

IgorPartola• 11 years ago

The big problem for me is that serializers are bidirectional. They should only work in one direction: server to client. Client to server should be either handled by a wrapper around Django form or by a separate deserializer type. Why? Take a simple example: you have a game character model which consists of things like name, color, and a set of randomly generated attributes such as strength, agility, etc. Now, you wan the server to generate the attributes, but not the name and the color. You write your code, do a POST to /characters/ and BAM: {"strength": ["This field is required."], ...}. Now you have to tell the serializer that this field is not required...

Oh, and try customizing error messages or providing customer error conditions... It's just not fun. I think this is the weakest part of DRF.

numlocked2• 11 years ago

I think it's somewhat rare to have generic relations (content type + model id) popping up this way in serializers. I tend to use generic FKs for things like "tags" or or "comments" that can apply to multiple objects. As a result, I'm almost always requesting the parent object ("post") and want the model with the generic FK serialized in that response ("post.comments"), not the other way around.

Just looking through largish DRF codebase, I'm not sure there are many places I could use this, thought it seems like a good idea if your code has a lot of these scenarios.

norbert_mate6• 11 years ago

Thanks.

mercnet• 11 years ago

For anyone looking for argument parsing and serialization with Flask, I recommend webargs (http://webargs.readthedocs.org/), marshmallow (http://marshmallow.readthedocs.org), and flask-marshmallow (http://flask-marshmallow.readthedocs.org/).

ddorian43• 11 years ago

what rest-framework do you use with flask ? or build one yourself using what components (except these 2)?

plq• 11 years ago

You can use Spyne[1]'s HttpRpc and JsonDocument protocols via Flask-Spyne[2].

You can actually use Spyne with Django as well[3] (and Spyne has a "general model serializer" that can be used with a lot less boilerplate than what's shown here).

Spyne supports a lot of protocols as well as persisting to a relational database via SQLAlchemy.

Disclaimer: I'm the author of spyne.

[1] http://spyne.io

[2] https://pypi.python.org/pypi/Flask-Spyne

[3] https://github.com/arskom/spyne/tree/master/examples/django/...

languagehacker• 11 years ago

Nice to see Django Rest Framework getting some press, but I don't understand how this provides value beyond the existing documentation on readthedocs.

norbert_mate6• 11 years ago

Hi languagehacker, this is not science fiction. It just shows you a way of implementing a serializer that works for all kind of models in Django Rest Framework.

Django Rest Framework general model serializer

Related Stories