How does Django's ORM manage to fetch Foreign objects when they are accessed

Been trying to figure this out for a couple of hours now and have gotten nowhere.

class other(models.Model): user = models.ForeignKey(User) others = other.objects.all() o = others[0]

At this point the ORM has not asked for the o.user object, but if I do ANYTHING that touches that object, it loads it from the database.


will cause a load from the database.

What I want to understand is HOW they do this magic. What is the pythonic pixie dust that causes it to happen. Yes, I have looked at the source, I'm stumped.

-------------Problems Reply------------

Django uses a metaclass (django.db.models.base.ModelBase) to customize the creation of model classes. For each object defined as a class attribute on the model (user is the one we care about here), Django first looks to see if it defines a contribute_to_class method. If the method is defined, Django calls it, allowing the object to customize the model class as it's being created. If the object doesn't define contribute_to_class, it is simply assigned to the class using setattr.

Since ForeignKey is a Django model field, it defines contribute_to_class. When the ModelBase metaclass calls ForeignKey.contribute_to_class, the value assigned to ModelClass.user is an instance of django.db.models.fields.related.ReverseSingleRelatedObjectDescriptor.

ReverseSingleRelatedObjectDescriptor is an object that implements Python's descriptor protocol in order to customize what happens when an instance of the class is accessed as an attribute of another class. In this case, the descriptor is used to lazily load and return the related model instance from the database the first time it is accessed.

# make a user and an instance of our model
>>> user = User(username="example")
>>> my_instance = MyModel(user=user)

# user is a ReverseSingleRelatedObjectDescriptor
>>> MyModel.user
<django.db.models.fields.related.ReverseSingleRelatedObjectDescriptor object>

# user hasn't been loaded, yet
>>> my_instance._user_cache
AttributeError: 'MyModel' object has no attribute '_user_cache'

# ReverseSingleRelatedObjectDescriptor.__get__ loads the user
>>> my_instance.user
<User: example>

# now the user is cached and won't be looked up again
>>> my_instance._user_cache
<User: example>

The ReverseSingleRelatedObjectDescriptor.__get__ method is called every time the user attribute is accessed on the model instance, but it's smart enough to only look up the related object once and then return a cached version on subsequent calls.

This will not explain how exactly Django goes about it, but what you are seeing is Lazy Loading in action. Lazy Loading is a well known design pattern to defer the initialization of objects right up until the point they are needed. In your case until either of o = others[0] or type(o.user) is executed. This Wikipedia article may give you some insights into the process.

Properties can be used to implement this behaviour. Basically, your class definition will generate a class similar to the following:

class other(models.Model):
def _get_user(self):
## o.users being accessed
return User.objects.get(

def _set_user(self, v):
## ...

user = property(_get_user, _set_user)

The query on User will not be performed until you access the .user of an 'other' instance.

Category:python Views:0 Time:2010-08-30
Tags: python orm django

Related post

  • What is the best way to access stored procedures in Django's ORM 2009-04-30

    I am designing a fairly complex database, and know that some of my queries will be far outside the scope of Django's ORM. Has anyone integrated SP's with Django's ORM successfully? If so, what RDBMS and how did you do it? --------------Solutions-----

  • Is there a way to make a select faster by making it read only in django's ORM? 2010-09-19

    If I am just getting a list of users, and I have no intention of updating this list, do I have the option of marking the query as 'read only' somehow? Reason being, I know most ORM's keep some sort of change tracking on the rows returned. So if I kno

  • Accelerate bulk insert using Django's ORM? 2010-11-27

    I'm planning to upload a billion records taken from ~750 files (each ~250MB) to a db using django's ORM. Currently each file takes ~20min to process, and I was wondering if there's any way to accelerate this process. I've taken the following measures

  • Including Duplicate Tables using Django's ORM Extra() 2011-07-28

    I'm trying to implement a simple triplestore using Django's ORM. I'd like to be able to search for arbitrarily complex triple patterns (e.g. as you would with SparQL). To do this, I'm attempting to use the .extra() method. However, even though the do

  • Django ORM: caching and manipulating ForeignKey objects 2009-03-25

    Consider the following skeleton of a for a space conquest game: class Fleet(models.Model): game = models.ForeignKey(Game, related_name='planet_set') owner = models.ForeignKey(User, related_name='planet_set', null=True, blank=True) home = mo

  • Automatically create an admin user when running Django's ./ syncdb 2009-09-23

    My project is in early development. I frequently delete the database and run syncdb to set up my app from scratch. Unfortunately, this always pops up: You just installed Django's auth system, which means you don't have any superusers define

  • Rewriting "SELECT DISTINCT ON -" using Django's ORM 2010-09-19

    I am using the following model with Django: class Hit(Model): image = ForeignKey(Image) user = ForeignKey(User) timestamp = DateTimeField(auto_now_add = True) What I need is basically a list that contains the count of "first hits" (i.e. hits with no

  • How to select another DB in Django's ORM when used in a desktop application? 2011-01-19

    I'm writing a desktop application with PyQt where we planned to use sqlite3-databases for file storage (instead of pickles, XML, YAML, etc). The reason is that our application is likely to migrate to a centralized data store later. (which then needs

  • django non-rel / Managing per-field indexes on App Engine 2011-04-28

    I'm trying to move my gae webapp project to django non-rel. I'm pretty new to Python and Django (non-rel). Right now I'm setting up the admin backend. When I want to look at the history of a model entry I encounter this problem: Caught DatabaseError

  • Comparing Object Fields with Django's ORM 2011-05-04

    Is comparing columns in different tables using less-than/greater-than operators supported in Django's ORM? For example, I'm trying to compare two object fields in a Django query, which would have the SQL equivalent of: SELECT FROM mytable a LEFT

  • How do I use Django's ORM to return a group of rows only when 3 or more rows will be GROUPed? 2011-11-07

    Given the following Model: class Enquiry(models.Model): enquiryparent = models.ForeignKey('self',default=None, null=True, blank=True) type = models.SmallIntegerField() name = models.CharField(max_length=200) mobile = models.CharField(max_length=40,bl

  • What are the limitations of Django's ORM? 2012-02-02

    I've heard developers not wanting to use ORM, but don't know why. What are the shortcomings of the ORM? --------------Solutions------------- Let me start by saying that I fully advocate the use of the ORM for most simple cases. It offers a lot of con

  • When to use Haystack/ElasticSearch vs Django's ORM 2013-06-06

    So I implemented Haystack with ElasticSearch a week ago within our BETA application. One thing I can notice is that getting some data (large amount) back to our users (for example listing all the users within the application) is much faster by going

  • Querying across relationships to compare dates in Django Models or Manager 2014-08-08

    class Report(models.Model): report_name = models.CharField(max_length=255) due_date = models.DateField(db_index=True) is_hidden = models.NullBooleanField(null=True, default=False) created_at = models.DateTimeField(auto_now_add=True) updated_at = mode

  • Django: writing a manager to filter query set results 2009-12-23

    I have the following code: class GroupDepartmentManager(models.Manager): def get_query_set(self): return super(GroupDepartmentManager, self).get_query_set().filter(group='1') class Department(models.Model): name = models.CharField(max_length=128) gro

  • Django: Don't want related manager for two foreign keys to the same model 2010-02-09

    I want to get rid of two related managers in a Model because I will never need them. How can I get rid of them? This is my User Profile: class UserProfile(models.Model): user = models.ForeignKey(User, unique=True) ... default_upload_container=models.

  • Can Django's ORM output the SQL query it is using? 2011-01-21

    I know that you can output the SQL to see tables that are created. Is it possible for Django to output the sql used for any query like: Protocols.objects.filter(active=False) ? I couldn't find this in the docs, so hopefully someone can point them to

  • Fetching related objects 2012-02-22

    In Symfony 2 book there is an example of how to do that for ONE $product: It is quite simple: public function showAction($id) { $product = $this->getDoctrine() ->getReposito

  • Django: How do you delete child class object without deleting parent class object? 2012-02-24

    I have the following models (I left out def __unicode__(...) for clarity): class Person(models.Model): first_name = models.CharField(max_length=64, null=True, blank=True) middle_name = models.CharField(max_length=32, null=True, blank=True) last_name

Copyright (C), All Rights Reserved.

processed in 0.137 (s). 11 q(s)