Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model is not selecting correct shard_group #65

Open
irttboopathy opened this issue Aug 26, 2017 · 5 comments
Open

Model is not selecting correct shard_group #65

irttboopathy opened this issue Aug 26, 2017 · 5 comments

Comments

@irttboopathy
Copy link

irttboopathy commented Aug 26, 2017

When the model_config is configured like below, the router is always sending None from method db_for_read. It uses hints to get the shard, but its always {exact_lookups = {}}

@model_config(shard_group='default', sharded_by_field='model_id')

Please help resolving the issue.

Right now Model am using is,

@model_config(database='default')
class ShardedCarIDs(TableStrategyModel):
    pass

@model_config(shard_group='default')
class Car(models.Model):
    another_id = TableShardedIDField(primary_key=True, source_table_name=ShardedCarIDs)
    ignition_type = models.CharField(max_length=120)

    def get_shard(self):
        print 'get shard method'
        return 'app_shard_001'
        # from django.contrib.auth import get_user_model
        # return get_user_model().objects.get(pk=self.user_pk).shard

    @staticmethod
    def get_shard_from_id(user_pk):
        return 'app_shard_001'

This is just for example, returning the shard db name.

@JBKahn
Copy link
Owner

JBKahn commented Aug 26, 2017

What's an example of a call causing this? Car.objects.? What call causes it to return None?

I also updated your question formatting for it was a bit easier for me to read. Let me know if I fudged anything.

Also source_table_name is supposed to be a string path to the model.

@irttboopathy
Copy link
Author

Am calling Car.objects.all()

I might be wrong somewhere. In example @model_config(sharded=True) is not working.. Can you send links some working example so that it would be helpful for me.

@JBKahn
Copy link
Owner

JBKahn commented Aug 27, 2017

You can see some working examples in the test code in this repository. You can't use Car.objects.all() because that would require Django to call each of your databases to get all the cars, something that is not done for you. You have to tell it which database you want to read from in this case, i.e. Car.objects.using(first_database_name).all().

@irttboopathy
Copy link
Author

Yeah.. I thought model's method get_shard before querying automatically in the framework. Anyways this works..

But IMHO if this method is called automatically from the framework, it would be too good..

Thanks.. 👍

@JBKahn
Copy link
Owner

JBKahn commented Aug 27, 2017

It would have to know what you wanted to do to do that step for you. Or automatically always look into all the shards. The package is meant to give you application level sharding, putting that control in your hands. Unlike something more akin to https://www.citusdata.com/ which is transparent to your application and you'd treat like a single database.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants