[patch] multi-select admin filter for RelatedFields

A couple of thoughts from reading through this patch (I have not tested it very thorougly yet):

1. Some extra changes appear to have snuck in. In particular, the change in contrib/auth/models.py does not look related. Can you check that you are only including changes relevant to this ticket in the diff?
2. In db/models/fields/related.py, your __repr__() method shoudl really be a __str__() (since it is returning a human-readable version). The problem is that it now does not tell you what the class is, so in general it is less useful. You could create a __str__ method and use that in the place where you want it (in management.py)
3. Probably should remove the commented code in management.py in the final version. I realise you are explaining your logic at the moment, which is good (it certainly helped me). And then the 'else' clause with the nested 'if' and a 'pass' can just become 'elif', which looks a bit easier to read.
4. Can you explain the comment in about COUNT(*) in db/models/query.py a bit more? It doesn't seem appropriate for final code, but it would be good to know what needs to be avoided. Is it more complicated than needing to do COUNT(DISTINCT(id_column)) there?
5. In db/models/query.py, it looks like you are adding a general option for grouping to the extra() args that can be passed to a QuerySet? Is this correct? If so, it might be worth separating that out and getting it right first (multiple grouping clauses and merging extra args with existing grouping are two areas it looks a little fragile at the moment) with tests, rather than as part of this patch.
6. In a couple of places you have some_string.split(LISTVALUE_SEPARATOR). Isn't this going to fail badly if some of the values are strings contain commas? Or can things only be numeric ids at this point (which does not seem right, since non-numeric primary keys are permitted in Django)? Normally you need to split on commas and then work out how they would be escaped and rejoin if the tail of one fragment is the escape character(s). The escaping character can vary slightly between databases, too, from memory. Need to check how Django is doing this generically so that we can do it in a portable fashion.

Anyway, nice work. The code looks like the right idea to me. Not sure if I'll use it myself, but some people might find it useful. I have no idea if the maintainers want to accept this idea or not (might be a good idea to get an indication on django-developers before going too much further in case it's a flop).

I'd be interested in knowing if you are trying to add a generic 'grouping' section to QuerySet.extra(), because we should then find out if Adrian et al want to add it and get it completely general, if so.

Point 1 is also valid for 2 and 3, these changes are unrelated to the actual ticket, it is just that these snuck in while making the screenshots. I will remove the changes for contrib/auth/models.py, core/management.py and db/models/fields/related.py entirely.

Points 4 through 6 are spot-on and I'll look into that when I have a bit of time (which should be within a day or two). This patch+ticket was mainly intended to see if this idea is acceptable, and if so, if it is going in the right direction.

The thread is on django-dev as http://groups.google.com/group/django-developers/browse_thread/thread/3b209e762f09ac8f

Getting back to point 4, why not a COUNT(*) or COUNT(DISTINCT(*)).

The usual sql statement (simplified, leaving out the joins) looks like

  select count(*)
  from auth_user_groups
  where group_id in (2,3)

This returns one row with the correct count. Note that the in(2,3) means that there are duplicates in the resultset, so a count(distinct(*)) and later a select distinct are necessary.

Now, to implement the and/or functionality, I have modified the sql to look like

  select count(*)
  from auth_user_groups
  where group_id in (2,3)
  group by user_id
  having count(user_id)>= X

with X=1 for OR, and X=len(id_list) for AND.

By using a GROUP BY, I no longer get a single result, but I get a row for each user that satisfies the criteria. The count(*) for each row is now the number of matches for the group_id in (id_list). For example, the AND case yields

user_id	count
2	2
4	2

And the OR case

user_id	count
3	1
2	2
4	2

Basically, the number of rows returned is once again the actual number of rows. So, to really tackle this without fetching the rows, I could use (but I don't do that yet) a subselect, like this

  select count(*) from (
    select count(*)
    from auth_user_groups
    where group_id in (2,3)
    group by user_id
    having count(user_id)>= X
  ) as temp

That should work in most database engines (it does in Postgres), but I know older MySQL versions didn't support subselects.

I forgot to mention that the GROUP BY gets rid of the duplicates quite nicely :)

Came across this again whilst going through all the patches we have outstanding. One comment on the earlier count(*) stuff: if we ever need count(distinct(...)), not that count(distinct(*)) is not supported in SQLite. You have to do count(distint(col_name)) there. Since the primary key is unique and not NULL, it acts as a good substitute for '*'.

Also are selects in the "from" clause supported everywhere (MySQL and SQLite being ther two most likely infringers here). Haven't checked this yet, just noting it as something to resolve. Also, would we be screwing over the MS SQL Server or Firebird guys at all (Oracle does support selects in the from clause, as does PostgreSQL, so no problems with those two).

no more screenshots :(

Oops, sorry, they're back now! (hadn't realized this was still under consideration while cleaning up some stuff)

At some point, we'll be reworking the admin interface, and this will be one of the things we examine at that time.

Hi, I am very interested in this feature. The patch (i.e. the diff file) works off a really old version of django. Is it possible to update the patch to work with either django 0.96 or trunk code base? I tried to manually update the files, but query.py has changed significantly. Thanks,

Replying to mtredinnick: SQL Server can happily do SELECT in FROM