Django

Code

Ticket #1453 (new)

Opened 3 years ago

Last modified 1 year ago

generic views sometimes inefficient with large data sets - especially archive_index

Reported by: hugo Assigned to: nobody
Milestone: Component: Generic views
Version: Keywords:
Cc: Triage Stage: Someday/Maybe
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

archive_index will give a list of years with available data to the template in the context. To discover these years, it will do the following SQL:

SELECT DATE_TRUNC(\'year\', "logviewer_message"."time")
FROM "logviewer_message"
WHERE ("logviewer_message"."channel_id" = 2 AND 
       "logviewer_message"."time" <= 2006-03-02 12:07:20.758531)
GROUP BY 1 ORDER BY 1 ASC

This is rather icky if you have very much data in your database. For example the IRC logger currently has over 100000 messages stored for the #django channel. It will do an index scan for the timestamp - but that is allways "now" in archive_index, so it won't help much. It will essentially dig through all rows sequentially.

I think either this list of years should be thrown out or should be made optional (add a with_list_of_years parameter to the generic view that people can set if they need this behaviour).

Attachments

Change History

03/02/06 12:28:42 changed by Esaj

How about putting an object in the context which can lazily get a list of years or months? I think it's more KISS than adding another parameter :) Getting a list of months in archive_index would actually be quite useful, in particular for blogs.

02/22/07 01:08:26 changed by Simon G. <dev@simon.net.nz>

  • stage changed from Unreviewed to Design decision needed.

12/02/07 12:18:52 changed by jacob

  • stage changed from Design decision needed to Someday/Maybe.

Add/Change #1453 (generic views sometimes inefficient with large data sets - especially archive_index)




Change Properties
Action