🇪🇦 Leer en Español, “GenList“

Previously we talked about CodenerixModel to understand how a functional model is built with CODENERIX; however, for everything to work in Django, in addition to models you also have to define views. To allow web content to be viewed, a set of inheritable views is available to the programmer, all based on Django‘s Generic Views, which, following the CODENERIX policy, are as simple to use as to inherit from them.

In this article, we will talk about GenList, which is the generic view of CODENERIX that allows you to view data as a list from the model. GenList inherits directly from Django‘s ViewList, so all properties of a ViewList are available in GenList. Let’s see what the list of a “Contact” model would look like:

from codenerix.views import GenList
from app.models import Contact

class ContactList(GenList):
    model = Contact

As simple as that, and the visual result would be equivalent to something like this (if you use the CODENERIX Examples):

In this way we have managed to visualize the Contact model of the “agenda” example that you have here:

class Contact(CodenerixModel):
    first_name = models.CharField(verbose_name=_(u'Name(s)'), max_length=128)
    last_name = models.CharField(verbose_name=_(u'Last name'), max_length=128, blank=True, null=True)
    alias = models.CharField(verbose_name=_(u'Alias'), max_length=32, blank=True, null=True)
    organization = models.CharField(verbose_name=_(u'OrganizaciĂłn'), max_length=64, blank=True, null=True)
    borndate = models.DateField(verbose_name=_(u'Cumpleaños'), blank=True, null=True)
    address = models.TextField(verbose_name=_(u'DirecciĂłn'), blank=True, null=True)
    created_by = models.ForeignKey(User, verbose_name=_(u'Creado por'), on_delete=models.CASCADE, related_name='contacts')

    def __fields__(self, info):
        return (
            ('first_name', _(Name(s)')),
            ('last_name', _(Last name')),
            ('alias', _(u'Alias')),
            ('organization', _(u'Organization')),
        )

You may have already noticed that CODENERIX has used the __fields__ method of the model to discover which fields it should display in the list. In addition, it felt free enough to automatically offer a list that enables cumulative ascending/descending sorting in each of its fields (see “Last name” column in the following screenshot) and if we press the filter icon (light blue colour in the following screenshot ) we can even see that it has also taken care of allowing us to search in each field one by one (see “Organization” column where we search for “Inc” limiting the list to 2 results):

In short, by inheriting from GenList we have achieved a great job with an excellent visual result for our users.

But GenList, like CodenerixModel, allows you to use the methods: __fields__, __limitQ__, __searchF__ and __searchQ__. In this way, when one of these methods is created in a GenList, the CodenerixModel or GenList method will be used as they exist. In fact, its operation is exactly the same as in CodenerixModel.

The following 3 tables show 3 columns:

  • The first indicates if the method is defined in CodenerixModel.
  • The second indicates if it is defined in GenList.
  • The third column indicates the result of applying these filters.

__fields__
CodenerixModelGenListApplicable
NONO— ERROR —
NOYES— ERROR —
YESNOCodenerixModel
YESYESGenList

__limitQ__
CodenerixModelGenListApplicable
NONONo filter is applied
NOYESGenList filter is applied
YESNOCodenerixModel filter is applied
YESYESGenList and CodenerixModel filters
are applied to put them together
with the logic operator “AND“.

__searchQ__ & __searchF__
CodenerixModelGenListApplicable
NONONo filter is applied
NOYESGenList filter is applied
YESNOCodenerixModel filter is applied
YESYESGenList filter is applied

So now we can decide that our view behaves differently from our model. This is done because sometimes we want to create more than one list of the same model where each list has different properties, shows different information depending on the user, or uses a different template.

GenList can also contain different attributes that allow you to configure its operation in many different ways. Here you have a table of what exists today:

Table of attributes allowed by GenList

AttributeDescription
annotationsIt is possible to introduce aggregators inside the Querysets used by CODENERIX (learn more about aggregators in Django). It is very useful when we want it to calculate maximums, minimums, averages, or simply group results by date, types, or other fields. Annotations may also work as a method of the class (see example):

annotations = { 'min_price': Min('books__price'), 'max_price': Max('books__price') } def annotations(self, info): anot = [] if info.user.is_superuser: anot['min_price'] = Min('books__price') anot['max_price'] = Max('books_price') return anot
appnameOverrides the automatic system to generate URLs so you can set your own app’s URL. This is commonly used to manage the same model over to different apps (for example several GenList). Be careful. This field alters the normal behaviour of CODENERIX, which can confuse you if used wrongly. You can find a fully working example on GitHub.
autofilteringIf set to False, it disables the automatic filter generation system. By default it is True.

autofiltering = False
client_contextIt is a dictionary and its content will be sent directly in the JSON response inside the “meta” structure as “context“. It is very useful to send data directly from the view to the AngularJS controller or template. Usage:

client_context = {"var1": "value1"}

The result will be embedded into the “meta” structure:
{ "filter": ..., "meta": { ... "context": { "var1": "value1", }, ... }, "table": ..., }
default_orderingThe attribute can be a string or a list of strings. The string may or may not start with “-”. This field indicates the field or fields that are going to be used in the sorting and the symbol “-” indicates that the sorting will be done in descending order (the reverse direction), if nothing is indicated it will be done in ascending order. If a list is given, the ordering will be applied scrupulously following the order in which the fields appear in this list.

default_ordering = '-name' default_ordering = ['-name', 'date', '-xz']
default_rows_per_pageUsed to indicate the number of rows per page that we want to display by default in this list. The standard value used in GenList is 50.

default_rows_per_page = 150
exportForce the download of a list as a file. Here is the format. The default is None.

export = "xlsx"
export_excelShows the export to Excel button. By default, it is True.

export_excel = False
export_nameName of the file resulting from the export. By default, it is a “list“.

export_name = "list_file"
extends_baseIt contains the base template path from which the List template will inherit. It usually goes to a “base/base.html“, but by changing this variable, you can tell it to load a different file to offer the user a different environment experience.

extends_base = "base/base.html"
extra_contextIt contains a dictionary that will be mixed with the final context that will be delivered to the render. During the GenList process, you can override the variables used here.

extra_context = { "variable1": "This is a CODENERIX variable that will be ready to use in the Template's Context", "variable2": { "a": 1, "b": 2}, "variable3": 3.4, }
field_checkActivates the display of the checkbox in the lists, it is a checkbox that when checked stores the pk (primary key) of the row and can be used in the AngularJS‘s $scope to do custom operations. This attribute has 3 possible states:

  • None: the checkbox is not displayed.
  • False: the checkbox is shown unchecked.
  • True: the checkbox is shown with everything checked.

field_check = True
field_deleteIt activates the display of the delete field in the lists, it is the icon of a trash can and when you click on it, the user is sent to delete the said record. By default it is False.

field_delete = True
haystackActivate support for Haystack in this listing. By default it is False.

haystack = True
jsonWhen this attribute is True, the view will always respond in JSON format. If the view responds in JSON format and you want to modify the response, you can work on the view’s json_builder() method, rewriting it or simply reprocessing its result. We must know that when the views receive a “json” parameter in GET/POST, the JSON response will automatically be activated. This attribute will also be activated when an “HTTP_X_REST” header is received and its value can be evaluated as a Boolean “True“. It is important to know that this attribute is set to True by default.

json = True
linkaddShows the “Add” button in the listings. By default, it is True.

linkadd = False
linkeditClicking on a list row goes to the editing mode of that specific record. By default it is True.

linkedit = False
modelModel to use.

model = Contact
modelnameOverrides the automatic system to generate URLs so you can set your own model’s URL. This is commonly used to manage the same model over different URLs (for example, several GenList). Be careful, this field alters the normal behaviour of CODENERIX which can confuse you if used wrongly. You can find a fully working example on GitHub.
must_be_staffWhether or not the user must be staff (ability to see /admin/ of Django according to the User model) to view this list content, its value is False by default.

must_be_staff = True
must_be_superuserWhether or not the user must be a superuser (according to Django’s User model) in order to view the content of this listing. By default its value is False.

must_be_superuser = True
ngincludesYou keep control over possible ng-includes that may appear in partials.

ngincludes = {'name':'path_to_partial'}
onlybaseIt will cause GenList to act only as a base file for another view. By default, it is False.

onlybase = True
permissionIt can be a string or a list of strings. Each string represents one permission. The user can see the list when they have at least one of the permissions in this attribute.

permission = 'permission1' permission = ['perm1', 'perm2', ...]
permission_groupIt can be a string or a list of strings. Each text string represents one group’s permission. The user will be able to see the list when any of the user’s groups have any of the permissions that appear in this attribute.

permission_group = 'permission1' permission_group = ['perm1', 'perm2', ...]
search_filter_buttonIf it is False, it disables the field filter button. By default it is True.

search_filter_button = False
show_detailsClicking on a list row goes to the detail mode of that specific record. By default it is False.

show_details = True
show_modalWhen on, it pushes the system to render in a modal window.

show_modal = True
static_app_rowLoad the AngularJS application file indicated here, otherwise use the default: codenerix/js/apps.js

static_app_row = "app/models_apps.js"
static_controllers_rowIt loads the AngularJS controller file indicated here. Otherwise, it does not load any.

static_controllers_row = "app/models_controllers.js"
static_filters_rowLoad the AngularJS filter file indicated here, otherwise use the default one: codenerix/js/rows.js

static_filters_row = "app/models_rows.js"
static_partial_headerCauses the indicated table “header partial” to be loaded, if not specified there will be no header partial.

static_partial_header = "app/models_header.html"
static_partial_rowCauses the row partial specified in this attribute to be loaded instead of the default partial: codenerix/partials/rows.html

static_partial_row = "app/models_rows.html"
static_partial_summaryCauses the “tail partial” of the table specified in this attribute to be loaded instead of the default partial: codenerix/partials/summary.html

static_partial_summary = "app/models_summary.html"
template_baseIt is the template that acts as a base for the rest. GenList generally renders through a series of templates that jump from one to another by inheritance. In the last step, the CODENERIX templates call a base template that is usually “base/base”, however, this can be changed with this attribute so that it uses another base template. Remember that CODENERIX will try to locate the rest of the name of the “web” file using the template detection algorithms.

template_base = 'frontend/web'
template_base_extExtension used for template_base.

template_base_ext = 'html'
template_modelThis is the template used as the entry point for the CODENERIX renderer. Generally, if the path doesn’t exist it will end up using “codenerix/list” (which will end up being “codenerix/list.html”) however it is possible to define our own entry point using this attribute. This is very useful when we want to add “extra_css” or “extra_js” in the rendering, and our base template supports these blocks.

template_model = 'app/contact_list'
template_model_extExtension used for template_model.

template_model_ext = 'html'
userAllows you to make a view behave specifically as a specific user. It is very useful when you want to view the list as if you were another user of the system. It’s a Django User object that will be used by all processes in the view and passed to all methods that request it.

IMPORTANT: the permissions management system is not affected by this variable. That is, it will continue to receive the user who has requested to view the list, and the access restriction policies will be applied to this user, not to the one that appears in this attribute.

user = User.objects.filter(is_superadmin=False).first()
vtableVTable is a technology by which the list becomes non-paged and CODENERIX does a proactive job to estimate the real width of the list on screen and virtualizes the environment in such a way that it preloads a few pages in the browser but not all and adjusts the scroll the same as it would be if all the records were loaded this way when we move in the list CODENERIX loads new pages and unloads old ones to allow the browser to work at an adequate rendering speed, but the user perceives the sensation of being working on a huge list and that all the records are there. This functionality is experimental because it has not been tested enough. By default this attribute is False.

vtable = True
ws_entry_pointThis attribute defines the route to the entry point for the operations that the list wants to carry out, for example, when loading the list, the route indicated by the “ws_entry_point” is used, and when editing its use is repeated. It is generally used in extra views that are added on top of an existing one on the model and when you want to introduce a new Angular controller or something else that modifies the standard behaviour of CODENERIX.

ws_entry_point = "planner/plane"

In addition to these attributes, we must know how GenList tries to locate the templates for rendering the page. GenList will try to build a series of URLs based on the user’s profile and language so that if the user is superadmin he will have an “admin” profile, and otherwise, he will not have any profile. The user’s language is detected by CODENERIX and will also be delivered to the template detection function.

The possible routes for the templates will be (following this order of priority):

  1. <path>/<file>.admin.<language>.html
  2. <path>/<file>.admin.html
  3. <path>/<file>.<username>.<language>.html
  4. <path>/<file>.<username>.html
  5. <path>/<file>.<language>.html
  6. <path>/<file>.html

In this way, if the user is “peter“, he is a superuser, and his language at the time of the query is English with code “en“, and the path to the file is “base/home“, the system will test the existence of the following files:

  1. base/home.admin.en.html
  2. base/home.admin.html
  3. base/home.peter.en.html
  4. base/home.peter.html
  5. base/home.en.html
  6. base/home.html

GenList also supports automatically detecting static files for loading partials and AngularJS files such as application files, controllers, or filters. To do this, GenList will try to locate these files in the application folder inside “/static” (or whatever the path of static files on disk is according to the STATIC_ROOT constant in Django‘s settings). For this detection, it will use the same algorithm that template detection uses. In this way, it will test for each static file the possible existence of this on disk and will load these instead of those that exist by default in CODENERIX. To generate the routes, the system will use the following:

  1. The path: it will be the path of the static files, the name of the APP where the model is located, and the name of the model ending in “s“.
  2. File type: it can be:
    • <none>: for the partial that declares the filters to display above the listing table, and generally to include the table with nginclude.
    • rows: for the partial of the rows
    • header: for the partial of the table header (below the declaration of the columns and above the rows).
    • summary: for the footer partial of the table (below the rows and above the closing table tags).
    • app: for application files.
    • controllers: for the controllers.
    • filters: for AngularJS filter files.
  3. The extension: will be that of each type of file:
    • html: for partials
    • js: for the rest
  4. The final result will be from the concatenation of the following:
    • The route
    • _” plus file type
    • The extension.
  5. For an app called “base“, with a model called “Contact“, to obtain the partial rows of a user “peter“, who is a superuser and visits the site in English, the system would try the following routes:
    • static/base/contacts_rows.admin.en.html
    • static/base/contacts_rows.admin.html
    • static/base/contacts_rows.peter.en.html
    • static/base/contacts_rows.peter.html
    • static/base/contacts_rows.en.html
    • static/base/contacts_rows.html
    • static/codenerix/partials/rows.html (if the above fails)


Finally, we must talk about the json_builder() method since it is a GenList method that allows you to control the result that will be offered to the user. The method is defined with 2 parameters:

  • answer: contains the answer pre-calculated by GenList, which only lacks the “body” that should be placed in answer[“table”][“body”]
  • context: contains the context available for rendering the JSON generator in this case. Within the context is the “object_list” containing the QuerySet resulting from applying the filters and what GenList should theoretically respond to.

There are 3 techniques to work with json_builder():

Technique 1: “Body building on your own”

You build the entire body of the answer yourself:

def json_builder(self,answer,context):
    # The body is a list of records (body or body of the list)
    body=[]

    # Process each element of the context's object_list in order to process all rows
    for or in context['object_list']:
        t={}
        t['name']=o.name
        t['surname']=o.surname
        phones=[]
        for o2 in o.phone.all():
            phone={}
            phone['country']=o2.country
            phone['prefix']=o2.prefix
            phone['number']=o2.number
            phones.append(t2)
        t['phone']=phones
        t['address']=o.address
        body.append(t)

    # Insert the body into the response body
    answer['table']['body']=body

    # Returns the ready response
    return answer

In this example, we build a list manually where we are responsible for filling each row or record of the body. When we finish processing the rows that we want to send to the user’s browser, we load these rows (list) in the response and return them.

Technique 2: “Body building with bodybuilder()”

Use the bodybuilder() method to help you build the body of the response:

def json_builder(self,answer,context):
    # We fill directly using bodybuilder
    answer['table']['body']=self.bodybuilder(context['object_list'],{
        'id:user__register__id':None,
        'name': None,
        'surname':None,
        phone: {
            'country':None,
            'prefix':None,
            'number':None,
            },
        'address':None,
        })

    # Answer the new context
    return answer

When bodybuilder() is used, it expects to receive the list of objects to process and the form of the records it should build. What bodybuilder() does is that for each record of the object_list it generates an entry in the body, where each of those entries is dictionaries that have the form of the second parameter of the call, in this way each record of this example would have 5 keys in the dictionary:

  • id: would be the result of extracting the “user” field from the object and thus the “register” field and, from this, the “id” field. If “id:…” was not specified as an alias, the key would contain the full path, which would be “user__register__id“.
  • name: the result of extracting the name field from the object or the result of calling the name() method.
  • surname: the result of extracting the surname field from the object or the result of calling the surname() method.
  • phone: it will be a dictionary with 3 keys:
    • country: the result of extracting the phone field from the object, and from it the country field or the result of calling the country() method.
    • prefix: the result of extracting the phone field from the object, and from it the prefix field or the result of calling the prefix() method.
    • number: the result of extracting the phone field from the object, and from this the number field or the result of calling the number() method.
  • address: the result of extracting the address field from the object or the result of calling the address() method.

Technique 3: “Body building with autorules() and bodybuilder()”

Where you use autorules() and bodybuilder() to build the response:

def json_builder(self,answer,context):
    # Request the rules to autorules
    rules=self.autorules()
    # Get one of the keys from autorules (field we don't want in the response)
    rules.pop('id')
    # Add another key (field we want in the response, in this case an alias of user->id)
    rules['id:user__id']=None
    # Call bodybuilder the new set of rules
    answer['table']['body']=self.bodybuilder(context['object_list'],rules)

    # return the result
    return answer

In this case, we ask GenList for the rules we should use. We adopt that set of rules as we wish and deliver it to bodybuilder() so that it finishes the job of generating the response body.

Notes on the query optimizer

We must consider that GenList includes a query optimizer to reduce the work done in Python and pass part of it to the database manager. This optimizer does not work when data is involved in the result that the database cannot compute on its own, that is, generally, when methods are included in the results and they are not purely data stored in the objects.

Another important detail is that the optimizer converts the result rows into dictionary objects so that it only contains the data as-is and does not instantiate the model objects as would be normal in Django.

Custom QuerySet or customized QuerySet

Finally, when we cannot work with the filters or when introducing certain limitations (annotates for example) the result no longer works as we wish….we will always have an “ACE up our sleeve“, the method “custom_queryset(queryset, info)”. This method must be rewritten in your view in order for CODENERIX to use it since. Otherwise, it will operate normally. This method receives two parameters: the first is the QuerySet calculated by GenList, and the second is the MODELINF object that contains internal information about the query, the model, arguments, etc… and expects you to return a valid QuerySet. This is why you can return the QuerySet you received with a few more parameters taking advantage of the power of Django’s ORM, or it could even be a QuerySet from a different model.