Skip to content
Snippets Groups Projects
  • Benjamin Hättasch's avatar
    26423909
    Upgrade django and other python dependencies · 26423909
    Benjamin Hättasch authored
    Upgrade django to 4.1
    Upgrade django-timezone-field, django-simple-history, django-debug-toolbar and django-bootstrap-datepicker-plus
    Upgrade mysqlclient
    Remove obsolete pytz
    Adapt timezone handling code to new interface of TimezoneInfo (part of python since 3.9 and backport included in 3.8)
    Drop support for python 3.7 (since Django 4 requires >= 3.8)
    26423909
    History
    Upgrade django and other python dependencies
    Benjamin Hättasch authored
    Upgrade django to 4.1
    Upgrade django-timezone-field, django-simple-history, django-debug-toolbar and django-bootstrap-datepicker-plus
    Upgrade mysqlclient
    Remove obsolete pytz
    Adapt timezone handling code to new interface of TimezoneInfo (part of python since 3.9 and backport included in 3.8)
    Drop support for python 3.7 (since Django 4 requires >= 3.8)
Code owners
Assign users and groups as approvers for specific file changes. Learn more.

AK Planning: Setup

This repository contains a Django project with several apps.

Requirements

AKPlanning has two types of requirements: System requirements are dependent on operating system and need to be installed manually beforehand. Python requirements will be installed inside a virtual environment (strongly recommended) during setup.

System Requirements

  • Python 3.8+ incl. development tools
  • Virtualenv
  • pdflatex & beamer class (texlive-latex-base texlive-latex-recommended texlive-latex-extra texlive-fonts-extra texlive-luatex)
  • for production using uwsgi:
    • C compiler e.g. gcc
    • uwsgi
    • uwsgi Python3 plugin
  • for production using Apache (in addition to uwsgi)
    • the mod proxy uwsgi plugin for apache2

Python Requirements

Python requirements are listed in requirements.txt. They can be installed with pip using -r requirements.txt.

Development Setup

  • create a new directory that should contain the files in future, e.g. mkdir AKPlanning
  • change into that directory cd AKPlanning
  • clone this repository git clone URL .

Automatic Setup

  1. execute the setup bash script Utils/setup.sh

Manual Setup

  1. setup a virtual environment using the proper python version virtualenv venv -p python3.7
  2. activate virtualenv source venv/bin/activate
  3. install python requirements pip install -r requirements.txt
  4. setup necessary database tables etc. python manage.py migrate
  5. prepare static files (can be omitted for dev setups) python manage.py collectstatic
  6. compile translations python manage.py compilemessages
  7. create a priviledged user, credentials are entered interactively on CLI python manage.py createsuperuser
  8. deactivate virtualenv deactivate

Development Server

Do not use this for deployment!

To start the application for development, in the root directory,

  1. activate virtualenv source venv/bin/activate
  2. start development server python manage.py runserver 0:8000
  3. In your browser, access http://127.0.0.1:8000/admin/ and continue from there.

Deployment Setup

This application can be deployed using a web server as any other Django application. Remember to use a secret key that is not stored in any repository or similar, and disable DEBUG mode (settings.py).

Step-by-Step Instructions

  1. log into your system with a sudo user

  2. install system requirements

  3. create a folder, e.g. mkdir /srv/AKPlanning/

  4. change to the new directory cd /srv/AKPlanning/

  5. clone this repository git clone URL .

  6. setup a virtual environment using the proper python version virtualenv venv -p python3.7

  7. activate virtualenv source venv/bin/activate

  8. update tools pip install --upgrade setuptools pip wheel

  9. install python requirements pip install -r requirements.txt

  10. create the file AKPlanning/settings_secrets.py (copy from settings_secrets.py.sample) and fill it with the necessary secrets (e.g. generated by tr -dc 'a-z0-9!@#$%^&*(-_=+)' < /dev/urandom | head -c50) (it is a good idea to restrict read permissions from others)

  11. if necessary enable uwsgi proxy plugin for Apache e.g.a2enmod proxy_uwsgi

  12. edit the apache config to serve the application and the static files, e.g. on a dedicated system in /etc/apache2/sites-enabled/000-default.conf within the VirtualHost tag add:

    Alias /static /srv/AKPlanning/static
    <Directory /srv/AKPlanning/static>
    Require all granted
    </Directory>
    
    ProxyPassMatch ^/static/ !
    ProxyPass / uwsgi://127.0.0.1:3035/

    or create a new config (.conf) file (similar to apache-akplanning.conf) replacing $SUBDOMAIN with the subdomain the system should be available under, and $MAILADDRESS with the e-mail address of your administrator and $PATHTO with the appropriate paths. Copy or symlink it to /etc/apache2/sites-available. Then symlink it to sites-enabled e.g. by using ln -s /etc/apache2/sites-available/akplanning.conf /etc/apache2/sites-enabled/akplanning.conf.

  13. restart Apache sudo systemctl restart apache2.service

  14. create a dedicated user, e.g. adduser django

  15. transfer ownership of the folder to the new user chown -R django:django /srv/AKPlanning

  16. Copy or symlink the uwsgi config in uwsgi-akplanning.ini to /etc/uwsgi/apps-available/ and then symlink it to /etc/uwsgi/apps-enabled/ using e.g., ln -s /srv/AKPlanning/uwsgi-akplanning.ini /etc/uwsgi/apps-available/akplanning.ini and ln -s /etc/uwsgi/apps-available/akplanning.ini /etc/uwsgi/apps-enabled/akplanning.ini start uwsgi using the configuration file uwsgi --ini uwsgi-akplanning.ini

  17. restart uwsgi sudo systemctl restart uwsgi

  18. execute the update script ./Utils/update.sh --prod

Deployment Setup using Docker

This project also provides a docker file for easy deployment.

The container described by the docker file only contains the project itself. Additional containers for the database and webserver are needed to use it.

The following docker-compose file shows a typical usage:

version: "3"

networks:
  akplanning:
    external: false

volumes:
  static-files:

services:
  mariadb:
    image: mariadb:10
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: supermegasecrey
      MYSQL_DATABASE: akplanning
      MYSQL_USER: akplanning
      MYSQL_PASSWORD: secret
      TZ: Europe/Berlin
    networks:
      - akplanning

  akplanning-server:
    image: neumantm/akplanning:2021-03-10
    restart: always
    environment:
      SECRET_KEY: superlongandsupersecret
      DB_HOST: mariadb
      DB_USER: akplanning
      DB_NAME: akplanning
      DB_PASSWORD: secret
      HOSTS: "['akplanning.example.net', 'akplanning.example.de']"
      TZ: Europe/Berlin
      AUTO_MIGRATE_DB: 'true'
      DJANGO_SUPERUSER_USERNAME: admin
      DJANGO_SUPERUSER_EMAIL: admin@example.com
      DJANGO_SUPERUSER_PASSWORD: supersecret
      EXTRA_DJANGO_SETTING_FOO: DJANGO_FOO = True\nDJANGO_BAR = False
    depends_on:
      - mariadb
    networks:
      - akplanning
    volumes:
      - static-files:/app/static

  web-server:
    image: nginx
    restart: always
    volumes:
      - /path/to/nginx.conf:/etc/nginx/nginx.conf:ro
      - static-files:/var/www/akplanning-static
    ports:
      - "8080:80"
    depends_on:
      - akplanning-server
    networks:
      - akplanning

The nginx.conf would look like this:

user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen      80;
        server_name localhost:8080;

        location /static/ {
            alias /var/www/akplanning-static/;
        }

        location / {
            include /etc/nginx/uwsgi_params;
            uwsgi_pass uwsgi://akplanning-server:3035;
        }
    }
}

Initializing and migrating database

On the first start, the database must be initialized (the Tables created and so on). When updating the project the database must be migrated. Both are done using the migrate command.

This can be done manually by running the following command after the container has started: docker-compose exec -it akplanning-server ./manage.py migrate

It can also be done automatically on each container start by setting AUTO_MIGRATE_DB to the string true (as shown in the docker-compose file above).

Database migration may lead to the corruption or loss of data in some cases. Make sure you have a backup before running the command and be very careful with enabling auto migration.

Creating initial superuser

There are two ways to create the initial superuser when using the docker container. For both the database must have been intialized before.

The first way is already shown in the docker-compose file above: Using the environment variables DJANGO_SUPERUSER_{USERNAME,EMAIL,PASSWORD}.

The second way is to run the following command after the container has started: docker-compose exec -it akplanning-server ./manage.py createsuperuser

Extra django settings

For simple cases you can pass environment variables starting with EXTRA_DJANGO_SETTING. The content of such variables is written into python files, which are loaded as settings.

For more complex scenarios you can also mount a docker volume to /app/AKPlanning/settings and add any number of python files to the volume.

Updates

To update the setup to the current version on the main branch of the repository use the update script Utils/update.sh or Utils/update.sh --prod in production.

Afterwards, you may check your setup by executing Utils/check.sh or Utils/check.sh --prod in production.

Updating when using docker

To update when using docker, just switch the tag of the image for akplanning-server. Then (if AUTO_MIGRATE_DB is not enabled), do a database migration as described in Initializing and migrating database