Skip to content

README.md

Latest commit

 

History

History
288 lines (202 loc) · 8.78 KB

README.md

File metadata and controls

288 lines (202 loc) · 8.78 KB
Raw

Boost.org Website

Overview

A Django based website that will power a new Boost website. See the documentation for more information about maintaining this project.

Links:

Local Development Setup

This project will use Python 3.13, Docker, and Docker Compose.

There are two options for development setups, nix and native. The native setup doesn't require nix to be installed. The nix-based setup works in a similar way to a python venv, with everything encapsulated for the project. Note: it is not a vm. It handles installation of all of the application and development environment's dependencies automatically:

  • The python version relevant to the project
  • awscli
  • gdk
  • just
  • opentofu
  • nodejs
  • yarn
  • asciidoctor
  • asciidoctor-boost gem
  • pre-commit
  • black
  • isort
  • pip-tools
  1. Development System setup
    1. Give your ssh key to Sam so he can add it to the boost.cpp.al server.
    2. Basic Setup Options:
      1. Nix-based
      2. Native
    3. Initial Data Setup
    4. Allauth Social Login Setup
  2. System Concepts
    1. GitHub Data Syncing
    2. Rendered Content Caching
    3. Static Content from S3
    4. 'manage' Commands
    5. API
    6. RSS Feeds
    7. Env Vars
    8. Calendar
    9. News
  3. Instructional
    1. Dependency Updates
    2. Release Reports
    3. User Management
    4. Non-Dev Server Allauth Setup
    5. Admin Features
    6. Mailing List Setup
    7. Connecting to Servers

After going through the "Development System setup" steps above to create the Docker image, install dependencies, and start the services in docker-compose.yml, run:

# start our services (and build them if necessary)
$ docker compose up

# to create database migrations
$ just makemigrations

# to run database migrations
$ just migrate

# to create a superuser
$ just manage createsuperuser

styles.css is still missing in a local docker-compose environment. Steps to add it:

# Each time - rebuild styles.css
yarn
yarn build  # or on windows: yarn dev-windows
cp static/css/styles.css static_deploy/css/styles.css

Access the site at http://localhost:8000.

Cleaning up

To shut down our database and any long running services, we shut everyone down using:

$ docker compose down

🔔 Updating the Docker image with new dependencies

If new dependencies exist in requirements.in, see Dependency Management for details on how to rebuild the Docker image with those new dependencies.

Running the tests

To run the tests, execute:

$ just test

or run:

$ docker compose run --rm web pytest

Yarn and Tailwind

To install dependencies, execute:

$ yarn

For development purposes, in a secondary shell run the following yarn script configured in package.json which will build styles.css with the watcher.

$ yarn dev

For production, execute:

$ yarn build

This is a one-time build that will generate the styles.css file. This file is currently generated by docker compose build and is included in the Docker image.

Setting up Mailman/Hyperkitty locally

Note: This is optional - not required for the website to function, here for reference in case a need arises.

shell sudo apt-get install sassc git clone git@gitlab.com:mailman/hyperkitty.git cd hyperkitty cp example_project/settings.py example_project/settings_local.py pip install -e '.[dev]' pip install psycopg2-binary change settings.py to use postgres database: 'ENGINE': 'django.db.backends.postgresql_psycopg2', Update database values in settings to use the same host, user, password, and the database name as in the .env file value for HYPERKITTY_DATABASE_NAME (lists_production_web by default).

run django-admin migrate --pythonpath example_project --settings settings

Give your ssh key to Sam so he can add it to the boost.cpp.al server, and then download the mailman db archive and cp the sql to the docker container

Create a database in your postgres instance called hyperkitty_db, then:

scp {user}@staging-db1.boost.cpp.al:/tmp/lists_stage_web.staging-db1-2.2025-02-06-08-00-01.sql.gz .
docker cp lists_stage_web.staging-db1-2.2025-02-06-08-00-01.sql website-v2-web-1:/lists_stage_web.staging-db1-2.2025-02-06-08-00-01.sql
docker exec -it website-v2-web-1 /bin/bash
apt update && apt -y install postgresql
psql -U postgres -W hyperkitty_db < /lists_stage_web.staging-db1-2.2025-02-06-08-00-01.sql

Syncing EmailData Locally

To work with mailinglist data locally, the django application expects to be able to query a copy of the hyperkitty database from HYPERKITTY_DATABASE_NAME. Then, just manage sync_mailinglist_stats management command can be run.

Syncing Large Static Images

Large static images and other assets are stored in S3 buckets rather than in the repository. Use the scripts/sync-large-static-images.sh script to manage these files.

Prerequisites

  • awscli must be installed.

  • Configure a set of credentials in your ~/.aws/credentials file with the profile name sync-boost-images:

    [sync-boost-images]
aws_access_key_id = <your_key_id>
aws_secret_access_key = <your_secret_key>

Usage

The script supports both uploading and downloading files.

Uploading

To upload files from your local directory (static/static-large/) to the default S3 bucket:

$ just up_sync_images

To upload to all S3 buckets:

$ just up_sync_images_all_buckets

Downloading

To download missing or outdated static items from the staging bucket to your local directory:

$ just down_sync_images

Deploying

TBD

Production Environment Considerations

TBD

Pre-commit Hooks

We use pre-commit hooks to check code for style, syntax, and other issues. They help to maintain consistent code quality and style across the project, and prevent issues from being introduced into the codebase.

Pre-commit Hook Description
Black Formats Python code using the black code formatter
Ruff Wrapper around flake8 and isort, among other linters
djlint Formats and lints Django HTML templates

Example commands for running specific hooks:

Hook Example
Black pre-commit run black
Ruff pre-commit run ruff
djlint pre-commit run djlint-reformat-django

Formatting

We use Prettier for JS/CSS and djlint for HTML templates. Both are checked in CI via yarn format:check.

Command Description
just format Format all JS, CSS, and HTML files
just format-check Check formatting without modifying files
just prettier Format JS and CSS only
just djlint Format HTML templates only

djlint quirks

  1. Multi-line Alpine.js attributes: djlint will mangle multi-line x-data, x-init, or similar Alpine.js attributes that contain JavaScript objects or functions. Wrap them with {# djlint:off #} / {# djlint:on #} to preserve formatting:
<div class="my-component"
  {# djlint:off #}
  x-data="{
    open: false,
    toggle() { this.open = !this.open }
  }"
  {# djlint:on #}
>

Simple single-line attributes like x-data="{ show: true }" do not need wrapping.

  1. <style> inside {% comment %} blocks: djlint parses <style> tags even inside Django {% comment %} blocks, which breaks indentation for the rest of the file. Avoid writing <style> in comment blocks — use backticks (e.g. `style`) or plain text instead.

Vendor JS

Third-party libraries and generated bundles live in static/js/vendor/ and are excluded from Prettier. Only custom project JS files at static/js/*.js are formatted. When adding a new vendor library or generated bundle, place it in its own subfolder under static/js/vendor/.

Git blame

Bulk formatting commits are listed in .git-blame-ignore-revs so they don't show up in git blame. To enable this locally:

git config blame.ignoreRevsFile .git-blame-ignore-revs