Skip to content

akvo/african-bamboo-dashboard

BranchesTags

Repository files navigation

African Bamboo Dashboard

A full-stack web application with a Next.js frontend, Django REST Framework backend, and PostgreSQL database, orchestrated with Docker Compose.

Tech Stack

Layer Technology
Frontend Next.js 15 (App Router, Turbopack), React 19, Tailwind CSS 4, Axios
Backend Django 4.2, Django REST Framework 3.16, SimpleJWT, drf-spectacular
Database PostgreSQL 13 (ltree extension)
Infra Docker Compose

Prerequisites

Getting Started

docker compose up

This starts all services:

Service URL Description
Frontend http://localhost:3000 Next.js application
Backend http://localhost:8000 Django REST API
API Docs http://localhost:8000/api/docs/ Swagger UI
PgAdmin http://localhost:5050 Database management UI

PgAdmin credentials: dev@akvo.org / password

Project Structure

├── frontend/          # Next.js 15 application
│   └── src/app/       # App Router pages and layouts
├── backend/           # Django project
│   ├── african_bamboo_dashboard/   # Project settings and root URLs
│   ├── utils/                      # Shared utilities (kobo_client, polygon, encryption)
│   └── api/v1/
│       ├── v1_init/               # Health-check / init endpoints
│       ├── v1_users/              # Auth & user management (JWT, Kobo login)
│       └── v1_odk/                # ODK data (forms, submissions, plots)
├── database/
│   ├── docker-entrypoint-initdb.d/ # DB initialization SQL
│   └── script/                     # Utility scripts (e.g., dump-db.sh)
├── docker-compose.yml              # Service definitions
└── docker-compose.override.yml     # Dev overrides (ports, pgadmin)

Development

Frontend

Commands run inside the frontend/ directory:

yarn dev       # Start dev server with Turbopack
yarn build     # Production build
yarn lint      # Run ESLint + Prettier checks

Backend

Commands run inside the backend/ directory:

python manage.py migrate                 # Apply database migrations
python manage.py createsuperuser         # Create admin user
python manage.py runserver 0.0.0.0:8000  # Start dev server

Code quality:

black .        # Format Python code
isort .        # Sort imports
flake8         # Lint (max line length: 80)

Run tests:

docker compose exec backend ./test.sh   # Full test suite with coverage

Database

Dump the database:

docker compose exec -T db pg_dump --user akvo --clean --create --format plain african_bamboo_dashboard > database/docker-entrypoint-initdb.d/001-init.sql

API

  • Versioning: URL-path based (/api/v1/...)
  • Authentication: JWT via SimpleJWT (12h access / 7d refresh tokens)
  • Pagination: LimitOffset (default page size: 10)
  • Schema: OpenAPI 3.0 auto-generated at /api/schema/
  • Documentation: Interactive Swagger UI at /api/docs/

Authentication (KoboToolbox)

All API requests (except login) require a Bearer token. Obtain one by logging in with your KoboToolbox credentials:

curl -X POST http://localhost:8000/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "kobo_url": "https://kf.kobotoolbox.org",
    "kobo_username": "your_username",
    "kobo_password": "your_password"
  }'

Use the returned token as a Bearer token:

Authorization: Bearer <token>

ODK API — Forms & Sync

The ODK API acts as a local proxy/cache for KoboToolbox:

  1. Register a form by its KoboToolbox asset_uid
  2. Configure field mappings to map form fields to plot attributes
  3. Trigger sync to fetch submissions and auto-generate plots
  4. Query locally — list, filter, and approve/reject submissions

Register a Form

curl -X POST http://localhost:8000/api/v1/odk/forms/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "asset_uid": "aYRqYXmmPLFfbcwC2KAULa", "name": "Bamboo Plot Survey" }'

Configure Field Mappings

Fetch available fields from KoboToolbox:

curl http://localhost:8000/api/v1/odk/forms/aYRqYXmmPLFfbcwC2KAULa/form_fields/ \
  -H "Authorization: Bearer <token>"

Save mappings (comma-separated for multi-value fields):

curl -X PATCH http://localhost:8000/api/v1/odk/forms/aYRqYXmmPLFfbcwC2KAULa/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "polygon_field": "consent_group/consented/boundary_mapping/Open_Area_GeoMapping",
    "region_field": "region",
    "sub_region_field": "woreda",
    "plot_name_field": "consent_group/consented/First_Name,consent_group/consented/Father_s_Name"
  }'

Sync Submissions

curl -X POST http://localhost:8000/api/v1/odk/forms/aYRqYXmmPLFfbcwC2KAULa/sync/ \
  -H "Authorization: Bearer <token>"

Response:

{
  "synced": 42,
  "created": 42,
  "plots_created": 42,
  "plots_updated": 0
}

Sync automatically creates/updates a Plot for each Submission using the configured field mappings.

ODK API — Submissions & Approval

List submissions:

curl "http://localhost:8000/api/v1/odk/submissions/?asset_uid=aYRqYXmmPLFfbcwC2KAULa" \
  -H "Authorization: Bearer <token>"

Approve or reject a submission:

# Approve (approval_status: 1)
curl -X PATCH http://localhost:8000/api/v1/odk/submissions/<uuid>/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "approval_status": 1 }'

# Reject (approval_status: 2) — reason_category is required
curl -X PATCH http://localhost:8000/api/v1/odk/submissions/<uuid>/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "approval_status": 2, "reason_category": "polygon_error", "reason_text": "Polygon too small" }'

Approval status: null = Pending, 1 = Approved, 2 = Rejected.

Rejection categories: polygon_error, overlap, duplicate, other.

ODK API — Plots

Plots are auto-generated during sync. They cannot be created manually (POST returns 405).

List plots with filters:

# By form
curl "http://localhost:8000/api/v1/odk/plots/?form_id=aYRqYXmmPLFfbcwC2KAULa" \
  -H "Authorization: Bearer <token>"

# By approval status
curl "http://localhost:8000/api/v1/odk/plots/?status=pending" \
  -H "Authorization: Bearer <token>"

Update plot geometry:

curl -X PATCH http://localhost:8000/api/v1/odk/plots/<uuid>/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "polygon_wkt": "POLYGON((38.7 9.0, 38.8 9.0, 38.8 9.1, 38.7 9.1, 38.7 9.0))",
    "min_lat": 9.0, "max_lat": 9.1, "min_lon": 38.7, "max_lon": 38.8
  }'

Find overlapping plots by bounding box:

curl -X POST http://localhost:8000/api/v1/odk/plots/overlap_candidates/ \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "min_lat": 9.0, "max_lat": 9.1, "min_lon": 38.7, "max_lon": 38.8 }'

Exported Shapefile — Polygons Not Visible in QGIS

If polygons don't appear after opening an exported Shapefile in QGIS, the plots may be spread across distant geographic regions. QGIS zooms to fit the full extent, making small polygons invisible at that scale.

To verify: open the attribute table, right-click a row, and select Zoom to Feature — the polygon should appear.

API Endpoint Summary

Method Endpoint Description
POST /api/v1/auth/login Login with KoboToolbox credentials
GET /api/v1/odk/forms/ List registered forms
POST /api/v1/odk/forms/ Register a new form
GET /api/v1/odk/forms/{asset_uid}/ Get form detail
PATCH /api/v1/odk/forms/{asset_uid}/ Update form (field mappings)
DELETE /api/v1/odk/forms/{asset_uid}/ Remove a form
GET /api/v1/odk/forms/{asset_uid}/form_fields/ List available KoboToolbox fields
POST /api/v1/odk/forms/{asset_uid}/sync/ Sync submissions from KoboToolbox
GET /api/v1/odk/submissions/ List submissions (?asset_uid= filter)
GET /api/v1/odk/submissions/{uuid}/ Get submission detail
PATCH /api/v1/odk/submissions/{uuid}/ Update approval status and notes
GET /api/v1/odk/submissions/latest_sync_time/ Latest sync time (?asset_uid= required)
GET /api/v1/odk/plots/ List plots (?form_id=, ?status= filters)
GET /api/v1/odk/plots/{uuid}/ Get plot detail
PATCH /api/v1/odk/plots/{uuid}/ Update plot (geometry)
DELETE /api/v1/odk/plots/{uuid}/ Delete a plot
POST /api/v1/odk/plots/overlap_candidates/ Find overlapping plots
GET /api/v1/settings/telegram/ Get Telegram notification config
PUT /api/v1/settings/telegram/ Update Telegram notification config
GET /api/v1/settings/telegram/groups/ List Telegram groups visible to the bot

Telegram Notifications

When a plot is rejected, the system can send notifications to Telegram groups after the rejection syncs to KoboToolbox.

Setup

  1. Create a bot — message @BotFather on Telegram, send /newbot, and follow the prompts. Copy the bot token.

  2. Add the bot to your group(s) — add the bot to the Telegram group(s) you want to receive notifications, then send at least one message in each group so the bot can see it.

  3. Configure via the UI — go to Settings > Telegram in the dashboard:

    • Enable notifications
    • Paste the bot token and click the refresh button to load available groups
    • Select supervisor and enumerator groups from the dropdowns
    • Save

    Alternatively, configure via environment variables in .env:

    TELEGRAM_ENABLED=True
TELEGRAM_BOT_TOKEN=<your_bot_token>
TELEGRAM_SUPERVISOR_GROUP_ID=<group_id>
TELEGRAM_ENUMERATOR_GROUP_ID=<group_id>

    DB settings (saved via the UI) override environment variables.

  4. Fetch groups via API (optional) — if you prefer the CLI:

    # List groups the bot can see
curl "http://localhost:8000/api/v1/settings/telegram/groups/" \
  -H "Authorization: Bearer <token>"

# Or with a specific bot token
curl "http://localhost:8000/api/v1/settings/telegram/groups/?bot_token=<your_bot_token>" \
  -H "Authorization: Bearer <token>"

    Response:

    [
  { "id": "-100123456789", "title": "Supervisors", "type": "supergroup" },
  { "id": "-100987654321", "title": "Enumerators", "type": "group" }
]

How it works

  • Rejections create a RejectionAudit record and dispatch a Kobo validation sync
  • After successful Kobo sync, a Telegram notification is queued automatically
  • Messages are sent to both supervisor and enumerator groups
  • If TELEGRAM_ENABLED=False (default), no notifications are sent

Image Attachments

During sync, image attachments from KoboToolbox submissions are downloaded and stored locally in storage/attachments/{submission_uuid}/. This avoids requiring Kobo credentials for every image request.

How it works

  1. Sync triggers an async task (download_submission_attachments) for each submission with image attachments
  2. Images are saved to storage/attachments/{submission_uuid}/{att_uid}.{ext}
  3. The API returns signed URLs (/storage/attachments/...?key=STORAGE_SECRET) in submission resolved_data
  4. Nginx serves files from /storage/attachments/ directly, requiring a key query parameter

EXIF Orientation Fix

Images uploaded via mobile devices often contain EXIF orientation metadata. During download, the system automatically applies EXIF transpose using Pillow so images display in the correct orientation.

To fix existing attachments that were downloaded before this feature:

# Preview what would be fixed
docker compose exec backend python manage.py fix_attachment_orientation --dry-run

# Fix all existing attachments (applies EXIF transpose locally, then re-downloads from Kobo large URL)
docker compose exec backend python manage.py fix_attachment_orientation

# Fix a single submission
docker compose exec backend python manage.py fix_attachment_orientation --submission-uuid <uuid>

Configuration

Set STORAGE_SECRET in .env to control the key used for signed URLs. If not set, it defaults to SECRET_KEY.

STORAGE_SECRET=your_secret_here

The nginx config (nginx/conf.d/default.conf) serves /storage/attachments/ with a key check and 7-day cache headers.

About

No description, website, or topics provided.

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages