Skip to content

Commit 607ddde

Browse files
MaffoochMaffooch
authored
Merge branch 'bugfix' into engagements
2 parents a57b1ed + 64931bc commit 607ddde

47 files changed

Lines changed: 2415 additions & 193 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CLAUDE.md‎

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
# DefectDojo Development Guide
2+
3+
## Project Overview
4+
5+
DefectDojo is a Django application (`dojo` app) for vulnerability management. The codebase is undergoing a modular reorganization to move from monolithic files toward self-contained domain modules.
6+
7+
## Module Reorganization
8+
9+
### Reference Pattern: `dojo/url/`
10+
11+
All domain modules should match the structure of `dojo/url/`. This is the canonical example of a fully reorganized module.
12+
13+
```
14+
dojo/{module}/
15+
├── __init__.py # import dojo.{module}.admin # noqa: F401
16+
├── models.py # Domain models, constants, factory methods
17+
├── admin.py # @admin.register() for the module's models
18+
├── services.py # Business logic (no HTTP concerns)
19+
├── queries.py # Complex DB aggregations/annotations
20+
├── signals.py # Django signal handlers
21+
├── [manager.py] # Custom QuerySet/Manager if needed
22+
├── [validators.py] # Field-level validators if needed
23+
├── [helpers.py] # Async task wrappers, tag propagation, etc.
24+
├── ui/
25+
│ ├── __init__.py # Empty
26+
│ ├── forms.py # Django ModelForms
27+
│ ├── filters.py # UI-layer django-filter classes
28+
│ ├── views.py # Thin view functions — delegates to services.py
29+
│ └── urls.py # URL routing
30+
└── api/
31+
├── __init__.py # path = "{module}"
32+
├── serializer.py # DRF serializers
33+
├── views.py # API ViewSets — delegates to services.py
34+
├── filters.py # API-layer filters
35+
└── urls.py # add_{module}_urls(router) registration
36+
```
37+
38+
### Architecture Principles
39+
40+
41+
**services.py is the critical layer**: Both `ui/views.py` and `api/views.py` call `services.py` for business logic. Services accept domain objects and primitives — never request/response objects, forms, or serializers.
42+
43+
**Backward-compatible re-exports**: When moving code out of monolithic files (`dojo/models.py`, `dojo/forms.py`, `dojo/filters.py`, `dojo/api_v2/serializers.py`, `dojo/api_v2/views.py`), always leave a re-export at the original location:
44+
```python
45+
from dojo.{module}.models import {Model} # noqa: F401 -- backward compat
46+
```
47+
Never remove re-exports until all consumers are updated in a dedicated cleanup pass.
48+
49+
### Current State
50+
51+
Modules in various stages of reorganization:
52+
53+
| Module | models.py | services.py | ui/ | api/ | Status |
54+
|--------|-----------|-------------|-----|------|--------|
55+
| **url** | In module | N/A | Done | Done | **Complete** |
56+
| **location** | In module | N/A | N/A | Done | **Complete** |
57+
| **product_type** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
58+
| **test** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
59+
| **engagement** | In dojo/models.py | Partial (32 lines) | Partial (views at root) | In dojo/api_v2/ | Needs work |
60+
| **product** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
61+
| **finding** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
62+
63+
### Monolithic Files Being Decomposed
64+
65+
These files still contain code for multiple modules. Extract code to the target module's subdirectory and leave a re-export stub.
66+
67+
- `dojo/models.py` (4,973 lines) — All model definitions
68+
- `dojo/forms.py` (4,127 lines) — All Django forms
69+
- `dojo/filters.py` (4,016 lines) — All UI and API filter classes
70+
- `dojo/api_v2/serializers.py` (3,387 lines) — All DRF serializers
71+
- `dojo/api_v2/views.py` (3,519 lines) — All API viewsets
72+
73+
---
74+
75+
## Reorganization Playbook
76+
77+
When asked to reorganize a module, follow these phases in order. Each phase should be independently verifiable.
78+
79+
### Phase 0: Pre-Flight (Read-Only)
80+
81+
Before any changes, identify all code to extract:
82+
83+
```bash
84+
# 1. Model classes and line ranges in dojo/models.py
85+
grep -n "class {Model}" dojo/models.py
86+
87+
# 2. Form classes in dojo/forms.py
88+
grep -n "class.*{Module}" dojo/forms.py
89+
grep -n "model = {Model}" dojo/forms.py
90+
91+
# 3. Filter classes in dojo/filters.py
92+
grep -n "class.*{Module}\|class.*{Model}" dojo/filters.py
93+
94+
# 4. Serializer classes
95+
grep -n "class.*{Model}" dojo/api_v2/serializers.py
96+
97+
# 5. ViewSet classes
98+
grep -n "class.*{Model}\|class.*{Module}" dojo/api_v2/views.py
99+
100+
# 6. Admin registrations
101+
grep -n "admin.site.register({Model}" dojo/models.py
102+
103+
# 7. All import sites (to verify backward compat)
104+
grep -rn "from dojo.models import.*{Model}" dojo/ unittests/
105+
106+
# 8. Business logic in current views
107+
# Scan dojo/{module}/views.py for: .save(), .delete(), create_notification(),
108+
# jira_helper.*, dojo_dispatch_task(), multi-model workflows
109+
```
110+
111+
### Phase 1: Extract Models
112+
113+
1. Create `dojo/{module}/models.py` with the model class(es) and associated constants
114+
2. Create `dojo/{module}/admin.py` with `admin.site.register()` calls (remove from `dojo/models.py`)
115+
3. Update `dojo/{module}/__init__.py` to `import dojo.{module}.admin # noqa: F401`
116+
4. Add re-exports in `dojo/models.py`
117+
5. Remove original model code (keep re-export line)
118+
119+
**Import rules for models.py:**
120+
- Upward FKs (e.g., Test -> Engagement): import from `dojo.models` if not yet extracted, or `dojo.{module}.models` if already extracted
121+
- Downward references (e.g., Product_Type querying Finding): use lazy imports inside method bodies
122+
- Shared utilities (`copy_model_util`, `_manage_inherited_tags`, `get_current_date`, etc.): import from `dojo.models`
123+
- Do NOT set `app_label` in Meta — all models inherit `dojo` app_label automatically
124+
125+
**Verify:**
126+
```bash
127+
python manage.py check
128+
python manage.py makemigrations --check
129+
python -c "from dojo.{module}.models import {Model}"
130+
python -c "from dojo.models import {Model}"
131+
```
132+
133+
### Phase 2: Extract Services
134+
135+
Create `dojo/{module}/services.py` with business logic extracted from UI views.
136+
137+
**What belongs in services.py:**
138+
- State transitions (close, reopen, status changes)
139+
- Multi-step creation/update workflows
140+
- External integration calls (JIRA, GitHub)
141+
- Notification dispatching
142+
- Copy/clone operations
143+
- Bulk operations
144+
- Merge operations
145+
146+
**What stays in views:**
147+
- HTTP request/response handling
148+
- Form instantiation and validation
149+
- Serialization/deserialization
150+
- Authorization checks (`@user_is_authorized`, `user_has_permission_or_403`)
151+
- Template rendering, redirects
152+
- Pagination, breadcrumbs
153+
154+
**Service function pattern:**
155+
```python
156+
def close_engagement(engagement: Engagement, user: User) -> Engagement:
157+
engagement.active = False
158+
engagement.status = "Completed"
159+
engagement.save()
160+
if jira_helper.get_jira_project(engagement):
161+
dojo_dispatch_task(jira_helper.close_epic, engagement.id, push_to_jira=True)
162+
return engagement
163+
```
164+
165+
Update UI views and API viewsets to call the service instead of containing logic inline.
166+
167+
### Phase 3: Extract Forms to `ui/forms.py`
168+
169+
1. Create `dojo/{module}/ui/__init__.py` (empty)
170+
2. Create `dojo/{module}/ui/forms.py` — move form classes from `dojo/forms.py`
171+
3. Add re-exports in `dojo/forms.py`
172+
173+
### Phase 4: Extract UI Filters to `ui/filters.py`
174+
175+
1. Create `dojo/{module}/ui/filters.py` — move module-specific filters from `dojo/filters.py`
176+
2. Shared base classes (`DojoFilter`, `DateRangeFilter`, `ReportBooleanFilter`) stay in `dojo/filters.py`
177+
3. Add re-exports in `dojo/filters.py`
178+
179+
### Phase 5: Move UI Views/URLs into `ui/`
180+
181+
1. Move `dojo/{module}/views.py` -> `dojo/{module}/ui/views.py`
182+
2. Move `dojo/{module}/urls.py` -> `dojo/{module}/ui/urls.py`
183+
3. Update URL imports:
184+
- product: update `dojo/asset/urls.py`
185+
- product_type: update `dojo/organization/urls.py`
186+
- others: update the include in `dojo/urls.py`
187+
188+
### Phase 6: Extract API Serializers to `api/serializer.py`
189+
190+
1. Create `dojo/{module}/api/__init__.py` with `path = "{module}"`
191+
2. Create `dojo/{module}/api/serializer.py` — move from `dojo/api_v2/serializers.py`
192+
3. Add re-exports in `dojo/api_v2/serializers.py`
193+
194+
### Phase 7: Extract API Filters to `api/filters.py`
195+
196+
1. Create `dojo/{module}/api/filters.py` — move `Api{Model}Filter` from `dojo/filters.py`
197+
2. Add re-exports
198+
199+
### Phase 8: Extract API ViewSets to `api/views.py`
200+
201+
1. Create `dojo/{module}/api/views.py` — move from `dojo/api_v2/views.py`
202+
2. Add re-exports in `dojo/api_v2/views.py`
203+
204+
### Phase 9: Extract API URL Registration
205+
206+
1. Create `dojo/{module}/api/urls.py`:
207+
```python
208+
from dojo.{module}.api import path
209+
from dojo.{module}.api.views import {ViewSet}
210+
211+
def add_{module}_urls(router):
212+
router.register(path, {ViewSet}, path)
213+
return router
214+
```
215+
2. Update `dojo/urls.py` — replace `v2_api.register(...)` with `add_{module}_urls(v2_api)`
216+
217+
### After Each Phase: Verify
218+
219+
```bash
220+
python manage.py check
221+
python manage.py makemigrations --check
222+
python -m pytest unittests/ -x --timeout=120
223+
```
224+
225+
---
226+
227+
## Cross-Module Dependencies
228+
229+
The model hierarchy is: Product_Type -> Product -> Engagement -> Test -> Finding
230+
231+
Extract in this order (top to bottom) so that upward FKs can import from already-extracted modules. The recommended order is: product_type, test, engagement, product, finding.
232+
233+
For downward references (e.g., Product_Type's cached properties querying Finding), always use lazy imports:
234+
```python
235+
@cached_property
236+
def critical_present(self):
237+
from dojo.models import Finding # lazy import
238+
return Finding.objects.filter(test__engagement__product__prod_type=self, severity="Critical").exists()
239+
```
240+
241+
---
242+
243+
## Key Technical Details
244+
245+
- **Single Django app**: Everything is under `app_label = "dojo"`. Moving models to subdirectories does NOT require migration changes.
246+
- **Model discovery**: Triggered by `__init__.py` importing `admin.py`, which imports `models.py`. This is the same chain `dojo/url/` uses.
247+
- **Signal registration**: Handled in `dojo/apps.py` via `import dojo.{module}.signals`. Already set up for test, engagement, product, product_type.
248+
- **Watson search**: Uses `self.get_model("Product")` in `apps.py` — works via Django's model registry regardless of file location.
249+
- **Admin registration**: Currently at the bottom of `dojo/models.py` (lines 4888-4973). Must be moved to `{module}/admin.py` and removed from `dojo/models.py` to avoid `AlreadyRegistered` errors.

‎docs/content/releases/pro/changelog.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,32 @@ For Open Source release notes, please see the [Releases page on GitHub](https://
1212

1313
## Apr 2026: v2.57
1414

15+
### Apr 20, 2026: v2.57.2
16+
17+
* **(Pro UI)** Search and filter state is now preserved when closing a Finding from a Finding list, so you don't lose your place after editing.
18+
* **(Risk Acceptance)** Bulk Edit no longer leaves Simple Risk Acceptance findings in an inconsistent "Active + Risk Accepted" state. Reactivating a previously risk-accepted Finding now behaves correctly.
19+
* **(Risk SLA)** Creating a Risk SLA no longer silently coerces unchecked `enforce_*_risk` options to `True`.
20+
* **(Surveys)** Fixed survey access for both authenticated users and anonymous links.
21+
* **(Universal Parser)** Non-ASCII scan names no longer cause a `UnicodeEncodeError` on import. CSV files with `""`-escaped quotes in multiline fields now parse correctly.
22+
* **(API)** Import/Reimport now validates consistency between ID-based and name-based identifiers, catching mismatched payloads earlier.
23+
* **(Permissions)** Moving an Engagement between Products now requires appropriate permission on both the source and target Product.
24+
* **(Reports)** Fixed a CSS overflow issue in rendered reports. Cleaned up endpoint template rendering for user fields.
25+
* **(Tools)** `govulncheck` parser now records `fix_available` and `fix_version`. Risk Recon parser now validates URLs via a shared SSRF utility. Added Mozilla Foundation security advisories as a supported Vulnerability ID source.
26+
27+
### Apr 13, 2026: v2.57.1
28+
29+
* **(Pro UI)** Object-level history views no longer default to a 31-day date filter, so the full history is visible on load.
30+
* **(Pro UI)** Audit Log "changes" filter now searches only the names of changed fields, reducing false matches.
31+
* **(Pro UI)** Predefined Finding filters now sync UI state correctly, so the active filter indicator reflects the applied filter.
32+
* **(Deduplication)** Added a UI for global component deduplication settings, behind a feature flag.
33+
* **(Rules Engine)** Fixed a preview timeout that occurred when rules were previewed against large Finding sets.
34+
* **(Universal Parser)** CSV/XML query path now displays correctly in the Universal Parser UI.
35+
* **(Import)** Additional parameters are now stored in import settings, making them available for reuse on reimport.
36+
* **(Tools)** Wazuh 4.8 parser now correctly attaches endpoints and locations to findings.
37+
* **(Tools)** Invicti parser now uses `FirstSeenDate` when populating Finding dates when `DD_USE_FIRST_SEEN` is enabled.
38+
* **(Tools)** `govulncheck` parser fixed for NDJSON output.
39+
* **(Tools)** Added CNNVD as a supported Vulnerability ID source.
40+
1541
### Apr 7, 2026: v2.57.0
1642

1743
* **(Custom Enrichment)** On-prem administrators can now configure custom URLs for EPSS and KEV enrichment data sources under **Settings → Finding Enrichment Settings**. Each source (EPSS scores and CISA Known Exploited Vulnerabilities) can be independently enabled and pointed to an internal mirror or proxy. A **Test Configuration** button validates connectivity before saving. Findings with CVE IDs are automatically enriched with EPSS score/percentile and KEV status during enrichment runs.

‎docs/content/triage_findings/finding_deduplication/PRO__deduplication_tuning.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ To adjust Same Tool Deduplication:
3434

3535
### Available Deduplication Algorithms
3636

37-
DefectDojo Pro offers three deduplication methods for same-tool deduplication:
37+
DefectDojo Pro offers the following deduplication methods for same-tool deduplication:
3838

3939
#### Hash Code
4040
Uses a combination of selected fields to generate a unique hash. When selected, a third dropdown will appear showing the fields being used to calculate the hash.
@@ -47,6 +47,9 @@ This algorithm can be useful when working with SAST scanners, or situations wher
4747
#### Unique ID From Tool or Hash Code
4848
Attempts to use the tool's unique ID first, then falls back to the hash code if no unique ID is available. This provides the most flexible deduplication option.
4949

50+
#### Global Component
51+
Matches findings by component name and version across **all Products** in the instance, rather than within a single Product or Engagement. Intended for SCA tools where the same vulnerable dependency appears in many Products. This algorithm is off by default and must be enabled by DefectDojo Support. See [Global Component Deduplication](/triage_findings/finding_deduplication/pro__global_component_deduplication/) for details.
52+
5053
## Cross Tool Deduplication
5154

5255
Cross Tool Deduplication is disabled by default, as deduplication between different security tools requires careful configuration due to variations in how tools report the same vulnerabilities.
@@ -59,7 +62,7 @@ To enable Cross Tool Deduplication:
5962
2. Change the **Deduplication Algorithm** from "Disabled" to "Hash Code"
6063
3. Select which fields should be used for generating the hash in the **Hash Code Fields** dropdown
6164

62-
Unlike Same Tool Deduplication, Cross Tool Deduplication only supports the Hash Code algorithm, as different tools rarely share compatible unique identifiers.
65+
Cross Tool Deduplication supports the Hash Code algorithm, which is suitable for most workflows, as different tools rarely share compatible unique identifiers. For SCA tools reporting the same dependencies, [Global Component Deduplication](/triage_findings/finding_deduplication/pro__global_component_deduplication/) is also available as a cross-tool option (off by default).
6366

6467
## Reimport Deduplication
6568

@@ -76,7 +79,7 @@ When configuring Reimport Deduplication:
7679
1. Select the **Security Tool** (Universal or Generic Parser)
7780
2. Choose the appropriate **Deduplication Algorithm**
7881

79-
The same three algorithm options are available for Reimport Deduplication as for Same Tool Deduplication:
82+
The following algorithm options are available for Reimport Deduplication:
8083
- Hash Code
8184
- Unique ID From Tool
8285
- Unique ID From Tool or Hash Code

0 commit comments

Comments
 (0)