Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ openapi.yaml
html
venv
__pycache__
export.*
249 changes: 249 additions & 0 deletions examples/resource_extra_fields_export.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
"""
eLabFTW Resource Export Script
--------------------------------

This script connects to an eLabFTW instance via the API and exports all items
from a specific category.

For each item, the script:
- Extracts the internal eLabFTW resource ID
- Extracts the item title
- Reads all configured extra fields (custom metadata fields)
- Exports the data in a predefined column order

The export is written to:
- A CSV file (for universal compatibility)
- An XLSX Excel file (with formatting, filters, and auto column width)

The FIELD_ORDER list defines the exact column structure of the export.
This ensures a stable and reproducible layout, which is important for:
- Regulatory documentation
- Data imports into other systems
- Standardized reporting
"""

import elabapi_python
import json
import csv
from openpyxl import Workbook
from openpyxl.styles import Font
from openpyxl.utils import get_column_letter

#########################
# CONFIG #
#########################

# Base URL of your eLabFTW API (must include /api/v2)
API_HOST_URL = 'https://YOUR-URL/api/v2'

# Personal API key generated in eLabFTW
API_KEY = 'YOUR API Key'

# Category ID from which items should be exported
CATEGORY_ID = 123456

# Output filenames
OUTPUT_FILE = "export.csv"
OUTPUT_XLSX = "export.xlsx"

#########################
# API CONFIGURATION #
#########################

# Create configuration object for the eLabFTW API
configuration = elabapi_python.Configuration()
configuration.api_key['api_key'] = API_KEY
configuration.api_key_prefix['api_key'] = 'Authorization'
configuration.host = API_HOST_URL

configuration.debug = False

# Set to True if valid SSL certificates are used
configuration.verify_ssl = True

# Create API client
api_client = elabapi_python.ApiClient(configuration)

# Set authorization header manually (required for authentication)
api_client.set_default_header(
header_name='Authorization',
header_value=API_KEY
)
Comment on lines +53 to +71
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

cfg="$(fd '^configuration\.py$' | head -n1)"
example_cfg="examples/client.py"

test -n "$cfg"
echo "Inspecting generated auth semantics in: $cfg"
sed -n '/def get_api_key_with_prefix/,/def /p' "$cfg"

echo
echo "Inspecting example auth setup in: $example_cfg"
rg -n -C2 'set_default_header|api_key_prefix|api_key\[' "$example_cfg"

Repository: elabftw/elabapi-python

Length of output: 48


🏁 Script executed:

# Find key files and understand repository structure
fd -t f -e py '(client\.py|configuration\.py|resource_extra)' | head -20

Repository: elabftw/elabapi-python

Length of output: 125


🏁 Script executed:

# List top-level structure
ls -la

Repository: elabftw/elabapi-python

Length of output: 763


🏁 Script executed:

# Search for elabapi_python module location and Configuration class
rg -l "class Configuration" --max-count=5

Repository: elabftw/elabapi-python

Length of output: 48


🏁 Script executed:

# Examine both example files to understand auth setup
echo "=== examples/client.py ===" 
cat -n examples/client.py

echo -e "\n=== examples/resource_extra_fields_export.py (first 80 lines) ===" 
head -80 examples/resource_extra_fields_export.py | cat -n

Repository: elabftw/elabapi-python

Length of output: 5742


🏁 Script executed:

# Check imports and find elabapi_python
rg "^import|^from" examples/ | head -20

Repository: elabftw/elabapi-python

Length of output: 1135


🏁 Script executed:

# Search for other uses of api_key_prefix in the codebase to understand the pattern
rg "api_key_prefix" --max-count=10

Repository: elabftw/elabapi-python

Length of output: 164


🏁 Script executed:

# Look at how other examples handle authentication
rg -l "Configuration\|ApiClient" examples/*.py | head -10

Repository: elabftw/elabapi-python

Length of output: 48


🏁 Script executed:

# Check if there are any other authentication patterns in examples
fd -t f -e py examples/ | xargs rg -l "set_default_header|api_key" | head -10

Repository: elabftw/elabapi-python

Length of output: 494


🏁 Script executed:

# Look for OpenAPI generated client documentation or patterns
fd -t f -e py -e md | xargs rg -l "get_api_key_with_prefix|prepend_prefix" 2>/dev/null | head -10

Repository: elabftw/elabapi-python

Length of output: 48


🏁 Script executed:

# Check all examples for their authentication patterns
echo "=== Checking all example files for auth setup ==="
for file in examples/*.py; do
  echo -e "\n--- $file ---"
  rg -n "Configuration\|set_default_header|api_key\[|api_key_prefix" "$file" | head -15
done

Repository: elabftw/elabapi-python

Length of output: 1334


🏁 Script executed:

# Let's look at the full context of both authentication setups in resource_extra_fields_export.py
echo "=== Full resource_extra_fields_export.py auth section ==="
sed -n '50,75p' examples/resource_extra_fields_export.py | cat -n

Repository: elabftw/elabapi-python

Length of output: 1008


🏁 Script executed:

# Check README or any documentation about authentication
cat README.md | head -100

Repository: elabftw/elabapi-python

Length of output: 3790


🏁 Script executed:

# Search for requirements or setup.py to understand elabapi_python version/source
fd -t f '(requirements|setup|pyproject)' | xargs cat 2>/dev/null | head -50

Repository: elabftw/elabapi-python

Length of output: 48


🏁 Script executed:

# Let's check if there's any documentation in comments about how api_key works
rg -i "api.?key|authorization" --type md | head -20

Repository: elabftw/elabapi-python

Length of output: 212


🏁 Script executed:

# Let's check all examples to see if any others use api_key/api_key_prefix pattern
rg -c "api_key\[|api_key_prefix" examples/*.py

Repository: elabftw/elabapi-python

Length of output: 108


🏁 Script executed:

# Check if examples/00-getting-started.py (first example) shows auth setup
sed -n '1,80p' examples/00-getting-started.py | cat -n

Repository: elabftw/elabapi-python

Length of output: 4970


Consolidate authentication configuration to the standard pattern.

Lines 55–56 employ configuration.api_key and configuration.api_key_prefix, which deviate from the established practice shown in the README and examples/client.py. Moreover, lines 68–71 redundantly set the same Authorization header, creating unnecessary duplication. Remove the configuration-based approach and rely solely on api_client.set_default_header(), as demonstrated in the standard examples/client.py (lines 55–58).

Suggested change
 # Create configuration object for the eLabFTW API
 configuration = elabapi_python.Configuration()
-configuration.api_key['api_key'] = API_KEY
-configuration.api_key_prefix['api_key'] = 'Authorization'
 configuration.host = API_HOST_URL
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Create configuration object for the eLabFTW API
configuration = elabapi_python.Configuration()
configuration.api_key['api_key'] = API_KEY
configuration.api_key_prefix['api_key'] = 'Authorization'
configuration.host = API_HOST_URL
configuration.debug = False
# Set to True if valid SSL certificates are used
configuration.verify_ssl = True
# Create API client
api_client = elabapi_python.ApiClient(configuration)
# Set authorization header manually (required for authentication)
api_client.set_default_header(
header_name='Authorization',
header_value=API_KEY
)
# Create configuration object for the eLabFTW API
configuration = elabapi_python.Configuration()
configuration.host = API_HOST_URL
configuration.debug = False
# Set to True if valid SSL certificates are used
configuration.verify_ssl = True
# Create API client
api_client = elabapi_python.ApiClient(configuration)
# Set authorization header manually (required for authentication)
api_client.set_default_header(
header_name='Authorization',
header_value=API_KEY
)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@examples/resource_extra_fields_export.py` around lines 53 - 71, Remove the
nonstandard config-based auth and the duplicate header: delete the two lines
that set configuration.api_key and configuration.api_key_prefix (the uses of
configuration.api_key and configuration.api_key_prefix after creating
Configuration()), keep the Configuration() setup for host and verify_ssl,
instantiate ApiClient(configuration) as before, and rely on a single call to
api_client.set_default_header(header_name='Authorization', header_value=API_KEY)
to set auth; this consolidates authentication to the standard pattern used in
examples.


# Load Items API endpoint
itemsApi = elabapi_python.ItemsApi(api_client)

###############################
# DEFINE EXPORT ORDER HERE #
###############################

"""
FIELD_ORDER defines the exact column order of metadata fields
in the exported files.

Why is this necessary?

eLabFTW stores extra fields dynamically in JSON format.
Without explicitly defining the order:
- Column positions could change
- Fields might appear in random order
- Downstream processing (e.g., Excel templates, validation scripts)
could break

By defining FIELD_ORDER:
- The export structure remains stable
- Reports remain consistent
- Future modifications can be controlled centrally

You can modify this list to match your own category setup.
Simply replace the example field names below with the exact
field titles used in your system.
"""

FIELD_ORDER = [
"Record Number", # e.g. internal running number
"Project ID", # e.g. reference or file number
"Organism Name", # e.g. E. coli strain
"Gene / Target", # e.g. transgene or modification
"Storage Location", # e.g. freezer or storage unit
"Experiment Purpose", # short description of use
"Donor Organism",
"Recipient Organism",
"Vector",
"Resistance Marker",
"Sequence Information",
"Risk Assessment Reference",
"Created By",
"Created At",
"Comments"
]

# --------------------------------------------------
# ----------- SCRIPT STARTS HERE ------------------
# --------------------------------------------------

print("Starting export...")

# Retrieve all items from the specified category
response = itemsApi.read_items(cat=CATEGORY_ID, _preload_content=False)
items = json.loads(response.data.decode("utf-8"))
Comment on lines +127 to +129
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

For eLabFTW / elabapi_python ItemsApi.read_items, what pagination parameters and default page size apply when listing items by category? Does one call without limitoroffset return all items or only the first page?

💡 Result:

For eLabFTW / elabapi_python ItemsApi.read_items, the pagination parameters are limit and offset, which are query parameters supported by the underlying REST API endpoint /api/v2/items. The default page size (limit) is 15. The cat parameter filters items by category ID (supports comma-separated lists including "null"). A call without limit or offset uses the defaults (limit=15, offset=0), returning only the first page, not all items.

Citations:


Implement pagination to retrieve all items, or document the intended limit explicitly.

This implementation retrieves only the first page of results. The ItemsApi.read_items() method operates with a default page size of 15 items when no limit parameter is specified. When a category exceeds this batch size, the export will terminate prematurely without notification to the operator.

To ensure complete data retrieval, you have two viable approaches: either implement pagination by iterating with offset parameters until all items are exhausted (as you have precedent in examples/002-extra-field-to-column.py with limit=9999), or explicitly set and document the limit parameter to match your intended scope. The current code leaves the behavior ambiguous and risks silent data loss.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@examples/resource_extra_fields_export.py` around lines 127 - 129, The current
call to ItemsApi.read_items(category=CATEGORY_ID) only fetches the first page
(default page size 15), so change it to either paginate or explicitly set a
larger limit: implement a loop that calls ItemsApi.read_items(cat=CATEGORY_ID,
limit=..., offset=offset) decoding response.data and appending to items until an
empty page is returned, or pass a documented limit (e.g., limit=9999) to
ItemsApi.read_items to guarantee full retrieval; update the variables response,
items, and the pagination logic around ItemsApi.read_items to reflect the chosen
approach.


print(f"Items found: {len(items)}")

Comment on lines +125 to +132
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Engage defensive protocols: the API call lacks error handling.

The read_items call on line 128 can fail due to network issues, authentication errors, or invalid category IDs. Currently, any exception would crash the script without a clear diagnostic message. Additionally, this script lacks a if __name__ == "__main__": guard, meaning it executes immediately upon import—problematic if someone attempts to reuse functions from this module.

🛡️ Proposed fix: Add main guard and API error handling
+def main():
+    print("Starting export...")
 
-print("Starting export...")
+    # Retrieve all items from the specified category
+    try:
+        items = itemsApi.read_items(cat=CATEGORY_ID)
+    except elabapi_python.ApiException as e:
+        print(f"API request failed: {e}")
+        return
 
-# Retrieve all items from the specified category
-items = itemsApi.read_items(cat=CATEGORY_ID)
+    print(f"Items found: {len(items)}")
+    # ... rest of logic ...
 
-print(f"Items found: {len(items)}")
+if __name__ == "__main__":
+    main()
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@examples/resource_extra_fields_export.py` around lines 125 - 131, Wrap the
script execution in a main guard (if __name__ == "__main__":) so importing this
module won't run the export immediately, and add defensive error handling around
the itemsApi.read_items call: call itemsApi.read_items(cat=CATEGORY_ID) inside a
try/except that catches broad request/API exceptions (and Exception as a
fallback), logs/prints a clear diagnostic including the exception message and
returns/exit non‑zero on failure; ensure variables referenced after the call
(like items and len(items)) are only used when the call succeeds. Reference:
itemsApi.read_items and the top-level script body where the print("Starting
export...") and items = itemsApi.read_items(...) occur.

rows = []

# Iterate through all retrieved items
for item in items:

row = {}

# Internal eLabFTW resource ID (primary identifier)
row["Ressourcen ID"] = item.get("id", "")

# Item title
row["Titel"] = item.get("title", "")

# Metadata is stored as JSON object
metadata_raw = item.get("metadata")

if metadata_raw:
if isinstance(metadata_raw, str):
metadata = json.loads(metadata_raw)
extra_fields = metadata.get("extra_fields", {})
else:
metadata = metadata_raw
extra_fields = getattr(metadata, "extra_fields", {}) or {}
Comment on lines +149 to +155
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

The dict-backed metadata branch loses every extra field.

After Line 129, item is a Python dict. If metadata_raw is already a dict, getattr(metadata_raw, "extra_fields", {}) always falls back to {}, so the export appears successful while silently blanking those columns. Handle dicts with .get("extra_fields", {}) before falling back to attribute access.

🖖 Suggested change
     if metadata_raw:
         if isinstance(metadata_raw, str):
             metadata = json.loads(metadata_raw)
             extra_fields = metadata.get("extra_fields", {})
+        elif isinstance(metadata_raw, dict):
+            extra_fields = metadata_raw.get("extra_fields", {}) or {}
         else:
-            metadata = metadata_raw
-            extra_fields = getattr(metadata, "extra_fields", {}) or {}
+            extra_fields = getattr(metadata_raw, "extra_fields", {}) or {}
     else:
         extra_fields = {}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@examples/resource_extra_fields_export.py` around lines 149 - 155, The
dict-backed metadata branch currently uses getattr(metadata, "extra_fields", {})
which discards extra fields when metadata_raw is a dict; change the branching so
that if metadata_raw is a dict you set metadata = metadata_raw and extract
extra_fields = metadata.get("extra_fields", {}) (i.e., check
isinstance(metadata_raw, dict) or use an elif for dict before the getattr
fallback), otherwise fall back to attribute access via getattr(metadata,
"extra_fields", {}) or {} to preserve compatibility with object-backed metadata.

else:
extra_fields = {}

# Normalize keys for case/space insensitive matching
normalized_fields = {}

if isinstance(extra_fields, dict):
normalized_fields = {k.strip().lower(): v for k, v in extra_fields.items()}
# backwards compatibility for elabapi_python versions
else:
for f in extra_fields:
key = getattr(f, "name", "").strip().lower()
normalized_fields[key] = f

# Extract all fields defined in FIELD_ORDER
for field in FIELD_ORDER:
key = field.strip().lower()

if key in normalized_fields:
field_obj = normalized_fields[key]

# Support dict or object style access
if isinstance(field_obj, dict):
value = field_obj.get("value", "")
else:
value = getattr(field_obj, "value", "")
else:
value = ""

# Convert list-type fields into comma-separated string
if isinstance(value, list):
value = ", ".join(value)

row[field] = value
Comment on lines +185 to +189
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Ensure type safety before joining list values.

If value contains non-string elements, the join operation will fail. A minor adjustment ensures we navigate safely through any data anomalies.

🛡️ Proposed fix: Convert elements to strings
         # Convert list-type fields into comma-separated string
         if isinstance(value, list):
-            value = ", ".join(value)
+            value = ", ".join(str(v) for v in value)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@examples/resource_extra_fields_export.py` around lines 161 - 165, The code
that converts list-type fields into a comma-separated string should coerce each
element to a string before joining to avoid TypeError for non-string items;
update the block that checks isinstance(value, list) (using the variables value,
field, row) to build a list of strings (e.g., [str(v) for v in value]) and then
join that list, so row[field] gets a safe string representation.


rows.append(row)

#########################
# WRITE CSV #
#########################

# Define final column structure
csv_columns = ["Ressourcen ID", "Titel"] + FIELD_ORDER

# Write CSV file (UTF-8 encoding for special characters)
with open(OUTPUT_FILE, "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=csv_columns)
writer.writeheader()
writer.writerows(rows)

print("CSV export finished.")

#########################
# WRITE XLSX #
#########################

wb = Workbook()
ws = wb.active
ws.title = "Export"

# Write header row
ws.append(csv_columns)

# Make header row bold
for cell in ws[1]:
cell.font = Font(bold=True)

# Write data rows
for row in rows:
ws.append([row[col] for col in csv_columns])

# Enable auto-filter for the entire sheet
ws.auto_filter.ref = ws.dimensions

# Automatically adjust column width based on content length
for col in ws.columns:
max_length = 0
column = col[0].column
column_letter = get_column_letter(column)

for cell in col:
try:
if cell.value:
max_length = max(max_length, len(str(cell.value)))
except:
pass

adjusted_width = max_length + 2
ws.column_dimensions[column_letter].width = adjusted_width
Comment on lines +236 to +244
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

A bare except clause is unworthy of Starfleet protocols.

This silently swallows all exceptions, making debugging nearly impossible when anomalies occur. We must be specific about what we catch, or at minimum, log the encounter.

🛡️ Proposed fix: Handle exceptions properly
     for cell in col:
         try:
             if cell.value:
                 max_length = max(max_length, len(str(cell.value)))
-        except:
-            pass
+        except (TypeError, AttributeError):
+            continue
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for cell in col:
try:
if cell.value:
max_length = max(max_length, len(str(cell.value)))
except:
pass
adjusted_width = max_length + 2
ws.column_dimensions[column_letter].width = adjusted_width
for cell in col:
try:
if cell.value:
max_length = max(max_length, len(str(cell.value)))
except (TypeError, AttributeError):
continue
adjusted_width = max_length + 2
ws.column_dimensions[column_letter].width = adjusted_width
🧰 Tools
🪛 Ruff (0.15.2)

[error] 216-216: Do not use bare except

(E722)


[error] 216-217: try-except-pass detected, consider logging the exception

(S110)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@examples/resource_extra_fields_export.py` around lines 212 - 220, The loop
over "for cell in col" silently swallows all exceptions via a bare except;
change it to catch specific exceptions (e.g., TypeError, ValueError,
AttributeError) when calling len(str(cell.value)) and handle them
explicitly—either skip the cell or record/log the error—so max_length is still
computed safely; update the block around the "for cell in col" loop and the
calculation of adjusted_width and ws.column_dimensions[column_letter].width to
use the specific exception types and emit a warning (or use the module logger)
when an unexpected cell value is encountered.


# Save Excel file
wb.save(OUTPUT_XLSX)

print("XLSX export finished successfully.")