Visual Servoing Navigation for Crop Row Detection

The goal of this project is to develop a visual servoing algorithm capable of navigating a field robotbetween two middle crop rows using data from a front-facing camera. The robot must continuously maintain its path by minimizing two key navigation errors:

1. Heading Error – the angular difference between the robot’s heading and the direction of thecentral navigation line formed between the two middle crop rows
2. Cross-Track Error – the lateral distance between the robot’s position and the midpoint lineseparating the two crop rows.

The visual servoing system thus acts as a perception-driven guidance layer, allowing the robot toalign itself relative to the crop structure without external localization aids.

This project implements three approaches for detecting crop rows in agricultural images/videos and calculating navigation errors (cross-track error and heading error) for autonomous robot navigation.

Project Structure

.
├── README.md                      # Project documentation
├── requirements.txt               # Python dependencies
├── src/                           # Source code directory
│   ├── __init__.py
│   ├── process_navigation_cv.py   # Computer vision approach: line-based detection with peak detection
│   ├── process_navigation_v0.py   # Prototype version (v0): polar coordinate histogram method
│   ├── process_navigation_yolo.py # YOLO segmentation approach: uses trained segmentation model
│   ├── eda_v0.ipynb               # Exploratory data analysis for prototype version
│   └── ai_line_det.ipynb          # AI line detection notebook
└── runs/                          # Model training results
    └── segment/
        └── crop_row_segmentation/ # YOLO segmentation model training outputs
            ├── weights/           # Trained model weights (best.pt, last.pt)
            ├── results.png        # Training metrics
            └── ...                # Additional training artifacts

Main Components

1. src/process_navigation_cv.py

• Method: Line-based approach with peak detection
• Features:
  • Processes center portion of image for efficiency
  • Uses green mask filtering (HSV-based) and peak detection to identify crop rows
  • Detects crop row lines using find_best_counts and peak detection
  • Selects the two middle crop rows
  • Calculates cross-track error and heading error
  • Supports image, directory, and video processing
  • Downscales images for faster processing

2. src/process_navigation_v0.py (Prototype Version)

• Method: Polar coordinate histogram with vanishing point estimation
• Features:
  • Uses Hough transform to estimate vanishing point
  • Detects crop rows using angle histogram peaks
  • Clusters pixels based on angular proximity to peaks
  • Calculates navigation errors
  • Supports image, directory, and video processing

3. src/process_navigation_yolo.py

• Method: YOLO segmentation model approach
• Features:
  • Uses trained YOLO segmentation model to generate binary masks
  • Detects crop row lines using find_best_counts and peak detection
  • Processes entire image (no center portion cropping)
  • Selects the two middle crop rows
  • Calculates cross-track error and heading error
  • Supports image, directory, and video processing
  • Uses model from runs/segment/crop_row_segmentation/weights/best.pt by default

4. src/eda_v0.ipynb

• Exploratory data analysis notebook for prototype version (v0)
• Contains experiments with K-means clustering and polar coordinate histograms
• Development and prototyping experiments

5. src/ai_line_det.ipynb

• AI line detection analysis notebook
• Contains ground truth image creation and line detection experiments

Usage

1. Install dependencies:

pip install -r requirements.txt

2. Process images/videos:

Computer Vision Approach (CV):

# Process a single image
python src/process_navigation_cv.py input_image.png -o outputs/

# Process a video
python src/process_navigation_cv.py input_video.mp4 -o outputs/

# Process a directory of images
python src/process_navigation_cv.py input_dir/ -o outputs/ -r

Prototype Version (v0):

# Process a single image
python src/process_navigation_v0.py input_image.png -o outputs/

# Process a video
python src/process_navigation_v0.py input_video.mp4 -o outputs/

YOLO Segmentation Approach:

# Process a single image (uses default model)
python src/process_navigation_yolo.py input_image.png -o outputs/

# Process with custom model
python src/process_navigation_yolo.py input_image.png -o outputs/ --model path/to/model.pt

# Process a video
python src/process_navigation_yolo.py input_video.mp4 -o outputs/

# Adjust confidence threshold
python src/process_navigation_yolo.py input_image.png -o outputs/ --conf 0.5

Output

All scripts generate visualized images/videos with:

• Detected crop row boundaries (orange lines)
• Ideal robot path (blue line)
• Current robot path (red dashed line)
• Cross-track error and heading error annotations

Model Training

The YOLO segmentation model is trained and stored in runs/segment/crop_row_segmentation/weights/. The training results and metrics are available in the same directory.

Notes

• v0 refers to the Prototype version, which was the initial approach using polar coordinate histograms
• The CV approach (process_navigation_cv.py) processes only the center portion of images for efficiency
• The YOLO approach requires a trained model (provided in runs/segment/crop_row_segmentation/weights/best.pt)