deepforest package

Subpackages

• deepforest.data package
  • Module contents

Submodules

deepforest.IoU module

IoU Module, with help from SpaceNetChallenge/utilities

deepforest.IoU.match_points(ground_truth: GeoDataFrame, submission: GeoDataFrame, norm: str = 'l2')

Find distance among all sets of ground truth and prediction points.

This function performs matching between a ground truth dataset and a submission or prediction dataset, typically the output from a validation or test run. In order to compute distances, we must know which points correspond between the datasets. This is performed by Hungarian matching, or linear sum assignment.

For each ground truth point, we compute the L2 distances of all overlapping points. The distance matrix is used as the input cost matrix for the assignment and the algorithm is such that at most one prediction is assigned to each ground truth, and each prediction is only used at most once, with the solver aiming to minimise the total distance. The matching indices are then returned, along with their distances and scores, to be used in downstream metrics like recall and precision.

No filtering on distance or score is performed.

Parameters

• ground_truth – a projected geopandas dataframe with geometry

• submission – a projected geopandas dataframe with geometry

• norm – distance norm to use (“l1” or “l2”)

Returns

dataframe of distances

Return type

dist_df

deepforest.IoU.match_points_box(ground_truth: GeoDataFrame, submission: GeoDataFrame)

Compute point-in-box matches between ground truth and submission.

Parameters

• predictions – a pandas dataframe. The labels in ground truth and predictions must match. For example, if one is numeric, the other must be numeric.

• ground_df – a pandas dataframe

Returns

pandas dataframe with crown ids of prediction and ground truth and the IoU score.

Return type

result

deepforest.IoU.match_polygons(ground_truth: GeoDataFrame, submission: GeoDataFrame)

Find area of overlap among all sets of ground truth and prediction.

This function performs matching between a ground truth dataset and a submission or prediction dataset, typically the output from a validation or test run. In order to compute IoU, we must know which boxes correspond between the datasets. This is performed by Hungarian matching, or linear sum assignment.

For each ground truth polygon, we compute the IoUs of all overlapping polygons. Intersection areas are used as the input cost matrix for the assignment and the algorithm is such that at most one prediction is assigned to each ground truth, and each prediction is only used at most once, with the solver aiming to maximise the total area of intersection. The matching indices are then returned, along with their IoUs and scores, to be used in downstream metrics like recall and precision.

No filtering on IoU or score is performed.

Parameters

• ground_truth – a projected geopandas dataframe with geoemtry

• submission – a projected geopandas dataframe with geometry

Returns

dataframe of IoU scores

Return type

iou_df

deepforest.callbacks module

DeepForest callback for logging images during training.

Callbacks must implement on_epoch_begin, on_epoch_end, on_fit_end, on_fit_begin methods and inject model and epoch kwargs.

class deepforest.callbacks.ImagesCallback(save_dir, prediction_samples=2, dataset_samples=5, every_n_epochs=5, select_random=False, color=None, thickness=2)

Bases: Callback

Log evaluation images during training.

Parameters

• save_dir – Directory to save predicted images

• n – Number of images to process

• every_n_epochs – Run interval in epochs

• select_random – Whether to select random images

• color – Bounding box color as BGR tuple

• thickness – Border line thickness in pixels

on_train_start(trainer, pl_module)

Log sample images from training and validation datasets at training start.

on_validation_end(trainer, pl_module)

Run callback at validation end.

class deepforest.callbacks.images_callback(savedir, **kwargs)

Bases: ImagesCallback

deepforest.evaluate module

Evaluation module.

deepforest.evaluate.compute_class_recall(results)

Given a set of evaluations, what proportion of predicted boxes match.

True boxes which are not matched to predictions do not count against accuracy.

deepforest.evaluate.evaluate_boxes(predictions: pandas.DataFrame | geopandas.geodataframe.GeoDataFrame, ground_df: pandas.DataFrame | geopandas.geodataframe.GeoDataFrame, iou_threshold: float = 0.4) → dict

Evaluate bounding box predictions against ground truth. Calls evaluate_geometry.

Parameters

• predictions – a pandas dataframe with geometry columns. The labels in ground truth and predictions must match. If one is numeric, the other must be numeric.

• ground_df – a pandas dataframe with geometry columns

• iou_threshold – intersection-over-union threshold, see deepforest.evaluate

Returns

a dictionary of results with keys, results, box_recall, box_precision, class_recall

Return type

results

deepforest.evaluate.evaluate_geometry(predictions: pandas.DataFrame | geopandas.geodataframe.GeoDataFrame, ground_df: pandas.DataFrame | geopandas.geodataframe.GeoDataFrame, iou_threshold: float = 0.4, distance_threshold: float = 10.0, geometry_type: str = 'box') → dict

Image annotated crown evaluation routine submission can be submitted as a .shp, existing pandas dataframe or .csv path.

Parameters

• predictions – a pandas dataframe with geometry columns. The labels in ground truth and predictions must match. If one is numeric, the other must be numeric.

• ground_df – a pandas dataframe with geometry columns

• iou_threshold – intersection-over-union threshold, see deepforest.evaluate

• l2_threshold – L2 distance threshold for point matching

• geometry_type – ‘box’, ‘polygon’ or ‘point’

Returns

a dataframe of match bounding boxes box_recall: proportion of true positives of box position, regardless of class box_precision: proportion of predictions that are true positive, regardless of class class_recall: a pandas dataframe of class level recall and precision with class sizes

Return type

results

deepforest.evaluate.match_predictions(predictions, ground_df, task='box')

Compute intersection-over-union matching among prediction and ground truth geometries for one image. The returned results are guaranteed to be at most one-to-one, but are not filtered for “quality” of match (i.e. IoU threshold).

Parameters

• predictions – a geopandas dataframe with geometry columns

• ground_df – a geopandas dataframe with geometry columns

Returns

pandas dataframe with crown ids of prediction and ground truth and the IoU score.

Return type

result

deepforest.main module

class deepforest.main.deepforest(model=None, transforms=None, existing_train_dataloader=None, existing_val_dataloader=None, config: str | dict | omegaconf.dictconfig.DictConfig | None = None, config_args: dict | None = None)

Bases: LightningModule

DeepForest model for tree crown detection in RGB images.

Parameters

• num_classes – Number of classes in the model

• model – DeepForest model object

• existing_train_dataloader – PyTorch dataloader for training data

• existing_val_dataloader – PyTorch dataloader for validation data

• config – DeepForest configuration object or name

• config_args – Dictionary of config overrides

calculate_empty_frame_accuracy(ground_df, predictions_df)

Calculate accuracy for empty frames (frames with no objects).

Parameters

• ground_df (pd.DataFrame) – Ground truth dataframe containing image paths and bounding boxes. Must have columns ‘image_path’, ‘xmin’, ‘ymin’, ‘xmax’, ‘ymax’.

• predictions_df (pd.DataFrame) – Model predictions dataframe containing image paths and predicted boxes. Must have column ‘image_path’.

Returns

Accuracy score for empty frame detection. A score of 1.0 means the model correctly

identified all empty frames (no false positives), while 0.0 means it predicted objects in all empty frames (all false positives). Returns None if there are no empty frames.

Return type

float or None

configure_optimizers()

Choose what optimizers and learning-rate schedulers to use in your optimization. Normally you’d need one. But in the case of GANs or similar you might have multiple. Optimization with multiple optimizers only works in the manual optimization mode.

Returns

Any of these 6 options.

• Single optimizer.

• List or Tuple of optimizers.

• Two lists - The first list has multiple optimizers, and the second has multiple LR schedulers (or multiple lr_scheduler_config).

• Dictionary, with an "optimizer" key, and (optionally) a "lr_scheduler" key whose value is a single LR scheduler or lr_scheduler_config.

• None - Fit will run without any optimizer.

The lr_scheduler_config is a dictionary which contains the scheduler and its associated configuration. The default configuration is shown below.

lr_scheduler_config = {
    # REQUIRED: The scheduler instance
    "scheduler": lr_scheduler,
    # The unit of the scheduler's step size, could also be 'step'.
    # 'epoch' updates the scheduler on epoch end whereas 'step'
    # updates it after a optimizer update.
    "interval": "epoch",
    # How many epochs/steps should pass between calls to
    # `scheduler.step()`. 1 corresponds to updating the learning
    # rate after every epoch/step.
    "frequency": 1,
    # Metric to monitor for schedulers like `ReduceLROnPlateau`
    "monitor": "val_loss",
    # If set to `True`, will enforce that the value specified 'monitor'
    # is available when the scheduler is updated, thus stopping
    # training if not found. If set to `False`, it will only produce a warning
    "strict": True,
    # If using the `LearningRateMonitor` callback to monitor the
    # learning rate progress, this keyword can be used to specify
    # a custom logged name
    "name": None,
}

When there are schedulers in which the .step() method is conditioned on a value, such as the torch.optim.lr_scheduler.ReduceLROnPlateau scheduler, Lightning requires that the lr_scheduler_config contains the keyword "monitor" set to the metric name that the scheduler should be conditioned on.

# The ReduceLROnPlateau scheduler requires a monitor
def configure_optimizers(self):
    optimizer = Adam(...)
    return {
        "optimizer": optimizer,
        "lr_scheduler": {
            "scheduler": ReduceLROnPlateau(optimizer, ...),
            "monitor": "metric_to_track",
            "frequency": "indicates how often the metric is updated",
            # If "monitor" references validation metrics, then "frequency" should be set to a
            # multiple of "trainer.check_val_every_n_epoch".
        },
    }


# In the case of two optimizers, only one using the ReduceLROnPlateau scheduler
def configure_optimizers(self):
    optimizer1 = Adam(...)
    optimizer2 = SGD(...)
    scheduler1 = ReduceLROnPlateau(optimizer1, ...)
    scheduler2 = LambdaLR(optimizer2, ...)
    return (
        {
            "optimizer": optimizer1,
            "lr_scheduler": {
                "scheduler": scheduler1,
                "monitor": "metric_to_track",
            },
        },
        {"optimizer": optimizer2, "lr_scheduler": scheduler2},
    )

Metrics can be made available to monitor by simply logging it using self.log('metric_to_track', metric_val) in your LightningModule.

Note

Some things to know:

• Lightning calls .backward() and .step() automatically in case of automatic optimization.

• If a learning rate scheduler is specified in configure_optimizers() with key "interval" (default “epoch”) in the scheduler configuration, Lightning will call the scheduler’s .step() method automatically in case of automatic optimization.

• If you use 16-bit precision (precision=16), Lightning will automatically handle the optimizer.

• If you use torch.optim.LBFGS, Lightning handles the closure function automatically for you.

• If you use multiple optimizers, you will have to switch to ‘manual optimization’ mode and step them yourself.

• If you need to control how often the optimizer steps, override the optimizer_step() hook.

create_model(initialize_model=False)

Initialize a deepforest architecture. This can be done in two ways. Passed as the model argument to deepforest __init__(), or as a named architecture in config.architecture, which corresponds to a file in models/, as is a subclass of model.Model(). The config args in the .yaml are specified.

Returns

None

create_trainer(logger=None, callbacks=None, **kwargs)

Create a pytorch lightning training by reading config files.

Parameters

• logger – Optional logger

• callbacks – Optional list of callbacks

• **kwargs – Additional trainer arguments

evaluate(csv_file, iou_threshold=None, root_dir=None, predictions=None)

Compute intersection-over-union and precision/recall for a given iou_threshold.

Deprecated since version 2.0.0: This method is deprecated. Users should use trainer.validate() instead to get evaluation statistics during training. This method will be removed in a future version.

Parameters

• csv_file – location of a csv file with columns “name”,”xmin”,”ymin”,”xmax”,”ymax”,”label”

• iou_threshold – float [0,1] intersection-over-union threshold for true positive

• predictions – list of predictions to use for evaluation. If None, predictions are generated from the model.

Returns

Results dictionary containing precision, recall and other metrics

Return type

dict

load_dataset(csv_file, root_dir=None, shuffle=True, transforms=None, augmentations=None, preload_images=False, batch_size=1)

Create a dataset for inference or training. Csv file format is .csv file with the columns “image_path”, “xmin”,”ymin”,”xmax”,”ymax” for the image name and bounding box position. Image_path is the relative filename, not absolute path, which is in the root_dir directory. One bounding box per line.

Parameters

• csv_file – path to csv file

• root_dir – directory of images. If none, uses “image_dir” in config

• transforms – Albumentations transforms

• batch_size – batch size

• preload_images – if True, preload the images into memory

• augmentations – augmentation configuration (str, list, or dict)

Returns

a pytorch dataset

Return type

ds

load_model(model_name=None, revision=None)

Loads a model that has already been pretrained for a specific task, like tree crown detection.

Models (technically model weights) are distributed via Hugging Face and designated the Hugging Face repository ID (model_name), which is in the form: ‘organization/repository’. For a list of models distributed by the DeepForest team (and the associated model names) see the documentation: https://deepforest.readthedocs.io/en/latest/installation_and_setup/prebuilt.html

Parameters

• model_name (str) – A repository ID for huggingface in the form of organization/repository

• revision (str) – The model version (‘main’, ‘v1.0.0’, etc.).

Returns

None

on_fit_start()

Called at the very beginning of fit.

If on DDP it is called on every process

on_load_checkpoint(checkpoint)

Called by Lightning to restore your model. If you saved something with on_save_checkpoint() this is your chance to restore this.

Parameters

checkpoint – Loaded checkpoint

Example:

def on_load_checkpoint(self, checkpoint):
    # 99% of the time you don't need to implement this method
    self.something_cool_i_want_to_save = checkpoint['something_cool_i_want_to_save']

Note

Lightning auto-restores global step, epoch, and train state including amp scaling. There is no need for you to restore anything regarding training.

on_save_checkpoint(checkpoint)

Called by Lightning when saving a checkpoint to give you a chance to store anything else you might want to save.

Parameters

checkpoint – The full checkpoint dictionary before it gets dumped to a file. Implementations of this hook can insert additional data into this dictionary.

Example:

def on_save_checkpoint(self, checkpoint):
    # 99% of use cases you don't need to implement this method
    checkpoint['something_cool_i_want_to_save'] = my_cool_pickable_object

Note

Lightning saves all aspects of training (epoch, global step, etc…) including amp scaling. There is no need for you to store anything about training.

on_validation_epoch_end()

Compute metrics and predictions at the end of the validation epoch.

on_validation_epoch_start()

Called in the validation loop at the very beginning of the epoch.

predict_batch(images, preprocess_fn=None)

Predict a batch of images with the deepforest model.

Parameters

• images (torch.Tensor or np.ndarray) – A batch of images with shape (B, C, H, W).

• preprocess_fn (callable, optional) – A function to preprocess images before prediction. If None, assumes images are preprocessed.

Returns

A list of dataframes with predictions for each image.

Return type

List[pd.DataFrame]

predict_dataloader(ds, batch_size=None)

Create a PyTorch dataloader for prediction.

Parameters

ds (torchvision.datasets.Dataset) – A torchvision dataset to be wrapped into a dataloader using config args.

Returns

A dataloader object that can be used for prediction.

Return type

torch.utils.data.DataLoader

predict_file(csv_file, root_dir, crop_model=None)

Create a dataset and predict entire annotation file CSV file format is .csv file with the columns “image_path”, “xmin”,”ymin”,”xmax”,”ymax” for the image name and bounding box position. Image_path is the relative filename, not absolute path, which is in the root_dir directory. One bounding box per line.

Parameters

• csv_file – path to csv file

• root_dir – directory of images. If none, uses “image_dir” in config

• crop_model – a deepforest.model.CropModel object to predict on crops

• size – the size of the image to resize to. Optional, if not provided, the image is not resized.

Returns

pandas dataframe with bounding boxes, label and scores for each image in the csv file

Return type

df

predict_image(image: numpy.ndarray | None = None, path: str | None = None)

Predict a single image with a deepforest model.

Parameters

• image – a float32 numpy array of a RGB with channels last format

• path – optional path to read image from disk instead of passing image arg

Returns

A pandas dataframe of predictions (Default)

Return type

result

predict_step(batch, batch_idx)

Predict a batch of images with the deepforest model. If batch is a list, concatenate the images, predict and then split the results, useful for main.predict_tile.

Parameters

• batch (torch.Tensor or np.ndarray) – A batch of images with shape (B, C, H, W).

• batch_idx (int) – The index of the batch.

Returns:

predict_tile(path=None, image=None, patch_size=400, patch_overlap=0.05, iou_threshold=0.15, dataloader_strategy='single', crop_model=None)

For images too large to input into the model, predict_tile cuts the image into overlapping windows, predicts trees on each window and reassambles into a single array.

Parameters

• path – Path or list of paths to images on disk. If a single string is provided, it will be converted to a list.

• image (array) – Numpy image array in BGR channel order following openCV convention. Not possible in combination with dataloader_strategy=’batch’.

• patch_size – patch size for each window

• patch_overlap – patch overlap among windows

• iou_threshold – Minimum iou overlap among predictions between windows to be suppressed

• dataloader_strategy – “single”, “batch”, or “window”. - “Single” loads the entire image into memory and passes individual windows to GPU and cannot be parallelized. - “batch” loads the entire image into GPU memory and creates views of an image as batch, requires in the entire tile to fit into GPU memory. CPU parallelization is possible for loading images. - “window” loads only the desired window of the image from the raster dataset. Most memory efficient option, but cannot parallelize across windows.

• crop_model – a deepforest.model.CropModel object to predict on crops

Returns

Predictions dataframe or (predictions, crops) tuple

Return type

pd.DataFrame or tuple

save_model(path)

Save the trainer checkpoint in user defined path, in order to access in future.

Parameters

Path – the path located the model checkpoint

set_labels(label_dict)

Set new label mapping, updating both the label dictionary (str -> int) and its inverse (int -> str).

Parameters

label_dict (dict) – Dictionary mapping class names to numeric IDs.

setup_metrics()

train_dataloader()

Train loader using the configurations.

Returns

loader

training_step(batch, batch_idx)

Train on a loaded dataset.

val_dataloader()

Create a val data loader only if specified in config.

Returns

a dataloader or a empty iterable.

validation_step(batch, batch_idx)

Evaluate a batch.

deepforest.model module

class deepforest.model.BaseModel(config)

Bases: object

Base class for DeepForest models.

Provides common train, eval, and predict functionality. To add new architectures, create a module in models/ and implement create_model().

Parameters

config – DeepForest configuration object

check_model() → None

Validate model follows DeepForest guidelines.

Tests model with dummy data to ensure proper input/output format. Raises ValueError if validation fails.

create_model() → Module

Create model from configuration.

Must be implemented by subclasses to return a PyTorch nn.Module.

class deepforest.model.CropModel(*args, **kwargs)

Bases: LightningModule, PyTorchModelHubMixin

A PyTorch Lightning module for classifying image crops from object detection models.

This class provides a flexible architecture for training classification models on cropped regions identified by object detection models. It supports using either a default ResNet-50 model or a custom provided model.

Parameters

• model (nn.Module, optional) – Custom PyTorch model to use. If None, uses ResNet-50. Defaults to None.

• config (DictConfig, optional) – Full configuration object. If None, loads default config. Defaults to None.

• config_args (dict, optional) – Dictionary to override cropmodel config settings (e.g., {“resize”: [300, 300], “balance_classes”: True}). Defaults to None.

model

The classification model (ResNet-50 or custom)

Type

nn.Module

accuracy

Per-class accuracy metric

Type

torchmetrics.Accuracy

total_accuracy

Overall accuracy metric

Type

torchmetrics.Accuracy

precision_metric

Precision metric

Type

torchmetrics.Precision

metrics

Collection of all metrics

Type

torchmetrics.MetricCollection

label_dict

Label to index mapping {“Bird”: 0, “Mammal”: 1}

Type

dict

Create a new instance of the class and handle config.

3 cases: - If self._hub_mixin_config is already set, do nothing. - If config is passed as a dataclass, set it as self._hub_mixin_config. - Otherwise, build self._hub_mixin_config from default values and passed values.

configure_optimizers()

Choose what optimizers and learning-rate schedulers to use in your optimization. Normally you’d need one. But in the case of GANs or similar you might have multiple. Optimization with multiple optimizers only works in the manual optimization mode.

Returns

Any of these 6 options.

• Single optimizer.

• List or Tuple of optimizers.

• Two lists - The first list has multiple optimizers, and the second has multiple LR schedulers (or multiple lr_scheduler_config).

• Dictionary, with an "optimizer" key, and (optionally) a "lr_scheduler" key whose value is a single LR scheduler or lr_scheduler_config.

• None - Fit will run without any optimizer.

The lr_scheduler_config is a dictionary which contains the scheduler and its associated configuration. The default configuration is shown below.

lr_scheduler_config = {
    # REQUIRED: The scheduler instance
    "scheduler": lr_scheduler,
    # The unit of the scheduler's step size, could also be 'step'.
    # 'epoch' updates the scheduler on epoch end whereas 'step'
    # updates it after a optimizer update.
    "interval": "epoch",
    # How many epochs/steps should pass between calls to
    # `scheduler.step()`. 1 corresponds to updating the learning
    # rate after every epoch/step.
    "frequency": 1,
    # Metric to monitor for schedulers like `ReduceLROnPlateau`
    "monitor": "val_loss",
    # If set to `True`, will enforce that the value specified 'monitor'
    # is available when the scheduler is updated, thus stopping
    # training if not found. If set to `False`, it will only produce a warning
    "strict": True,
    # If using the `LearningRateMonitor` callback to monitor the
    # learning rate progress, this keyword can be used to specify
    # a custom logged name
    "name": None,
}

When there are schedulers in which the .step() method is conditioned on a value, such as the torch.optim.lr_scheduler.ReduceLROnPlateau scheduler, Lightning requires that the lr_scheduler_config contains the keyword "monitor" set to the metric name that the scheduler should be conditioned on.

# The ReduceLROnPlateau scheduler requires a monitor
def configure_optimizers(self):
    optimizer = Adam(...)
    return {
        "optimizer": optimizer,
        "lr_scheduler": {
            "scheduler": ReduceLROnPlateau(optimizer, ...),
            "monitor": "metric_to_track",
            "frequency": "indicates how often the metric is updated",
            # If "monitor" references validation metrics, then "frequency" should be set to a
            # multiple of "trainer.check_val_every_n_epoch".
        },
    }


# In the case of two optimizers, only one using the ReduceLROnPlateau scheduler
def configure_optimizers(self):
    optimizer1 = Adam(...)
    optimizer2 = SGD(...)
    scheduler1 = ReduceLROnPlateau(optimizer1, ...)
    scheduler2 = LambdaLR(optimizer2, ...)
    return (
        {
            "optimizer": optimizer1,
            "lr_scheduler": {
                "scheduler": scheduler1,
                "monitor": "metric_to_track",
            },
        },
        {"optimizer": optimizer2, "lr_scheduler": scheduler2},
    )

Metrics can be made available to monitor by simply logging it using self.log('metric_to_track', metric_val) in your LightningModule.

Note

Some things to know:

• Lightning calls .backward() and .step() automatically in case of automatic optimization.

• If a learning rate scheduler is specified in configure_optimizers() with key "interval" (default “epoch”) in the scheduler configuration, Lightning will call the scheduler’s .step() method automatically in case of automatic optimization.

• If you use 16-bit precision (precision=16), Lightning will automatically handle the optimizer.

• If you use torch.optim.LBFGS, Lightning handles the closure function automatically for you.

• If you use multiple optimizers, you will have to switch to ‘manual optimization’ mode and step them yourself.

• If you need to control how often the optimizer steps, override the optimizer_step() hook.

create_model(num_classes)

Create a model with the given number of classes.

create_trainer(**kwargs)

Create a pytorch lightning trainer object.

expand_bbox_to_square(bbox, image_width, image_height)

Expand a bounding box to a square by extending the shorter side.

Parameters:

bboxlist or tuple

Bounding box in format [xmin, ymin, xmax, ymax]

image_widthint

Width of the original image

image_heightint

Height of the original image

Returns:

list

Square bounding box in format [xmin, ymin, xmax, ymax]

forward(x)

Same as torch.nn.Module.forward().

Parameters

• *args – Whatever you decide to pass into the forward method.

• **kwargs – Keyword arguments are also possible.

Returns

Your model’s output

classmethod from_pretrained(repo_id: str, **kwargs)

Load a model from the Hugging Face Hub.

Parameters

• repo_id – Hugging Face repo id, e.g. “username/my-cropmodel”.

• **kwargs – Additional arguments to pass to the from_pretrained method.

get_transform(augmentations)

Returns the data transformation pipeline for the model.

Parameters

augmentations (str, list, dict, optional) – Augmentation configuration. If None, no augmentations are applied. If “HorizontalFlip” or [“HorizontalFlip”], applies random horizontal flip.

Returns

The composed data transformation pipeline.

Return type

torchvision.transforms.Compose

load_from_disk(train_dir, val_dir)

Load the training and validation datasets from disk.

Parameters

• train_dir (str) – The directory containing the training dataset.

• val_dir (str) – The directory containing the validation dataset.

Returns

None

classmethod load_model(repo_id, revision=None)

Load a model from the Hugging Face Hub.

Parameters

• repo_id – Hugging Face repo id, e.g. “username/my-cropmodel”.

• revision – Optional git revision/branch/tag. Defaults to repo default.

Returns

The loaded and eval-mode model instance.

Return type

CropModel

normalize()

on_load_checkpoint(checkpoint)

Called by Lightning to restore your model. If you saved something with on_save_checkpoint() this is your chance to restore this.

Parameters

checkpoint – Loaded checkpoint

Example:

def on_load_checkpoint(self, checkpoint):
    # 99% of the time you don't need to implement this method
    self.something_cool_i_want_to_save = checkpoint['something_cool_i_want_to_save']

Note

Lightning auto-restores global step, epoch, and train state including amp scaling. There is no need for you to restore anything regarding training.

on_save_checkpoint(checkpoint)

Called by Lightning when saving a checkpoint to give you a chance to store anything else you might want to save.

Parameters

checkpoint – The full checkpoint dictionary before it gets dumped to a file. Implementations of this hook can insert additional data into this dictionary.

Example:

def on_save_checkpoint(self, checkpoint):
    # 99% of use cases you don't need to implement this method
    checkpoint['something_cool_i_want_to_save'] = my_cool_pickable_object

Note

Lightning saves all aspects of training (epoch, global step, etc…) including amp scaling. There is no need for you to store anything about training.

on_validation_epoch_end()

Called in the validation loop at the very end of the epoch.

postprocess_predictions(predictions)

Postprocess predictions to get class labels and scores.

predict_dataloader(ds)

Prediction data loader.

predict_step(batch, batch_idx)

Step function called during predict(). By default, it calls forward(). Override to add any processing logic.

The predict_step() is used to scale inference on multi-devices.

To prevent an OOM error, it is possible to use BasePredictionWriter callback to write the predictions to disk or database after each batch or on epoch end.

The BasePredictionWriter should be used while using a spawn based accelerator. This happens for Trainer(strategy="ddp_spawn") or training on 8 TPU cores with Trainer(accelerator="tpu", devices=8) as predictions won’t be returned.

Parameters

• batch – The output of your data iterable, normally a DataLoader.

• batch_idx – The index of this batch.

• dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns

Predicted output (optional).

Example

class MyModel(LightningModule):

    def predict_step(self, batch, batch_idx, dataloader_idx=0):
        return self(batch)

dm = ...
model = MyModel()
trainer = Trainer(accelerator="gpu", devices=2)
predictions = trainer.predict(model, dm)

push_to_hub(repo_id, commit_message='Add model', **kwargs)

Upload model checkpoint to the Hub.

Use allow_patterns and ignore_patterns to precisely filter which files should be pushed to the hub. Use delete_patterns to delete existing remote files in the same commit. See [upload_folder] reference for more details.

Parameters

• repo_id (str) – ID of the repository to push to (example: “username/my-model”).

• config (dict or DataclassInstance, optional) – Model configuration specified as a key/value dictionary or a dataclass instance.

• commit_message (str, optional) – Message to commit while pushing.

• private (bool, optional) – Whether the repository created should be private. If None (default), the repo will be public unless the organization’s default is private.

• token (str, optional) – The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running hf auth login.

• branch (str, optional) – The git branch on which to push the model. This defaults to “main”.

• create_pr (boolean, optional) – Whether or not to create a Pull Request from branch with that commit. Defaults to False.

• allow_patterns (list[str] or str, optional) – If provided, only files matching at least one pattern are pushed.

• ignore_patterns (list[str] or str, optional) – If provided, files matching any of the patterns are not pushed.

• delete_patterns (list[str] or str, optional) – If provided, remote files matching any of the patterns will be deleted from the repo.

• model_card_kwargs (dict[str, Any], optional) – Additional arguments passed to the model card template to customize the model card.

Returns

The url of the commit of your model in the given repository.

push_to_hub_in_memory(repo_id, **kwargs)

Push the model to the Hugging Face Hub.

Parameters

• repo_id – Hugging Face repo id, e.g. “username/my-cropmodel”.

• **kwargs – Additional arguments to pass to the push_to_hub method.

train_dataloader()

Train data loader.

training_step(batch, batch_idx)

Here you compute and return the training loss and some additional metrics for e.g. the progress bar or logger.

Parameters

• batch – The output of your data iterable, normally a DataLoader.

• batch_idx – The index of this batch.

• dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns

• Tensor - The loss tensor

• dict - A dictionary which can include any keys, but must include the key 'loss' in the case of automatic optimization.

• None - In automatic optimization, this will skip to the next batch (but is not supported for multi-GPU, TPU, or DeepSpeed). For manual optimization, this has no special meaning, as returning the loss is not required.

In this step you’d normally do the forward pass and calculate the loss for a batch. You can also do fancier things like multiple forward passes or something model specific.

Example:

def training_step(self, batch, batch_idx):
    x, y, z = batch
    out = self.encoder(x)
    loss = self.loss(out, x)
    return loss

To use multiple optimizers, you can switch to ‘manual optimization’ and control their stepping:

def __init__(self):
    super().__init__()
    self.automatic_optimization = False


# Multiple optimizers (e.g.: GANs)
def training_step(self, batch, batch_idx):
    opt1, opt2 = self.optimizers()

    # do training_step with encoder
    ...
    opt1.step()
    # do training_step with decoder
    ...
    opt2.step()

Note

When accumulate_grad_batches > 1, the loss returned here will be automatically normalized by accumulate_grad_batches internally.

val_dataloader()

Validation data loader.

val_dataset_confusion(return_images=False)

Create a labels and predictions from the validation dataset to be created into a confusion matrix.

validation_step(batch, batch_idx)

Operates on a single batch of data from the validation set. In this step you’d might generate examples or calculate anything of interest like accuracy.

Parameters

• batch – The output of your data iterable, normally a DataLoader.

• batch_idx – The index of this batch.

• dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns

• Tensor - The loss tensor

• dict - A dictionary. Can include any keys, but must include the key 'loss'.

• None - Skip to the next batch.

# if you have one val dataloader:
def validation_step(self, batch, batch_idx): ...


# if you have multiple val dataloaders:
def validation_step(self, batch, batch_idx, dataloader_idx=0): ...

Examples:

# CASE 1: A single validation dataset
def validation_step(self, batch, batch_idx):
    x, y = batch

    # implement your own
    out = self(x)
    loss = self.loss(out, y)

    # log 6 example images
    # or generated text... or whatever
    sample_imgs = x[:6]
    grid = torchvision.utils.make_grid(sample_imgs)
    self.logger.experiment.add_image('example_images', grid, 0)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    val_acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs!
    self.log_dict({'val_loss': loss, 'val_acc': val_acc})

If you pass in multiple val dataloaders, validation_step() will have an additional argument. We recommend setting the default value of 0 so that you can quickly switch between single and multiple dataloaders.

# CASE 2: multiple validation dataloaders
def validation_step(self, batch, batch_idx, dataloader_idx=0):
    # dataloader_idx tells you which dataset this is.
    x, y = batch

    # implement your own
    out = self(x)

    if dataloader_idx == 0:
        loss = self.loss0(out, y)
    else:
        loss = self.loss1(out, y)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs separately for each dataloader
    self.log_dict({f"val_loss_{dataloader_idx}": loss, f"val_acc_{dataloader_idx}": acc})

Note

If you don’t need to validate you don’t need to implement this method.

Note

When the validation_step() is called, the model has been put in eval mode and PyTorch gradients have been disabled. At the end of validation, the model goes back to training mode and gradients are enabled.

write_crops(root_dir, images, boxes, labels, savedir)

Write crops to disk.

Parameters

• root_dir (str) – The root directory where the images are located.

• images (list) – A list of image filenames.

• boxes (list) – A list of bounding box coordinates in the format [xmin, ymin, xmax, ymax].

• labels (list) – A list of labels corresponding to each bounding box.

• savedir (str) – The directory where the cropped images will be saved.

Returns

None

deepforest.model.simple_resnet_50(num_classes: int = 2) → Module

Create a simple ResNet-50 model for classification.

Parameters

num_classes – Number of output classes for the final layer

Returns

ResNet-50 model with modified final layer

Return type

torch.nn.Module

deepforest.predict module

deepforest.predict.across_class_nms(predicted_boxes, iou_threshold=0.15)

Perform non-max suppression for a dataframe of results (see visualize.format_boxes) to remove boxes that overlap by iou_thresholdold of IoU.

deepforest.predict.apply_nms(boxes, scores, labels, iou_threshold)

Apply non-maximum suppression to boxes.

Parameters

• boxes – tensor of shape (N, 4) containing box coordinates

• scores – tensor of shape (N,) containing confidence scores

• labels – array of shape (N,) containing labels

• iou_threshold – IoU threshold for NMS

Returns

DataFrame with filtered boxes

deepforest.predict.mosiac(predictions, iou_threshold=0.1)

Mosaic predictions from overlapping windows.

Parameters

• predictions – A pandas dataframe containing predictions from overlapping windows from a single image.

• iou_threshold – The IoU threshold for non-max suppression.

Returns

A pandas dataframe of predictions.

deepforest.predict.transform_coordinates(boxes)

Transform box coordinates from window space to original image space.

Parameters

boxes – DataFrame of predictions with xmin, ymin, xmax, ymax, window_xmin, window_ymin columns

Returns

DataFrame with transformed coordinates

deepforest.preprocess module

The preprocessing module is used to reshape data into format suitable for training or prediction.

For example cutting large tiles into smaller images.

deepforest.preprocess.compute_windows(numpy_image, patch_size, patch_overlap)

Create a sliding window object from a raster tile.

Parameters

numpy_image (array) – Raster object as numpy array to cut into crops, channels first

Returns

a sliding windows object

Return type

windows (list)

deepforest.preprocess.create_empty_annotation(image_annotations, image_basename, index)

Create empty annotation record when allow_empty=True.

deepforest.preprocess.image_name_from_path(image_path)

Convert path to image name for use in indexing.

deepforest.preprocess.load_annotations(annotations_file, root_dir)

Load and validate annotations file.

Parameters

• annotations_file – Path to file, DataFrame, or GeoDataFrame

• root_dir – Root directory for relative paths

Returns

GeoDataFrame with annotations

deepforest.preprocess.preprocess_image(image)

Preprocess a single RGB numpy array as a prediction from channels last, to channels first.

deepforest.preprocess.process_with_annotations(numpy_image, windows, image_annotations, image_name, save_dir, allow_empty)

Process raster with annotations.

deepforest.preprocess.process_without_annotations(numpy_image, windows, image_name, save_dir)

Process raster without annotations.

deepforest.preprocess.save_crop(base_dir, image_name, index, crop)

Save window crop as an image file to be read by PIL.

Parameters

• base_dir (str) – The base directory to save the image file.

• image_name (str) – The name of the original image.

• index (int) – The index of the window crop.

• crop (numpy.ndarray) – The window crop as a NumPy array.

Returns

The filename of the saved image.

Return type

str

deepforest.preprocess.select_annotations(annotations, window)

Select annotations that overlap with selected image crop.

Parameters

• annotations – a geopandas dataframe of annotations with a geometry column

• windows – A sliding window object (see compute_windows)

Returns

a pandas dataframe of annotations

Return type

selected_annotations

deepforest.preprocess.split_raster(annotations_file=None, path_to_raster=None, numpy_image=None, root_dir=None, patch_size=400, patch_overlap=0.05, allow_empty=False, image_name=None, save_dir='.')

Split a large raster into smaller patches for processing.

Parameters

• annotations_file – Path to annotations CSV or DataFrame with columns: image_path, xmin, ymin, xmax, ymax, label

• path_to_raster – Path to raster file on disk

• numpy_image – Numpy array in (channels, height, width) order

• root_dir – Root directory for annotations file

• patch_size – Size of square patches

• patch_overlap – Overlap between patches (0-1)

• allow_empty – Include patches with no annotations

• image_name – Name for the raster image

• save_dir – Directory to save patches

Returns

DataFrame with annotations for training, or list of patch filenames

deepforest.preprocess.validate_annotations(annotations, numpy_image, path_to_raster)

Validate annotation coordinate systems and bounds.

deepforest.utilities module

class deepforest.utilities.DeepForest_DataFrame(*args, **kwargs)

Bases: GeoDataFrame

Custom GeoDataFrame that preserves a root_dir attribute if present.

class deepforest.utilities.DownloadProgressBar(*_, **__)

Bases: tqdm

Download progress bar class.

update_to(b=1, bsize=1, tsize=None)

Update class attributes :param b: :param bsize: :param tsize:

Returns:

deepforest.utilities.check_image(image)

Check an image is three channel, channel last format :param image: numpy array

Returns: None, throws error on assert

deepforest.utilities.collate_fn(batch)

deepforest.utilities.convert_point_to_bbox(gdf: GeoDataFrame, buffer_size: float) → GeoDataFrame

Convert an input point type annotation to a bounding box by buffering the point with a fixed size.

Parameters

• gdf (GeoDataFrame) – The input point type annotation.

• buffer_size (float) – The size of the buffer to be applied to the point.

Returns

The output bounding box type annotation.

Return type

gdf (GeoDataFrame)

deepforest.utilities.crop_annotations_to_bounds(gdf, bounds)

Crop a geodataframe of annotations to a bounding box :param gdf: a geodataframe of annotations :param bounds: a tuple of (left, bottom, right, top) bounds

Returns

a geodataframe of annotations cropped to the bounds

Return type

gdf

deepforest.utilities.crop_raster(bounds, rgb_path=None, savedir=None, filename=None, driver='GTiff')

Crop a raster to a bounding box, save as projected or unprojected crop :param bounds: a tuple of (left, bottom, right, top) bounds :param rgb_path: path to the rgb image :param savedir: directory to save the crop :param filename: filename to save the crop “{}.tif”.format(filename)” :param driver: rasterio driver to use, default to GTiff, can be ‘GTiff’ for projected data or ‘PNG’ unprojected data

Returns

path to the saved crop, if savedir specified img: a numpy array of the crop, if savedir not specified

Return type

filename

deepforest.utilities.determine_geometry_type(df)

Determine the geometry type of a prediction or annotation :param df: a pandas dataframe

Returns

a string of the geometry type

Return type

geometry_type

deepforest.utilities.format_boxes(prediction, scores=True)

Format a retinanet prediction into a pandas dataframe for a single image.

Parameters

• prediction – a dictionary with keys ‘boxes’ and ‘labels’ coming from a retinanet

• scores – Whether boxes come with scores, during prediction, or without scores, as in during training.

Returns

a pandas dataframe

Return type

df

deepforest.utilities.format_geometry(predictions, scores=True, geom_type=None)

Format a retinanet prediction into a pandas dataframe for a batch of images :param predictions: a list of dictionaries with keys ‘boxes’ and ‘labels’ coming from a retinanet :param scores: Whether boxes come with scores, during prediction, or without scores, as in during training.

Returns

a pandas dataframe None if the dataframe is empty

Return type

df

deepforest.utilities.geo_to_image_coordinates(gdf, image_bounds, image_resolution)

Convert from projected coordinates to image coordinates :param gdf: a pandas type dataframe with columns: name, xmin, ymin, xmax, ymax. Name is the relative path to the root_dir arg. :param image_bounds: bounds of the image :param image_resolution: resolution of the image

Returns

a geopandas dataframe with the transformed to image origin. CRS is removed

Return type

gdf

deepforest.utilities.image_to_geo_coordinates(gdf, root_dir=None, flip_y_axis=False)

Convert from image coordinates to geographic coordinates.

Parameters

• gdf – A geodataframe.

• root_dir – Directory of images to lookup image_path column. If None, it will attempt to use gdf.root_dir.

• flip_y_axis – If True, reflect predictions over y axis to align with raster data in QGIS, which uses a negative y origin compared to numpy.

Returns

A geospatial dataframe with the boxes optionally transformed to the target crs.

Return type

transformed_gdf

deepforest.utilities.load_config(config_name: str = 'config.yaml', overrides: omegaconf.dictconfig.DictConfig | dict | None = None, strict: bool = False) → DictConfig

Loads the DeepForest structured config, merges with YAML and overrides.

If config_name is found to be a valid path, it will be loaded, otherwise it’s assumed to be a named config in the deepforest package. The loaded config will be validated against the schema.

You can load a config in strict mode, which will not allow any additional keys. This may be useful for debugging, but it may cause issues due to the way OmegaConf handles dictionary config items, like label_dict.

Parameters

• config_name (str) – Path to config file

• overrides (DictConfig or dict) – Overrides to config

• strict (bool) – If True, disallows unexpected keys.

Returns

composed configuration

Return type

config (DictConfig)

deepforest.utilities.read_coco(json_file)

Read a COCO format JSON file and return a pandas dataframe.

Parameters

json_file – Path to the COCO segmentation JSON file

Returns

A pandas dataframe with image_path, geometry, and label columns

Return type

df

deepforest.utilities.read_file(input: str | pandas.DataFrame, root_dir: str | None = None, image_path: str | None = None, label: str | None = None) → GeoDataFrame

Read file and return GeoDataFrame in DeepForest format.

Parameters

• input – Path to file, DataFrame, or GeoDataFrame

• root_dir – Root directory for image files

• image_path – Path relative to root_dir to a single image that will be assigned as the image_path column for all annotations. The full path will be constructed by joining the root_dir and the image_path. Overrides any image_path column in input.

• label – Single label to be assigned as the label for all annotations. Overrides any label column in input.

Returns

GeoDataFrame with geometry, image_path, and label columns

deepforest.utilities.read_pascal_voc(xml_path)

Load annotations from xml format (e.g. RectLabel editor) and convert them into retinanet annotations format.

Parameters

xml_path (str) – Path to the annotations xml, formatted by RectLabel

Returns

in the

format -> path-to-image.png,x1,y1,x2,y2,class_name

Return type

Annotations (pandas dataframe)

deepforest.utilities.round_with_floats(x)

Check if string x is float or int, return int, rounded if needed.

deepforest.utilities.shapefile_to_annotations(shapefile: str | geopandas.geodataframe.GeoDataFrame, rgb: str | None = None, root_dir: str | None = None, buffer_size: float | None = None, convert_point: bool = False, label: str | None = None) → GeoDataFrame

deepforest.visualize module

deepforest.visualize.convert_to_sv_format(df: DataFrame, image: numpy.ndarray[tuple[Any, ...], numpy.dtype[numpy._typing._array_like._ScalarT]] | None = None) → supervision.detection.core.Detections | supervision.key_points.core.KeyPoints

Convert DeepForest prediction results into a supervision object.

Parameters

• df (pd.DataFrame) – Results from predict_image or predict_tile. Must include columns: [‘geometry’, ‘label’, ‘score’, ‘image_path’].

• width (int, optional) – Image width in pixels. Required for polygon geometry.

• height (int, optional) – Image height in pixels. Required for polygon geometry.

Returns

Object type depends on geometry.

Return type

sv.Detections | sv.KeyPoints

deepforest.visualize.label_to_color(label: int) → tuple

Return an RGB color tuple for a given (integer) label.

deepforest.visualize.plot_annotations(annotations: DataFrame, savedir: str | None = None, color: list | supervision.draw.color.ColorPalette | None = None, thickness: int = 2, basename: str | None = None, root_dir: str | None = None, radius: int = 3, image: numpy.ndarray[tuple[Any, ...], numpy.dtype[numpy._typing._array_like._ScalarT]] | str | PIL.Image.Image | None = None, show: bool = True) → Figure

Plot prediction results or ground truth annotations for a single image.

Parameters

• annotations – DataFrame with annotations

• savedir – Directory to save plot

• color – Color for annotations

• thickness – Line thickness

• basename – Base name for saved file

• root_dir – Root directory for images

• radius – Point radius

• image – Image array or path

• show – Whether to display the plot (default: True). Set to False for testing.

Returns

The figure object for further customization

Return type

matplotlib.figure.Figure

deepforest.visualize.plot_results(results: DataFrame, ground_truth: pandas.DataFrame | None = None, savedir: str | None = None, results_color: list | supervision.draw.color.ColorPalette | None = None, ground_truth_color: list | supervision.draw.color.ColorPalette | None = None, thickness: int = 2, basename: str | None = None, radius: int = 3, image: numpy.ndarray[tuple[Any, ...], numpy.dtype[numpy._typing._array_like._ScalarT]] | str | PIL.Image.Image | None = None, axes: bool = False, show: bool = True)

Plot predicted annotations with optional ground truth.

Creates a figure that can be displayed or saved. Pass axes=True to return the Matplotlib Axes for additional plotting.

Parameters

• results – Pandas DataFrame of prediction results.

• ground_truth – Optional DataFrame of ground-truth annotations.

• savedir – Optional path to save the figure; if None, plots interactively.

• results_color – Single RGB list (e.g., [245, 135, 66]) or an sv.ColorPalette for per-label colors.

• ground_truth_color – Single RGB list (e.g., [0, 165, 255]) or an sv.ColorPalette.

• thickness – Line thickness in pixels.

• basename – Optional base name for the saved figure; if None, derived from the image path.

• radius – Point marker radius in pixels.

• image – Optional NumPy array, image path, or PIL Image to annotate; if None, loaded from the results DataFrame.

• axes – If True, return the Matplotlib Axes object.

• show – Whether to display the plot (default: True). Set to False for testing.

Returns

The Figure (default) or Axes (when axes=True).

Return type

matplotlib.figure.Figure | matplotlib.axes.Axes

Module contents

Top-level package for DeepForest.