deepforest package#

Subpackages#

deepforest.data package
- Module contents

Submodules#

deepforest.IoU module#

IoU Module, with help from SpaceNetChallenge/utilities

deepforest.IoU.compute_IoU(ground_truth, submission)[source]#

Parameters:

ground_truth – a projected geopandas dataframe with geoemtry
submission – a projected geopandas dataframe with geometry

Returns:

dataframe of IoU scores

Return type:

iou_df

deepforest.IoU.create_rtree_from_poly(poly_list)[source]#

deepforest.callbacks module#

A deepforest callback Callbacks must have the following methods on_epoch_begin, on_epoch_end, on_fit_end, on_fit_begin methods and inject model and epoch kwargs.

class deepforest.callbacks.images_callback(savedir, n=2, every_n_epochs=5, select_random=False, color=None, thickness=1)[source]#

Bases: Callback

Run evaluation on a file of annotations during training.

Parameters:

savedir – optional, directory to save predicted images
probability_threshold – minimum probablity for inclusion, see deepforest.evaluate
n – number of images to upload
select_random (False) – whether to select random images or the first n images
every_n_epochs – run epoch interval
color – color of the bounding box as a tuple of BGR color, e.g. orange annotations is (0, 165, 255)
thickness – thickness of the rectangle border line in px

Returns:

either prints validation scores or logs them to the pytorch-lightning logger

Return type:

None

log_images(pl_module)[source]#

on_validation_epoch_end(trainer, pl_module)[source]#: Called when the val epoch ends.

deepforest.dataset module#

Dataset model.

https://pytorch.org/docs/stable/torchvision/models.html#object-detection-instance-segmentation-and-person-keypoint-detection

During training, the model expects both the input tensors, as well as a targets (list of dictionary), containing:

boxes (FloatTensor[N, 4]): the ground-truth boxes in [x1, y1, x2, y2] format, with values between 0 and H and 0 and W

labels (Int64Tensor[N]): the class label for each ground-truth box

https://colab.research.google.com/github/benihime91/pytorch_retinanet/blob/master/demo.ipynb#scrollTo=0zNGhr6D7xGN

class deepforest.dataset.BoundingBoxDataset(df, root_dir, transform=None, augment=False)[source]#

Bases: Dataset

An in memory dataset for bounding box predictions.

Parameters:

df – a pandas dataframe with image_path and xmin,xmax,ymin,ymax columns
transform – a function to apply to the image
root_dir – the directory where the image is stored

Returns:

a tensor of shape (3, height, width)

Return type:

rgb

class deepforest.dataset.RasterDataset(raster_path, patch_size, patch_overlap)[source]#

Bases: object

Dataset for predicting on raster windows.

Parameters:

raster_path (str) – Path to raster file
patch_size (int) – Size of windows to predict on
patch_overlap (float) – Overlap between windows as fraction (0-1)

Returns:

A dataset of raster windows

class deepforest.dataset.TileDataset(tile: ndarray | None, preload_images: bool = False, patch_size: int = 400, patch_overlap: float = 0.05)[source]#

Bases: Dataset

Parameters:

tile – an in memory numpy array.
patch_size (int) – The size for the crops used to cut the input raster into smaller pieces. This is given in pixels, not any geographic unit.
patch_overlap (float) – The horizontal and vertical overlap among patches
preload_images (bool) – If true, the entire dataset is loaded into memory. This is useful for small datasets, but not recommended for large datasets since both the tile and the crops are stored in memory.

Returns:

a pytorch dataset

Return type:

ds

class deepforest.dataset.TreeDataset(csv_file, root_dir, transforms=None, label_dict={'Tree': 0}, train=True, preload_images=False)[source]#

Bases: Dataset

Parameters:

csv_file (string) – Path to a single csv file with annotations.
root_dir (string) – Directory with all the images.
transform (callable, optional) – Optional transform to be applied on a sample.
label_dict – a dictionary where keys are labels from the csv column and values are numeric labels “Tree” -> 0

Returns:

If train, path, image, targets else image

deepforest.dataset.bounding_box_transform(augment=False)[source]#

deepforest.dataset.get_transform(augment)[source]#: Albumentations transformation of bounding boxs.

deepforest.evaluate module#

Evaluation module.

deepforest.evaluate.compute_class_recall(results)[source]#

Given a set of evaluations, what proportion of predicted boxes match.

True boxes which are not matched to predictions do not count against accuracy.

deepforest.evaluate.evaluate_boxes(predictions, ground_df, root_dir, iou_threshold=0.4, savedir=None)[source]#

Image annotated crown evaluation routine submission can be submitted as a .shp, existing pandas dataframe or .csv path.

Parameters:

predictions – a pandas dataframe, if supplied a root dir is needed to give the relative path of files in df.name. The labels in ground truth and predictions must match. If one is numeric, the other must be numeric.
ground_df – a pandas dataframe, if supplied a root dir is needed to give the relative path of files in df.name
root_dir – location of files in the dataframe ‘name’ column.
savedir – optional directory to save image with overlaid predictions and annotations

Returns:

a dataframe of match bounding boxes box_recall: proportion of true positives of box position, regardless of class box_precision: proportion of predictions that are true positive, regardless of class class_recall: a pandas dataframe of class level recall and precision with class sizes

Return type:

results

deepforest.evaluate.evaluate_image_boxes(predictions, ground_df, root_dir, savedir=None)[source]#

Compute intersection-over-union matching among prediction and ground truth boxes for one image.

Parameters:

df – a geopandas dataframe with geometry columns
predictions – a geopandas dataframe with geometry columns
ground_df – a geopandas dataframe with geometry columns
summarize – Whether to group statistics by plot and overall score
image_coordinates – Whether the current boxes are in coordinate system of the image, e.g. origin (0,0) upper left.
root_dir – Where to search for image names in df
savedir – optional directory to save image with overlaid predictions and annotations

Returns:

pandas dataframe with crown ids of prediciton and ground truth and the IoU score.

Return type:

result

deepforest.evaluate.point_recall(predictions, ground_df, root_dir=None, savedir=None)[source]#

Evaluate the proportion on ground truth points overlap with predictions submission can be submitted as a .shp, existing pandas dataframe or .csv path For bounding box recall, see evaluate().

Parameters:

predictions – a pandas dataframe, if supplied a root dir is needed to give the relative path of files in df.name. The labels in ground truth and predictions must match. If one is numeric, the other must be numeric.
ground_df – a pandas dataframe, if supplied a root dir is needed to give the relative path of files in df.name
root_dir – location of files in the dataframe ‘name’ column.
savedir – optional directory to save image with overlaid predictions and annotations

Returns:

a dataframe of matched bounding boxes and ground truth labels box_recall: proportion of true positives between predicted boxes and ground truth points, regardless of class class_recall: a pandas dataframe of class level recall and precision with class sizes

Return type:

results

deepforest.main module#

class deepforest.main.deepforest(*args, **kwargs)[source]#

Bases: LightningModule, PyTorchModelHubMixin

Class for training and predicting tree crowns in RGB images.

Parameters:

num_classes (int) – number of classes in the model
config_file (str) – path to deepforest config file
model (model.Model()) – a deepforest model object, see model.Model()
config_args (dict) – a dictionary of key->value to update config file at run time. e.g. {“batch_size”:10}. This is useful for iterating over arguments during model testing.
existing_train_dataloader – a Pytorch dataloader that yields a tuple path, images, targets
existing_val_dataloader – a Pytorch dataloader that yields a tuple path, images, targets

Returns:

a deepforest pytorch lightning module

Return type:

self

Create a new instance of the class and handle config.

3 cases: - If self._hub_mixin_config is already set, do nothing. - If config is passed as a dataclass, set it as self._hub_mixin_config. - Otherwise, build self._hub_mixin_config from default values and passed values.

calculate_empty_frame_accuracy(ground_df, predictions_df)[source]#

Calculate accuracy for empty frames (frames with no objects).

Parameters:

ground_df (pd.DataFrame) – Ground truth dataframe containing image paths and bounding boxes. Must have columns ‘image_path’, ‘xmin’, ‘ymin’, ‘xmax’, ‘ymax’.
predictions_df (pd.DataFrame) – Model predictions dataframe containing image paths and predicted boxes. Must have column ‘image_path’.

Returns:

Accuracy score for empty frame detection. A score of 1.0 means the model correctly: identified all empty frames (no false positives), while 0.0 means it predicted objects in all empty frames (all false positives). Returns None if there are no empty frames.

Return type:

float or None

configure_optimizers()[source]#

Choose what optimizers and learning-rate schedulers to use in your optimization. Normally you’d need one. But in the case of GANs or similar you might have multiple. Optimization with multiple optimizers only works in the manual optimization mode.

Returns:

Any of these 6 options.

Single optimizer.
List or Tuple of optimizers.
Two lists - The first list has multiple optimizers, and the second has multiple LR schedulers (or multiple lr_scheduler_config).
Dictionary, with an "optimizer" key, and (optionally) a "lr_scheduler" key whose value is a single LR scheduler or lr_scheduler_config.
None - Fit will run without any optimizer.

The lr_scheduler_config is a dictionary which contains the scheduler and its associated configuration. The default configuration is shown below.

lr_scheduler_config = {
    # REQUIRED: The scheduler instance
    "scheduler": lr_scheduler,
    # The unit of the scheduler's step size, could also be 'step'.
    # 'epoch' updates the scheduler on epoch end whereas 'step'
    # updates it after a optimizer update.
    "interval": "epoch",
    # How many epochs/steps should pass between calls to
    # `scheduler.step()`. 1 corresponds to updating the learning
    # rate after every epoch/step.
    "frequency": 1,
    # Metric to to monitor for schedulers like `ReduceLROnPlateau`
    "monitor": "val_loss",
    # If set to `True`, will enforce that the value specified 'monitor'
    # is available when the scheduler is updated, thus stopping
    # training if not found. If set to `False`, it will only produce a warning
    "strict": True,
    # If using the `LearningRateMonitor` callback to monitor the
    # learning rate progress, this keyword can be used to specify
    # a custom logged name
    "name": None,
}

When there are schedulers in which the .step() method is conditioned on a value, such as the torch.optim.lr_scheduler.ReduceLROnPlateau scheduler, Lightning requires that the lr_scheduler_config contains the keyword "monitor" set to the metric name that the scheduler should be conditioned on.

# The ReduceLROnPlateau scheduler requires a monitor
def configure_optimizers(self):
    optimizer = Adam(...)
    return {
        "optimizer": optimizer,
        "lr_scheduler": {
            "scheduler": ReduceLROnPlateau(optimizer, ...),
            "monitor": "metric_to_track",
            "frequency": "indicates how often the metric is updated",
            # If "monitor" references validation metrics, then "frequency" should be set to a
            # multiple of "trainer.check_val_every_n_epoch".
        },
    }


# In the case of two optimizers, only one using the ReduceLROnPlateau scheduler
def configure_optimizers(self):
    optimizer1 = Adam(...)
    optimizer2 = SGD(...)
    scheduler1 = ReduceLROnPlateau(optimizer1, ...)
    scheduler2 = LambdaLR(optimizer2, ...)
    return (
        {
            "optimizer": optimizer1,
            "lr_scheduler": {
                "scheduler": scheduler1,
                "monitor": "metric_to_track",
            },
        },
        {"optimizer": optimizer2, "lr_scheduler": scheduler2},
    )

Metrics can be made available to monitor by simply logging it using self.log('metric_to_track', metric_val) in your LightningModule.

Note

Some things to know:

Lightning calls .backward() and .step() automatically in case of automatic optimization.
If a learning rate scheduler is specified in configure_optimizers() with key "interval" (default “epoch”) in the scheduler configuration, Lightning will call the scheduler’s .step() method automatically in case of automatic optimization.
If you use 16-bit precision (precision=16), Lightning will automatically handle the optimizer.
If you use torch.optim.LBFGS, Lightning handles the closure function automatically for you.
If you use multiple optimizers, you will have to switch to ‘manual optimization’ mode and step them yourself.
If you need to control how often the optimizer steps, override the optimizer_step() hook.

create_model()[source]#

Define a deepforest architecture. This can be done in two ways. Passed as the model argument to deepforest __init__(), or as a named architecture in config[“architecture”], which corresponds to a file in models/, as is a subclass of model.Model(). The config args in the .yaml are specified.

Returns:: None

create_trainer(logger=None, callbacks=[], **kwargs)[source]#

Create a pytorch lightning training by reading config files.

Parameters:

logger – A pytorch lightning logger
callbacks (list) – a list of pytorch-lightning callback classes
**kwargs – Additional arguments to pass to the trainer

Returns:

None

evaluate(csv_file, root_dir, iou_threshold=None, savedir=None)[source]#

Compute intersection-over-union and precision/recall for a given iou_threshold.

Parameters:

csv_file – location of a csv file with columns “name”,”xmin”,”ymin”,”xmax”,”ymax”,”label”
root_dir – location of files in the dataframe ‘name’ column
iou_threshold – float [0,1] intersection-over-union threshold for true positive
savedir – location to save images with bounding boxes

Returns:

Results dictionary containing precision, recall and other metrics

Return type:

dict

load_dataset(csv_file, root_dir=None, augment=False, shuffle=True, batch_size=1, train=False)[source]#

Create a tree dataset for inference Csv file format is .csv file with the columns “image_path”, “xmin”,”ymin”,”xmax”,”ymax” for the image name and bounding box position. Image_path is the relative filename, not absolute path, which is in the root_dir directory. One bounding box per line.

Parameters:

csv_file – path to csv file
root_dir – directory of images. If none, uses “image_dir” in config
augment – Whether to create a training dataset, this activates data augmentations

Returns:

a pytorch dataset

Return type:

ds

load_model(model_name='weecology/deepforest-tree', revision='main')[source]#

Loads a model that has already been pretrained for a specific task, like tree crown detection.

Models (technically model weights) are distributed via Hugging Face and designated the Hugging Face repository ID (model_name), which is in the form: ‘organization/repository’. For a list of models distributed by the DeepForest team (and the associated model names) see the documentation: https://deepforest.readthedocs.io/en/latest/installation_and_setup/prebuilt.html

Parameters:

model_name (str) – A repository ID for huggingface in the form of organization/repository
revision (str) – The model version (‘main’, ‘v1.0.0’, etc.).

Returns:

None

on_fit_start()[source]#

Called at the very beginning of fit.

If on DDP it is called on every process

on_validation_epoch_end()[source]#: Compute metrics.

on_validation_epoch_start()[source]#: Called in the validation loop at the very beginning of the epoch.

predict_batch(images, preprocess_fn=None)[source]#

Predict a batch of images with the deepforest model.

Parameters:

images (torch.Tensor or np.ndarray) – A batch of images with shape (B, C, H, W) or (B, H, W, C).
preprocess_fn (callable, optional) – A function to preprocess images before prediction. If None, assumes images are preprocessed.

Returns:

A list of dataframes with predictions for each image.

Return type:

List[pd.DataFrame]

predict_dataloader(ds)[source]#

Create a PyTorch dataloader for prediction.

Parameters:: ds (torchvision.datasets.Dataset) – A torchvision dataset to be wrapped into a dataloader using config args.
Returns:: A dataloader object that can be used for prediction.
Return type:: torch.utils.data.DataLoader

predict_file(csv_file, root_dir, savedir=None, color=None, thickness=1)[source]#

Create a dataset and predict entire annotation file Csv file format is .csv file with the columns “image_path”, “xmin”,”ymin”,”xmax”,”ymax” for the image name and bounding box position. Image_path is the relative filename, not absolute path, which is in the root_dir directory. One bounding box per line.

Deprecation warning: The return_plot argument is deprecated and will be removed in 2.0. Use visualize.plot_results on the result instead.

Parameters:

csv_file – path to csv file
root_dir – directory of images. If none, uses “image_dir” in config
savedir ((deprecated)) – directory to save images with bounding boxes
color ((deprecated)) – color of the bounding box as a tuple of BGR color, e.g. orange annotations is (0, 165, 255)
thickness ((deprecated)) – thickness of the rectangle border line in px

Returns:

pandas dataframe with bounding boxes, label and scores for each image in the csv file

Return type:

df

predict_image(image: ndarray | None = None, path: str | None = None, return_plot: bool = False, color: tuple | None = (0, 165, 255), thickness: int = 1)[source]#

Predict a single image with a deepforest model.

Deprecation warning: The ‘return_plot’, and related ‘color’ and ‘thickness’ arguments are deprecated and will be removed in 2.0. Use visualize.plot_results on the result instead.

Parameters:

image – a float32 numpy array of a RGB with channels last format
path – optional path to read image from disk instead of passing image arg
return_plot – return a plot of the image with predictions overlaid (deprecated)
color – color of the bounding box as a tuple of BGR color (deprecated)
thickness – thickness of the rectangle border line in px (deprecated)

Returns:

A pandas dataframe of predictions (Default) img: The input with predictions overlaid (Optional)

Return type:

result

predict_step(batch, batch_idx)[source]#

Step function called during predict(). By default, it calls forward(). Override to add any processing logic.

The predict_step() is used to scale inference on multi-devices.

To prevent an OOM error, it is possible to use BasePredictionWriter callback to write the predictions to disk or database after each batch or on epoch end.

The BasePredictionWriter should be used while using a spawn based accelerator. This happens for Trainer(strategy="ddp_spawn") or training on 8 TPU cores with Trainer(accelerator="tpu", devices=8) as predictions won’t be returned.

Parameters:

batch – The output of your data iterable, normally a DataLoader.
batch_idx – The index of this batch.
dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns:

Predicted output (optional).

Example

class MyModel(LightningModule):

    def predict_step(self, batch, batch_idx, dataloader_idx=0):
        return self(batch)

dm = ...
model = MyModel()
trainer = Trainer(accelerator="gpu", devices=2)
predictions = trainer.predict(model, dm)

predict_tile(raster_path=None, image=None, patch_size=400, patch_overlap=0.05, iou_threshold=0.15, in_memory=True, return_plot=False, mosaic=True, sigma=0.5, thresh=0.001, color=None, thickness=1, crop_model=None, crop_transform=None, crop_augment=False)[source]#

For images too large to input into the model, predict_tile cuts the image into overlapping windows, predicts trees on each window and reassambles into a single array.

Parameters:

raster_path – Path to image on disk
image (array) – Numpy image array in BGR channel order following openCV convention
patch_size – patch size for each window
patch_overlap – patch overlap among windows
iou_threshold – Minimum iou overlap among predictions between windows to be suppressed
in_memory – If true, the entire dataset is loaded into memory
mosaic – Return a single prediction dataframe (True) or a tuple of image crops and predictions (False)
sigma – variance of Gaussian function used in Gaussian Soft NMS
thresh – the score thresh used to filter bboxes after soft-nms performed
crop_model – a deepforest.model.CropModel object to predict on crops
crop_transform – a torchvision.transforms object to apply to crops
crop_augment – a boolean to apply augmentations to crops
return_plot – return a plot of the image with predictions overlaid (deprecated)
color – color of the bounding box as a tuple of BGR color (deprecated)
thickness – thickness of the rectangle border line in px (deprecated)

Returns:

Predictions dataframe or (predictions, crops) tuple

Return type:

pd.DataFrame or tuple

save_model(path)[source]#

Save the trainer checkpoint in user defined path, in order to access in future.

Parameters:: Path – the path located the model checkpoint

set_labels(label_dict)[source]#

Set new label mapping, updating both the label dictionary (str -> int) and its inverse (int -> str).

Parameters:: label_dict (dict) – Dictionary mapping class names to numeric IDs.

train_dataloader()[source]#

Train loader using the configurations.

Returns:: loader

training_step(batch, batch_idx)[source]#: Train on a loaded dataset.

use_bird_release(check_release=True)[source]#

Use the latest DeepForest bird model release from Hugging Face, downloading if necessary. model. Optionally download if release doesn’t exist.

Parameters:: check_release (logical) – Deprecated, not in use.
Returns:: A trained pytorch model
Return type:: model (object)

use_release(check_release=True)[source]#

Use the latest DeepForest model release from Hugging Face, downloading if necessary. Optionally download if release doesn’t exist.

Parameters:: check_release (logical) – Deprecated, not in use.
Returns:: A trained PyTorch model
Return type:: model (object)

val_dataloader()[source]#

Create a val data loader only if specified in config.

Returns:: a dataloader or a empty iterable.

validation_step(batch, batch_idx)[source]#: Evaluate a batch.

deepforest.model module#

class deepforest.model.CropModel(num_classes, batch_size=4, num_workers=0, lr=0.0001, model=None, label_dict=None)[source]#

Bases: LightningModule

A PyTorch Lightning module for classifying image crops from object detection models.

This class provides a flexible architecture for training classification models on cropped regions identified by object detection models. It supports using either a default ResNet-50 model or a custom provided model.

Parameters:

num_classes (int) – Number of classes for classification
batch_size (int, optional) – Batch size for training. Defaults to 4.
num_workers (int, optional) – Number of worker processes for data loading. Defaults to 0.
lr (float, optional) – Learning rate for optimization. Defaults to 0.0001.
model (nn.Module, optional) – Custom PyTorch model to use. If None, uses ResNet-50. Defaults to None.
label_dict (dict, optional) – Mapping of class labels to numeric indices. Defaults to None.

model#

The classification model (ResNet-50 or custom)

Type:: nn.Module

accuracy#

Per-class accuracy metric

Type:: torchmetrics.Accuracy

total_accuracy#

Overall accuracy metric

Type:: torchmetrics.Accuracy

precision_metric#

Precision metric

Type:: torchmetrics.Precision

metrics#

Collection of all metrics

Type:: torchmetrics.MetricCollection

batch_size#

Batch size for training

Type:: int

num_workers#

Number of data loading workers

Type:: int

lr#

Learning rate

Type:: float

label_dict#

Label to index mapping {“Bird”: 0, “Mammal”: 1}

Type:: dict

numeric_to_label_dict#

Index to label mapping {0: “Bird”, 1: “Mammal”}

Type:: dict

configure_optimizers()[source]#

Choose what optimizers and learning-rate schedulers to use in your optimization. Normally you’d need one. But in the case of GANs or similar you might have multiple. Optimization with multiple optimizers only works in the manual optimization mode.

Returns:

Any of these 6 options.

Single optimizer.
List or Tuple of optimizers.
Two lists - The first list has multiple optimizers, and the second has multiple LR schedulers (or multiple lr_scheduler_config).
Dictionary, with an "optimizer" key, and (optionally) a "lr_scheduler" key whose value is a single LR scheduler or lr_scheduler_config.
None - Fit will run without any optimizer.

The lr_scheduler_config is a dictionary which contains the scheduler and its associated configuration. The default configuration is shown below.

lr_scheduler_config = {
    # REQUIRED: The scheduler instance
    "scheduler": lr_scheduler,
    # The unit of the scheduler's step size, could also be 'step'.
    # 'epoch' updates the scheduler on epoch end whereas 'step'
    # updates it after a optimizer update.
    "interval": "epoch",
    # How many epochs/steps should pass between calls to
    # `scheduler.step()`. 1 corresponds to updating the learning
    # rate after every epoch/step.
    "frequency": 1,
    # Metric to to monitor for schedulers like `ReduceLROnPlateau`
    "monitor": "val_loss",
    # If set to `True`, will enforce that the value specified 'monitor'
    # is available when the scheduler is updated, thus stopping
    # training if not found. If set to `False`, it will only produce a warning
    "strict": True,
    # If using the `LearningRateMonitor` callback to monitor the
    # learning rate progress, this keyword can be used to specify
    # a custom logged name
    "name": None,
}

When there are schedulers in which the .step() method is conditioned on a value, such as the torch.optim.lr_scheduler.ReduceLROnPlateau scheduler, Lightning requires that the lr_scheduler_config contains the keyword "monitor" set to the metric name that the scheduler should be conditioned on.

# The ReduceLROnPlateau scheduler requires a monitor
def configure_optimizers(self):
    optimizer = Adam(...)
    return {
        "optimizer": optimizer,
        "lr_scheduler": {
            "scheduler": ReduceLROnPlateau(optimizer, ...),
            "monitor": "metric_to_track",
            "frequency": "indicates how often the metric is updated",
            # If "monitor" references validation metrics, then "frequency" should be set to a
            # multiple of "trainer.check_val_every_n_epoch".
        },
    }


# In the case of two optimizers, only one using the ReduceLROnPlateau scheduler
def configure_optimizers(self):
    optimizer1 = Adam(...)
    optimizer2 = SGD(...)
    scheduler1 = ReduceLROnPlateau(optimizer1, ...)
    scheduler2 = LambdaLR(optimizer2, ...)
    return (
        {
            "optimizer": optimizer1,
            "lr_scheduler": {
                "scheduler": scheduler1,
                "monitor": "metric_to_track",
            },
        },
        {"optimizer": optimizer2, "lr_scheduler": scheduler2},
    )

Metrics can be made available to monitor by simply logging it using self.log('metric_to_track', metric_val) in your LightningModule.

Note

Some things to know:

Lightning calls .backward() and .step() automatically in case of automatic optimization.
If a learning rate scheduler is specified in configure_optimizers() with key "interval" (default “epoch”) in the scheduler configuration, Lightning will call the scheduler’s .step() method automatically in case of automatic optimization.
If you use 16-bit precision (precision=16), Lightning will automatically handle the optimizer.
If you use torch.optim.LBFGS, Lightning handles the closure function automatically for you.
If you use multiple optimizers, you will have to switch to ‘manual optimization’ mode and step them yourself.
If you need to control how often the optimizer steps, override the optimizer_step() hook.

create_trainer(**kwargs)[source]#: Create a pytorch lightning trainer object.

dataset_confusion(loader)[source]#: Create a confusion matrix from a data loader.

forward(x)[source]#

Same as torch.nn.Module.forward().

Parameters:

*args – Whatever you decide to pass into the forward method.
**kwargs – Keyword arguments are also possible.

Returns:

Your model’s output

get_transform(augment)[source]#

Returns the data transformation pipeline for the model.

Parameters:: augment (bool) – Flag indicating whether to apply data augmentation.
Returns:: The composed data transformation pipeline.
Return type:: torchvision.transforms.Compose

load_from_disk(train_dir, val_dir)[source]#

normalize()[source]#

predict_dataloader(ds)[source]#: Prediction data loader.

predict_step(batch, batch_idx)[source]#

Step function called during predict(). By default, it calls forward(). Override to add any processing logic.

The predict_step() is used to scale inference on multi-devices.

To prevent an OOM error, it is possible to use BasePredictionWriter callback to write the predictions to disk or database after each batch or on epoch end.

The BasePredictionWriter should be used while using a spawn based accelerator. This happens for Trainer(strategy="ddp_spawn") or training on 8 TPU cores with Trainer(accelerator="tpu", devices=8) as predictions won’t be returned.

Parameters:

batch – The output of your data iterable, normally a DataLoader.
batch_idx – The index of this batch.
dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns:

Predicted output (optional).

Example

class MyModel(LightningModule):

    def predict_step(self, batch, batch_idx, dataloader_idx=0):
        return self(batch)

dm = ...
model = MyModel()
trainer = Trainer(accelerator="gpu", devices=2)
predictions = trainer.predict(model, dm)

train_dataloader()[source]#: Train data loader.

training_step(batch, batch_idx)[source]#

Here you compute and return the training loss and some additional metrics for e.g. the progress bar or logger.

Parameters:

batch – The output of your data iterable, normally a DataLoader.
batch_idx – The index of this batch.
dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns:

Tensor - The loss tensor
dict - A dictionary which can include any keys, but must include the key 'loss' in the case of automatic optimization.
None - In automatic optimization, this will skip to the next batch (but is not supported for multi-GPU, TPU, or DeepSpeed). For manual optimization, this has no special meaning, as returning the loss is not required.

In this step you’d normally do the forward pass and calculate the loss for a batch. You can also do fancier things like multiple forward passes or something model specific.

Example:

def training_step(self, batch, batch_idx):
    x, y, z = batch
    out = self.encoder(x)
    loss = self.loss(out, x)
    return loss

To use multiple optimizers, you can switch to ‘manual optimization’ and control their stepping:

def __init__(self):
    super().__init__()
    self.automatic_optimization = False


# Multiple optimizers (e.g.: GANs)
def training_step(self, batch, batch_idx):
    opt1, opt2 = self.optimizers()

    # do training_step with encoder
    ...
    opt1.step()
    # do training_step with decoder
    ...
    opt2.step()

Note

When accumulate_grad_batches > 1, the loss returned here will be automatically normalized by accumulate_grad_batches internally.

val_dataloader()[source]#: Validation data loader.

validation_step(batch, batch_idx)[source]#

Operates on a single batch of data from the validation set. In this step you’d might generate examples or calculate anything of interest like accuracy.

Parameters:

batch – The output of your data iterable, normally a DataLoader.
batch_idx – The index of this batch.
dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns:

Tensor - The loss tensor
dict - A dictionary. Can include any keys, but must include the key 'loss'.
None - Skip to the next batch.

# if you have one val dataloader:
def validation_step(self, batch, batch_idx): ...


# if you have multiple val dataloaders:
def validation_step(self, batch, batch_idx, dataloader_idx=0): ...

Examples:

# CASE 1: A single validation dataset
def validation_step(self, batch, batch_idx):
    x, y = batch

    # implement your own
    out = self(x)
    loss = self.loss(out, y)

    # log 6 example images
    # or generated text... or whatever
    sample_imgs = x[:6]
    grid = torchvision.utils.make_grid(sample_imgs)
    self.logger.experiment.add_image('example_images', grid, 0)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    val_acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs!
    self.log_dict({'val_loss': loss, 'val_acc': val_acc})

If you pass in multiple val dataloaders, validation_step() will have an additional argument. We recommend setting the default value of 0 so that you can quickly switch between single and multiple dataloaders.

# CASE 2: multiple validation dataloaders
def validation_step(self, batch, batch_idx, dataloader_idx=0):
    # dataloader_idx tells you which dataset this is.
    ...

Note

If you don’t need to validate you don’t need to implement this method.

Note

When the validation_step() is called, the model has been put in eval mode and PyTorch gradients have been disabled. At the end of validation, the model goes back to training mode and gradients are enabled.

write_crops(root_dir, images, boxes, labels, savedir)[source]#

Write crops to disk.

Parameters:

root_dir (str) – The root directory where the images are located.
images (list) – A list of image filenames.
boxes (list) – A list of bounding box coordinates in the format [xmin, ymin, xmax, ymax].
labels (list) – A list of labels corresponding to each bounding box.
savedir (str) – The directory where the cropped images will be saved.

Returns:

None

class deepforest.model.Model(config)[source]#

Bases: object

A architecture agnostic class that controls the basic train, eval and predict functions. A model should optionally allow a backbone for pretraining. To add new architectures, simply create a new module in models/ and write a create_model. Then add the result to the if else statement below.

Parameters:

num_classes (int) – number of classes in the model
nms_thresh (float) – non-max suppression threshold for intersection-over-union [0,1]
score_thresh (float) – minimum prediction score to keep during prediction [0,1]

Returns:

a pytorch nn module

Return type:

model

check_model()[source]#: Ensure that model follows deepforest guidelines, see ##### If fails, raise ValueError.

create_model()[source]#

This function converts a deepforest config file into a model.

An architecture should have a list of nested arguments in config that match this function

deepforest.model.simple_resnet_50(num_classes=2)[source]#

deepforest.predict module#

deepforest.predict.across_class_nms(predicted_boxes, iou_threshold=0.15)[source]#: Perform non-max suppression for a dataframe of results (see visualize.format_boxes) to remove boxes that overlap by iou_thresholdold of IoU.

deepforest.predict.mosiac(boxes, windows, sigma=0.5, thresh=0.001, iou_threshold=0.1)[source]#

deepforest.preprocess module#

The preprocessing module is used to reshape data into format suitable for training or prediction.

For example cutting large tiles into smaller images.

deepforest.preprocess.compute_windows(numpy_image, patch_size, patch_overlap)[source]#

Create a sliding window object from a raster tile.

Parameters:: numpy_image (array) – Raster object as numpy array to cut into crops
Returns:: a sliding windows object
Return type:: windows (list)

deepforest.preprocess.image_name_from_path(image_path)[source]#: Convert path to image name for use in indexing.

deepforest.preprocess.preprocess_image(image)[source]#: Preprocess a single RGB numpy array as a prediction from channels last, to channels first.

deepforest.preprocess.save_crop(base_dir, image_name, index, crop)[source]#

Save window crop as an image file to be read by PIL.

Parameters:

base_dir (str) – The base directory to save the image file.
image_name (str) – The name of the original image.
index (int) – The index of the window crop.
crop (numpy.ndarray) – The window crop as a NumPy array.

Returns:

The filename of the saved image.

Return type:

str

deepforest.preprocess.select_annotations(annotations, window)[source]#

Select annotations that overlap with selected image crop.

Parameters:

annotations – a geopandas dataframe of annotations with a geometry column
windows – A sliding window object (see compute_windows)

Returns:

a pandas dataframe of annotations

Return type:

selected_annotations

deepforest.preprocess.split_raster(annotations_file=None, path_to_raster=None, numpy_image=None, root_dir=None, base_dir=None, patch_size=400, patch_overlap=0.05, allow_empty=False, image_name=None, save_dir='.')[source]#

Divide a large tile into smaller arrays. Each crop will be saved to file.

Parameters:

numpy_image – a numpy object to be used as a raster, usually opened from rasterio.open.read(), in order (height, width, channels)
root_dir – (str): Root directory of annotations file, if not supplied, will be inferred from annotations_file
path_to_raster – (str): Path to a tile that can be read by rasterio on disk
annotations_file (str or pd.DataFrame) – A pandas dataframe or path to annotations csv file to transform to cropped images. In the format -> image_path, xmin, ymin, xmax, ymax, label. If None, allow_empty is ignored and the function will only return the cropped images.
save_dir (str) – Directory to save images
base_dir (str) – Directory to save images
patch_size (int) – Maximum dimensions of square window
patch_overlap (float) – Percent of overlap among windows 0->1
allow_empty – If True, include images with no annotations to be included in the dataset. If annotations_file is None, this is ignored.
image_name (str) – If numpy_image arg is used, what name to give the raster?

Note

When allow_empty is True, the function will return 0’s for coordinates, following torchvision style, the label will be ignored, so for continuity, the first label in the annotations_file will be used.

Returns:: If annotations_file is provided, a pandas dataframe with annotations file for training. A copy of this file is written to save_dir as a side effect. If not, a list of filenames of the cropped images.

deepforest.utilities module#

class deepforest.utilities.DownloadProgressBar(*_, **__)[source]#

Bases: tqdm

Download progress bar class.

update_to(b=1, bsize=1, tsize=None)[source]#

Update class attributes :param b: :param bsize: :param tsize:

Returns:

deepforest.utilities.annotations_to_shapefile(df, transform, crs)[source]#

Convert output from predict_image and predict_tile to a geopandas data.frame.

Parameters:

df – prediction data.frame with columns [‘xmin’,’ymin’,’xmax’,’ymax’,’label’,’score’]
transform – A rasterio affine transform object
crs – A rasterio crs object

Returns:

a geopandas dataframe where every entry is the bounding box for a detected tree.

Return type:

results

deepforest.utilities.boxes_to_shapefile(df, root_dir, projected=True, flip_y_axis=False)[source]#

Convert from image coordinates to geographic coordinates Note that this assumes df is just a single plot being passed to this function :param df: a pandas type dataframe with columns: name, xmin, ymin, xmax, ymax. Name is the relative path to the root_dir arg. :param root_dir: directory of images to lookup image_path column :param projected: If True, convert from image to geographic coordinates, if False, keep in image coordinate system :param flip_y_axis: If True, reflect predictions over y axis to align with raster data in QGIS, which uses a negative y origin compared to numpy. See https://gis.stackexchange.com/questions/306684/why-does-qgis-use-negative-y-spacing-in-the-default-raster-geotransform

Returns:: a geospatial dataframe with the boxes optionally transformed to the target crs
Return type:: df

deepforest.utilities.check_image(image)[source]#

Check an image is three channel, channel last format :param image: numpy array

Returns: None, throws error on assert

deepforest.utilities.collate_fn(batch)[source]#

deepforest.utilities.convert_point_to_bbox(gdf, buffer_size)[source]#

Convert an input point type annotation to a bounding box by buffering the point with a fixed size.

Parameters:

gdf (GeoDataFrame) – The input point type annotation.
buffer_size (float) – The size of the buffer to be applied to the point.

Returns:

The output bounding box type annotation.

Return type:

gdf (GeoDataFrame)

deepforest.utilities.crop_annotations_to_bounds(gdf, bounds)[source]#

Crop a geodataframe of annotations to a bounding box :param gdf: a geodataframe of annotations :param bounds: a tuple of (left, bottom, right, top) bounds

Returns:: a geodataframe of annotations cropped to the bounds
Return type:: gdf

deepforest.utilities.crop_raster(bounds, rgb_path=None, savedir=None, filename=None, driver='GTiff')[source]#

Crop a raster to a bounding box, save as projected or unprojected crop :param bounds: a tuple of (left, bottom, right, top) bounds :param rgb_path: path to the rgb image :param savedir: directory to save the crop :param filename: filename to save the crop “{}.tif”.format(filename)” :param driver: rasterio driver to use, default to GTiff, can be ‘GTiff’ for projected data or ‘PNG’ unprojected data

Returns:: path to the saved crop, if savedir specified img: a numpy array of the crop, if savedir not specified
Return type:: filename

deepforest.utilities.determine_geometry_type(df)[source]#

Determine the geometry type of a prediction or annotation :param df: a pandas dataframe

Returns:: a string of the geometry type
Return type:: geometry_type

deepforest.utilities.geo_to_image_coordinates(gdf, image_bounds, image_resolution)[source]#

Convert from projected coordinates to image coordinates :param gdf: a pandas type dataframe with columns: name, xmin, ymin, xmax, ymax. Name is the relative path to the root_dir arg. :param image_bounds: bounds of the image :param image_resolution: resolution of the image

Returns:: a geopandas dataframe with the transformed to image origin. CRS is removed
Return type:: gdf

deepforest.utilities.image_to_geo_coordinates(gdf, root_dir, flip_y_axis=False)[source]#

Convert from image coordinates to geographic coordinates.

Parameters:

gdf – A geodataframe.
root_dir – Directory of images to lookup image_path column.
flip_y_axis – If True, reflect predictions over y axis to align with raster data in QGIS, which uses a negative y origin compared to numpy.

Returns:

A geospatial dataframe with the boxes optionally transformed to the target crs.

Return type:

transformed_gdf

deepforest.utilities.project_boxes(df, root_dir, transform=True)[source]#: Convert from image coordinates to geographic coordinates Note that this assumes df is just a single plot being passed to this function df: a pandas type dataframe with columns: name, xmin, ymin, xmax, ymax. Name is the relative path to the root_dir arg. root_dir: directory of images to lookup image_path column transform: If true, convert from image to geographic coordinates

deepforest.utilities.read_coco(json_file)[source]#

Read a COCO format JSON file and return a pandas dataframe.

Parameters:: json_file – Path to the COCO segmentation JSON file
Returns:: A pandas dataframe with image_path and geometry columns
Return type:: df

deepforest.utilities.read_config(config_path)[source]#: Read config yaml file.

deepforest.utilities.read_file(input, root_dir=None)[source]#

Read a file and return a geopandas dataframe.

This is the main entry point for reading annotations into deepforest. :param input: a path to a file or a pandas dataframe :param root_dir: location of the image files, if not in the same directory as the annotations file :type root_dir: str

Returns:: a geopandas dataframe with the properly formatted geometry column df.root_dir: the root directory of the image files
Return type:: df

deepforest.utilities.read_pascal_voc(xml_path)[source]#

Load annotations from xml format (e.g. RectLabel editor) and convert them into retinanet annotations format.

Parameters:

xml_path (str) – Path to the annotations xml, formatted by RectLabel

Returns:

in the: format -> path-to-image.png,x1,y1,x2,y2,class_name

Return type:

Annotations (pandas dataframe)

deepforest.utilities.round_with_floats(x)[source]#: Check if string x is float or int, return int, rounded if needed.

deepforest.utilities.shapefile_to_annotations(shapefile, rgb=None, root_dir=None, buffer_size=None, convert_point=False, geometry_type=None, save_dir=None)[source]#

Convert a shapefile of annotations into annotations csv file for DeepForest training and evaluation.

Parameters:

shapefile – Path to a shapefile on disk. If a label column is present, it will be used, else all labels are assumed to be “Tree”
rgb – Path to the RGB image on disk
root_dir – Optional directory to prepend to the image_path column

Returns:

a pandas dataframe

Return type:

results

deepforest.utilities.xml_to_annotations(xml_path)[source]#

deepforest.visualize module#

deepforest.visualize.convert_to_sv_format(df, width=None, height=None)[source]#

Convert DeepForest prediction results to a supervision Detections object.

Parameters:

df (pd.DataFrame) – The results from predict_image or predict_tile. Expected columns includes: [‘geometry’, ‘label’, ‘score’, ‘image_path’] for bounding boxes
width (int) – The width of the image in pixels. Only required if the geometry type is ‘polygon’.
height (int) – The height of the image in pixels. Only required if the geometry type is ‘polygon’.

Returns:

Depending on the geometry type, the function returns either a Detections or a KeyPoints object from the supervision library.

deepforest.visualize.format_boxes(prediction, scores=True)[source]#

Format a retinanet prediction into a pandas dataframe for a single image.

Parameters:

prediction – a dictionary with keys ‘boxes’ and ‘labels’ coming from a retinanet
scores – Whether boxes come with scores, during prediction, or without scores, as in during training.

Returns:

a pandas dataframe

Return type:

df

deepforest.visualize.format_geometry(predictions, scores=True)[source]#

Format a retinanet prediction into a pandas dataframe for a batch of images :param predictions: a list of dictionaries with keys ‘boxes’ and ‘labels’ coming from a retinanet :param scores: Whether boxes come with scores, during prediction, or without scores, as in during training.

Returns:: a pandas dataframe
Return type:: df

deepforest.visualize.label_to_color(label)[source]#

deepforest.visualize.plot_annotations(annotations, savedir=None, height=None, width=None, color=[245, 135, 66], thickness=2, basename=None, root_dir=None, radius=3, image=None)[source]#

Plot the prediction results.

Parameters:

annotations – a pandas dataframe with prediction results
savedir – optional path to save the figure. If None (default), the figure will be interactively plotted.
height – height of the image in pixels. Required if the geometry type is ‘polygon’.
width – width of the image in pixels. Required if the geometry type is ‘polygon’.
results_color (list or sv.ColorPalette) – color of the results annotations as a tuple of RGB color (if a single color), e.g. orange annotations is [245, 135, 66], or an supervision.ColorPalette if multiple labels and specifying colors for each label
thickness – thickness of the rectangle border line in px
basename – optional basename for the saved figure. If None (default), the basename will be extracted from the image path.
root_dir – optional path to the root directory of the images. If None (default), the root directory will be extracted from the annotations dataframe.root_dir attribute.
radius – radius of the points in px

Returns:

None

deepforest.visualize.plot_points(image, points, color=None, radius=5, thickness=1)[source]#

Plot points on an image :param image: a numpy array in BGR color order! Channel order is channels first :param points: a numpy array of shape (N, 2) representing the coordinates of the points :param color: color of the points as a tuple of BGR color, e.g. orange points is (0, 165, 255) :param radius: radius of the points in px :param thickness: thickness of the point border line in px

Returns:: a numpy array with drawn points
Return type:: image

deepforest.visualize.plot_prediction_and_targets(image, predictions, targets, image_name, savedir)[source]#

Plot an image, its predictions, and its ground truth targets for debugging.

Parameters:

image – torch tensor, RGB color order
targets – torch tensor

Returns:

path on disk with saved figure

Return type:

figure_path

deepforest.visualize.plot_prediction_dataframe(df, root_dir, savedir, color=None, thickness=1, ground_truth=None)[source]#

For each row in dataframe, call plot predictions and save plot files to disk. For multi-class labels, boxes will be colored by labels. Ground truth boxes will all be same color, regardless of class.

Parameters:

df – a pandas dataframe with image_path, xmin, xmax, ymin, ymax and label columns. The image_path column should be the relative path from root_dir, not the full path.
root_dir – relative dir to look for image names from df.image_path
ground_truth – an optional pandas dataframe in same format as df holding ground_truth boxes
savedir – save the plot to an optional directory path.

Returns:

list of filenames written

Return type:

written_figures

deepforest.visualize.plot_predictions(image, df, color=None, thickness=1)[source]#

Plot a set of boxes on an image By default this function does not show, but only plots an axis Label column must be numeric! Image must be BGR color order!

Parameters:

image – a numpy array in BGR color order! Channel order is channels first
df – a pandas dataframe with xmin, xmax, ymin, ymax and label column
color – color of the bounding box as a tuple of BGR color, e.g. orange annotations is (0, 165, 255)
thickness – thickness of the rectangle border line in px

Returns:

a numpy array with drawn annotations

Return type:

image

deepforest.visualize.plot_results(results, ground_truth=None, savedir=None, height=None, width=None, results_color=[245, 135, 66], ground_truth_color=[0, 165, 255], thickness=2, basename=None, radius=3, image=None, axes=False)[source]#

Plot the prediction results.

Parameters:

results – a pandas dataframe with prediction results
ground_truth – an optional pandas dataframe with ground truth annotations
savedir – optional path to save the figure. If None (default), the figure will be interactively plotted.
height – height of the image in pixels. Required if the geometry type is ‘polygon’.
width – width of the image in pixels. Required if the geometry type is ‘polygon’.
results_color (list or sv.ColorPalette) – color of the results annotations as a tuple of RGB color (if a single color), e.g. orange annotations is [245, 135, 66], or an supervision.ColorPalette if multiple labels and specifying colors for each label
ground_truth_color (list) – color of the ground truth annotations as a tuple of RGB color, e.g. blue annotations is [0, 165, 255]
thickness – thickness of the rectangle border line in px
basename – optional basename for the saved figure. If None (default), the basename will be extracted from the image path.
radius – radius of the points in px
image – an optional numpy array of an image to annotate. If None (default), the image will be loaded from the results dataframe.
axes – returns matplotlib axes object if True

Returns:

Matplotlib axes object if axes=True, otherwise None

deepforest.visualize.view_dataset(ds, savedir=None, color=None, thickness=1)[source]#

Plot annotations on images for debugging purposes.

Parameters:

ds – a deepforest pytorch dataset, see deepforest.dataset or deepforest.load_dataset() to start from a csv file
savedir – optional path to save figures. If none (default) images will be interactively plotted
color – color of the bounding box as a tuple of BGR color, e.g. orange annotations is (0, 165, 255)
thickness – thickness of the rectangle border line in px

Module contents#

Top-level package for DeepForest.

deepforest.get_data(path)[source]#: Helper function to get package sample data.

deepforest package#

Subpackages#

Submodules#

deepforest.IoU module#

deepforest.callbacks module#

deepforest.dataset module#

deepforest.evaluate module#

deepforest.main module#

deepforest.model module#

deepforest.predict module#

deepforest.preprocess module#

deepforest.utilities module#

deepforest.visualize module#

Module contents#

This Page