The Detection Problem

Author

Deon Roos

Published

June 6, 2026

How many animals are there, and are they surviving?

Two of the most fundamental questions in ecology and conservation are: how many individuals of a species are there, and how well are they surviving? How many Atlantic salmon are in the River Dee? What proportion of juveniles survive to adulthood? Is that survival rate declining? Is a conservation intervention actually working?

Lily, in your case these questions become; How many willow warblers are present at your study sites? How well are they surviving between breeding seasons? Is survival declining? Are any interventions or habitat changes making a difference?

These are not easy questions to answer, and the reason comes down to the same problem that runs through this entire tutorial: you will never see every bird that is there.

These questions sound simple but they’re not. And the reason for that comes down to a problem that seems simple but is hard to deal with.

The naive approach

Imagine I want to estimate how many otters live along a stretch of river. I go out and count them. I spot 23 otters.

So there are 23 otters?

Maybe. But I wouldn’t bet a £100 that I saw every single otter that was there. Some might’ve been in burrows. Some were maybe around the bend in the river. Some maybe saw me coming and hid. The 23 I counted is not the true population size. It is the true population size multiplied by my probability of detecting any given otter.

\[\text{Count} = N \times p\]

where \(N\) is the true number of otters and \(p\) is my detection probability. If \(p = 1\) (i.e. I have 100% chance of seeing each otter), my count equals the true population. If \(p < 1\), and it is almost always less than 1, my count is an underestimate. How badly it underestimates depends entirely on how low \(p\) is.

This is imperfect detection. This is the default condition of almost every single wildlife survey ever conducted. If you have ever counted animals in the field and assumed that count represented the true number present, you were almost certainly wrong.

Mark individuals

Here is where the “mark” part of mark-recapture comes in. The idea is pretty simple:

If you catch some animals, mark them, release them, and then go back and catch animals again, the proportion of marked animals in your second sample tells you something about how many unmarked ones you missed.

Say I catch 20 otters, tag each one with a unique ID, and release them back into the river. A week later I come back and catch another 20 otters. Of these 20, I find that 5 of them have the tags I put on a week ago. So I recaptured 5 out of the 20 I originally marked. That recapture rate of \(\frac{5}{20} = 25\%\) is an estimate of my detection probability. And if I only detected 25% of last weeks marked animals in my second sample, I probably only detected about 25% of unmarked animals too. Which means my second sample of 20 likely represents about 25% of the true population, meaning there are roughly \(\frac{20}{0.25} = 80\) otters in total.

That’s the Lincoln-Petersen estimator, and we’ll work through it properly on the next page. For now, the main idea is that recaptures allow us to figure out detection probability, which is what allows us to estimate true population size rather than just report a count and hope for the best.

All good and well but there’s actually a second problem on top of imperfect detection.

The second problem: individuals can die

Individual animals are not immortal and static. They move, disperse, and/or they die.

This matters a lot because it creates a new version of the detection problem.

Imagine I tag 50 fish in June. In August I go back and sample again. I catch 30 fish, of which 12 have tags. Fine - same as before with the otters. But now I want you to think about the 38 tagged fish I did not recapture. Those “non-recaptures” have two possible explanations:

  1. They are still alive, I just did not catch them. My detection probability is (almost always) less than 100%, so that’s entirely plausible. (This is what we assumed above with the otters)

  2. They are dead.

A non-recapture has two possible meanings: missed, or dead. The crap part of that: both explanations predict exactly the same observation in your data - a zero. In both cases you don’t see the fish but the biological distinction between missed and dead is obviously important…

If you cannot separate these two explanations, you cannot estimate either detection probability or survival probability reliably. They are completely tangled up in each other.

The following simulation hopefully makes this tangible (don’t worry about the code). We follow 50 tagged individuals across four sampling occasions, assuming a survival probability of 0.75 between occasions and detection probabilities that range from 0.30 to 0.80 across the four surveys.

Code 
library(ggplot2)
library(dplyr)

set.seed(1988)

n_tagged <- 50
occasions <- 4

phi_true <- 0.75
p_true <- c(0.3, 0.5, 0.6, 0.8)

survival_mat <- matrix(NA, nrow = n_tagged, ncol = occasions)
survival_mat[, 1] <- 1

for (t in 2:occasions) {
  survival_mat[, t] <- rbinom(n_tagged, 1, survival_mat[, t - 1] * phi_true)
}

detection_mat <- matrix(NA, nrow = n_tagged, ncol = occasions)
for (t in 1:occasions) {
  detection_mat[, t] <- rbinom(n_tagged, 1, survival_mat[, t] * p_true[t])
}

observed_counts <- colSums(detection_mat)
true_alive <- colSums(survival_mat)

summary_df <- data.frame(
  occasion = 1:occasions,
  true_alive = true_alive,
  observed = observed_counts
)

ggplot(summary_df) +
  geom_col(aes(x = occasion, y = true_alive),
           fill = "#00A68A", alpha = 0.4, width = 0.6) +
  geom_col(aes(x = occasion, y = observed),
           fill = "#FF5733", alpha = 0.8, width = 0.3) +
  labs(
    x = "Sampling occasion",
    y = "Number of tagged individuals",
    caption = "Green = truly alive  |  Orange = actually detected"
  ) +
  scale_x_continuous(breaks = 1:occasions) +
  theme_dark_site()

The green bars show how many animals are truly alive at each occasion. The orange bars show how many we actually detect. The gap between them is entirely due to imperfect detection, not death. But if I only showed you the orange bars and asked you to calculate a survival rate, you’d probably conclude things look ok, but based on the green bars, this population is crashing fast.

This is where using the right stats matters a lot. Imagine this is a critically endangered population, and you’re the conservation officer. If you use basic stats you’re going to tell everyone “don’t worry, the species are doing fine. They’re stable”. In doing so, you’ve likely just condemned a species to extinction. We don’t use complicated stats for the sake of it. We use it because there’s a god damned moral imperative to make sure you make the best decision with the data you have.

Try it yourself: dead or hiding?

Before we formalise this problem, try to get a proper sense of it. Below are four tagged animals. Their detection histories are shown one occasion at a time. Each time your survey draws a blank, you must decide: was the animal dead, or was it alive and just not detected?

The problem

To describe these two competing explanations precisely, we need to name two quantities. The first is \(p\), the probability of detecting an individual that is alive and present during a survey. The second is \(\phi\) (the Greek letter “phi”), the probability an individual survives from one survey to the next. These two quantities sit at the heart of everything that follows in this tutorial.

When you do not recapture a tagged animal, you are facing two competing hypotheses:

  • Hypothesis A: The animal is alive but you missed it. This happens with probability \(\phi \times (1 - p)\): the animal survived (\(\phi\)) the interval between surveys but you failed to detect it (\(1 - p\), the probability of not seeing it).

  • Hypothesis B: The animal is dead. This happens with probability \(1 - \phi\): the animal did not survive the time between surveys.

Both predict the same observation; 0. When all you have is a capture occasion you cannot tell them apart. Use the sliders below to explore how the balance between these two explanations shifts depending on the true values of \(\phi\) and \(p\).

This is why a single survey with recaptures is not enough. You need a framework that estimates \(\phi\) and \(p\) simultaneously, using the full pattern of detections and non-detections across multiple occasions to untangle them.

What we need to make progress

To estimate survival and detection simultaneously, rather than having them hopelessly tangled together, we need three things.

First, individually marked animals. Not just a count of how many we saw, but a record of which specific individuals we saw on each occasion. This is what allows us to track fates through time rather than just tallying up numbers.

Second, multiple sampling occasions. A single occasion gives us a count. Multiple occasions give us a pattern of detections and non-detections across time, and that pattern contains the information we need to estimate both \(\phi\) and \(p\) separately.

Third, a model that represents both processes explicitly. One that says, formally, that observing an animal requires both that it survived to the next occasion and that we actually detected it, and uses that structure to estimate each parameter on its own terms. If you have worked through the GLM refresher, you already have the statistical foundation for this. The models we will build are extensions of that same framework — they simply need to account for these two processes, survival and detection, at the same time.

The next page introduces the simplest version of this thinking: the Lincoln-Petersen estimator. It does not quite tick all three boxes but it builds the intuition clearly and cleanly before we move on to models that do.