# Simulation and Estimation Diagnostics

Model diagnostics are used to check if the model fits the data well. Diagnostics for continuous data are different than for discrete data.

## The `inspect`

Function

The main function to call when looking at diagnostics from a fit is `inspect`

with the signature:

```
inspect(fpm::AbstractFittedPumasModel;
wres_approx::LikelihoodApproximation,
nsim::Int,
rng)
```

Output a summary of the model predictions, residuals, Empirical Bayes estimates, and NPDEs (when requested).

Called on a fit output and allows the keyword argument wres_approx for approximation method to be used in residual calculation. The default value is the approximation method used for the marginal likelihood calculation in the fit that produced `fpm`

. The keyword `nsim`

controls the number of times each subject is simulated for the `npde`

computations. `rng`

is a random number generator that can be used to seed any random elements such as those in `npde`

computations.

`inspect`

returns a `FittedPumasModelInspection`

object with the

- model predictions
- residuals
- Empirical Bayes estimates

Example syntax

`my_inspect = DataFrame(inspect(my_fit))`

The result of `inspect(my_fit)`

is a `FittedPumasModelInspection`

object, which can then be converted into tabular form using the `DataFrame`

call. This is useful if one is interested to export the results into a spreadsheet, or use the resulting `DataFrame`

for plotting.

### Weighted Residuals

In Pumas you can calculate individual and population weighted residuals using `wresiduals`

as well as expected individual weighted residuals with the signature

```
wresiduals(fpm::AbstractFittedPumasModel,
approx::LikelihoodApproximation;
nsim=nothing)
```

Takes a fit result, an approximation method for the marginal likelihood calculation which defaults to the method used in the fit and the number of simulations with the keyword argument `nsim`

. If `nsim`

is specified only the Expected Simulation based Individual Weighted Residuals (EIWRES) is included in the output as individual residual and population residual is not computed. Using the FO approximation method corresponds to the WRES and while FOCE corresponds to CWRES. The output is a `SubjectResidual`

object that stores the population (wres) and individual (iwres) residuals along with the subject and approximation method (approx).

### Predictions

To calculate predictions from a fit use the `predict`

function with the signature:

```
predict(
fpm::AbstractFittedPumasModel,
[population::Union{Subject,Population};
[obstimes::AbstractVector]
)
```

By default, the predictions are computed for the estimation data but the predictions can also be computed for user supplied data by passing either a single subject or a vector of subjects (Population) as the population argument. If the optional `obstimes`

argument is passed then the time points in `obstimes`

are used for the predictions. Otherwise, the time points of the observations for each subject in the population are used for the predictions.

Any optional keyword arguments used when fitting fpm are reused when computing the predictions.

### Model Metrics

`aic(model, population, param, approximation)`

: Akaike information criterion`bic(model, population, param, approximation)`

: Bayesian information criterion

### Shrinkage

`ηshrinkage(model, population, param, approximation)`

`ϵshrinkage(model, population, param, approximation[, rfx])`

If the observations for an individual is informative, η-shrinkage will be low. If, on the other hand, the observations are non-informative, the individual estimates (EBEs) will shrink towards the population mean. Hence, the variance of the EBEs will also shrink and η-shrinkage will be high.

When data gets sparse the IWRES distribution also shrink towards it’s mean (zero). This is often called overfitting or ϵ-shrinkage.