Diagnostics

Research Summary

Mark and his collaborators have been involved in the creation or development of several diagnostics to identify data conflict and model misspecification including the likelihood component profile, Age-Structured Production Model (ASPM) diagnostic, and Catch-Curve diagnostic. In 2021 he organized and chaired the joint IATTC-CAPAM workshop on diagnostics. Maunder and Starr (2001) used likelihood component profiles over the initial exploitation rate to determine the information content of different composition data sets. Wang et al. (2014) developed a metric to measure data conflict, but it did not perform well. Other studies further investigated the properties of the R0 profile and applied it to several species (e.g. Lee et al., 2014; Minte-Vera et al., 2021). Maunder and Piner (2015) proposed using the age-structured production model (ASPM) as a diagnostic and several other studies investigated its properties and applied it to several species (e.g. Minte-Vera et al., 2021). They also proposed the catch curve diagnostic (Carvalho et al., 2017) and applied it to a variety of stocks (e.g. Minte-Vera et al., 2021). Other diagnostics have also been developed and investigated including those based on simulation (e.g. Piner et al. 2011) and the empirical selectivity (Maunder et al. 2020). Much of this work has been summarized in Carvalho et al. (2017; 2021). Maunder and Piner (2017) developed the “Law of conflicting data” and an algorithm based on diagnostics to identify and correct model misspecification. There is a lack of diagnostics and Mark continues his research into the development of diagnostics and how to use them to identify and correct model misspecification to ensure that the best models are used for management advice.           

Relevant Papers

Minte-Vera, C.V., Maunder, M.N., Aires-da-Silva, A.M. 2021. Auxiliary diagnostic analyses used to detect model misspecification and highlight potential solutions in stock assessments: application to yellowfin tuna in the eastern Pacific Ocean. ICES Journal of Marine Science 78 (10), 3521-3537. https://academic.oup.com/icesjms/article/78/10/3521/6430633?login=true

Carvalho, F., Winker, H., Courtney, D., Kapur, M., Kell, L., Cardinale, M., ... Maunder, M.N., … 2021. A cookbook for using model diagnostics in integrated stock assessments. Fish. Res. 240, 105959. https://www.sciencedirect.com/science/article/pii/S0165783621000874

Maunder, M.N., Xu, H., Lennert-Cody, C.E., Valero, J.L., Aires-da-Silva, A., Minte-Vera, C. 2020. Implementing reference point-based fishery harvest control rules within a probabilistic framework that considers multiple hypoetheses. IATTC Document SAC-11 INF-F REV. https://www.iattc.org/getattachment/46edbd8e-22f9-4bb3-8d26-d4cfd24a472c/SAC-11-INF-F_Implementing-risk-analysis.pdf

Carvalho, F., Punt, A. E., Chang, Y. J., Maunder, M. N., Piner, K. R. 2017. Can diagnostic tests help identify model misspecification in integrated stock assessments? Fisheries Research. 192: 28-40. https://www.sciencedirect.com/science/article/pii/S0165783616303113

Maunder, M. N. and Piner, K. R. 2017. Dealing with data conflicts in statistical inference of population assessment models that integrate information from multiple diverse data sets. Fisheries Research. 192: 16-27. https://www.sciencedirect.com/science/article/abs/pii/S0165783616301394

Maunder, M.N., Piner, K.R. 2015. Contemporary fisheries stock assessment: many issues still remain. ICES Journal of Marine Science. 72 (1): 7-18. https://academic.oup.com/icesjms/article/72/1/7/819352

Lee, H. H., Piner, K. R., Methot, R. D., Maunder, M. N. 2014. Use of likelihood profiling over a global scaling parameter to structure the population dynamics model: An example using blue marlin in the Pacific Ocean. Fisheries Research, 158: 138-146. https://www.sciencedirect.com/science/article/abs/pii/S0165783613003111

Wang, S. P., Maunder, M. N., Piner, K. R., Aires-da-Silva, A. Lee, H. H. 2014. Evaluation of virgin recruitment profiling as a diagnostic for selectivity curve structure in integrated stock assessment models. Fisheries Research, 158: 158-164. https://www.sciencedirect.com/science/article/abs/pii/S0165783613003032

Piner, K.R., Lee, H-H., and Maunder, M.N. 2011. A Simulation-Based Method to Determine Model Misspecification: Examples Using Natural Mortality and Population Dynamics Models. Marine and Coastal Fisheries, 3(1): 336-343. https://www.tandfonline.com/doi/full/10.1080/19425120.2011.611005

Maunder, M.N. and Starr, P.J. 2001. Bayesian Assessment of the SNA1 snapper (Pagrus auratus) stock on the northeast coast of New Zealand. New Zealand Journal of Marine and Freshwater Research, 35: 87-110. https://www.tandfonline.com/doi/abs/10.1080/00288330.2001.9516980

Reports

Harley, S.J. and Maunder, M.N. (2003) Recommended diagnostics for large statistical stock assessment models. SCTB16 WG-3.

Mark's presentations on Diagnostics from the CAPAM workshop on Model Weighting