background
Research and Development

DEVICE ACCREDITATION

The demand by decision-makers for strategic intelligence on new and emerging technologies has arguably never been higher given the rapid pace of technological change, growing needs in the spheres of health and environment and the need to anticipate and mitigate potential risks and negative consequences of certain technologies.…

— OECD Science, Technology and Industry Policy Papers, No. 146.
Overview

The Challenge

The Australian meat industry uses intramuscular fat (IMF%), which is a key determinant of eating quality in lamb, as an input into the new Meat Standards Australia (MSA) cuts-based eating quality prediction model.
The challenge for industry is that IMF% measurements are obtained using a laboratory-based Soxhlet fat extraction method that cannot be used in abattoirs because:

  • The test is destructive - hence the product is destroyed to obtain the IMF%,
  • The test has a slow processing speed, and cannot keep pace with lamb-abattoir,
  • New, non-destructive technologies are required to measure this trait.

Technologies are being developed to enable the implementation of the Meat Standards Australia prediction models, by measuring IMF%, using sensors capable of predicting IMF% with the required speed for industrial processing. These new technologies need to be accredited for use in the MSA models based on industry acceptable accuracy.

Project
deviceaccreditation

The Approach

The Australian Meat Industry Language and Standards Committee are tasked with specifying minimum accuracy standards for measuring IMF% in lamb. The standards they have mandated to test accuracy is that 67% of these estimates must fall within ±1 IMF% of their laboratory value, while 95% of predictions must fall within ±2 IMF%.While these rules closely align with a Standard Normal distribution, we are using a sample, and as such a device that is close to the required 1 IMF% of accuracy would likely require a large test sample to pass. Simulations highlighted that a device with a standard deviation of 0.95 IMF%, which is within acceptable operating accuracy, was only likely to pass, with a total of 800 samples, around 40% of the time. PAG used a regression modelling approach to provide a transparent, robust, and reproducible framework to assess a pass/fail of a device. For full details see: Gardner G.E. & Alston-Knox C.L. (2025). Accreditation of new technologies for predicting intramuscular fat percentage: Combining Bayesian models and industry rules for transparent decisions. PLoS ONE 20(3): e0314714. https://doi.org/10.1371/journal.pone.0314714

The Solution

PAG used a regression model is as follows:
IMF%Lab − IMF%Device = β0 + βQ2,Q3,Q4 + ϵ
If we had near perfect correspondence between the device and the laboratory estimate of IMF%, we would expect {β0} to be approximately zero (0), and if the disparity between measures is uniform across quarters we expect {βQ2 , βQ3 , βQ4} to also be approximately 0. For the variability between the device and the laboratory met the required accuracy we would expect ϵ~ N(0,≤ 12). In itself, this model would not reduce the required sample size. We implemented this model using a Bayesian framework, as using prior knowledge allowed the team to significantly reduce the sample size required to fairly test a device for accreditation. The prior distributions were based on industry discussions and approval - acknowledging the reality that device manufacturers would only seek accreditation if they were in the ballpark of accuracy.

deviceaccreditation
deviceaccreditation

The Impact

The proposed model has been implemented for all devices seeking accreditation using an R Shiny App, which allows assessors and manufacturers to work together and understand the characteristics of the device that may make it unsuitable for accreditation. Since the Bayesian model has been deployed, several devices have obtained accreditation, with the reporting functionality of the App providing understandable metrics for assessment personnel to rely upon. One of the devices to achieve accreditation is a handheld microwave system, and it has achieved its first commercial installation at WA’s Dardanup Butchering Company.“From a producer perspective, the key highlight of this new device is its potential to aid in the delivery of fairer and more accurate grading of carcase eating quality. Now that the technology is here, there’s an opportunity for the sheepmeat supply chain to take advantage of the information it provides to create premium lamb brands with MSA’s sheep cuts model” Professor Graham Gardner, Murdoch University.