Dairy Science & Technology · Foundations · also Computer Applications, Information Technology, Dairy Informatics
Spreadsheets, databases, programming basics and the digital systems a dairy plant now runs on: milk collection software, ERP, LIMS and traceability.
In the exams: ICAR AIEEA-PG group 17 Unit II (computer applications in milk plants)
A modern dairy runs on data as much as on milk. Every litre is weighed and tested where it is collected, and the farmer is paid on those results; every silo, pasteuriser and filler is monitored by sensors and controlled by programmable controllers; every laboratory result is recorded against a sample, a specification and a lot; and every pack carries a code that links it back to the milk it came from. A dairy technologist who cannot build a reliable spreadsheet, query a database, read a dashboard or recognise a cybersecurity risk is at a disadvantage in production, quality, procurement and management alike.
This manual covers the computing and data skills a dairy degree now requires. It starts with spreadsheets for plant and laboratory calculations, databases and data management, and introductory programming and scripting. It then describes the digital systems specific to dairying: automatic milk collection units and farmer payment systems; enterprise resource planning; laboratory information management systems; and traceability and recall software. It ends with data analysis and visualisation, sensors and the Internet of Things, and the basics of cybersecurity for plant control systems. Commercial products are described by type and function, never by brand; the principles apply whatever software a company uses.
Postgraduate entrance examinations in dairy technology include computer applications in milk plants. Some printed syllabi still name software of the 1980s; the questions in this subject are written to the concepts those syllabi intend, applied to current tools: data types, formulas and references, database structure and queries, algorithms, milk payment calculations, the functions of ERP, LIMS and traceability systems, sensors, and the protection of control systems.
A spreadsheet is a grid of cells, each identified by its column letter and row number, such as B4. A cell can hold a number, text, a date or a formula. A formula begins with an equals sign and calculates a value from other cells, constants and functions: =B4*C4 multiplies two cells, and =SUM(B4:B20) adds a range. When a formula is copied to another cell, its cell references change relative to the new position; this relative referencing lets one formula be filled down a column for every farmer or every batch. An absolute reference, written with dollar signs such as $B$1, does not change when copied, and is used for constants such as a price per kilogram of fat or a conversion factor kept in one cell. Mixed references, such as $B4 or B$4, fix only the column or only the row.
Built-in functions cover arithmetic, statistics, logic, text, dates and lookups. AVERAGE, STDEV and COUNT summarise data; IF returns one value or another depending on a condition, as in flagging a delivery whose temperature exceeds a limit; lookup functions find a value in a table, as in reading the price for a given fat and solids-not-fat content from a rate chart; ROUND controls rounding. Named ranges, such as calling a cell Fat_Price, make formulas easier to read and check.
Spreadsheets also draw charts, sort and filter data, and summarise large tables with pivot tables, which group records, for example deliveries by route and week, and calculate totals and averages. Built-in tools fit regression lines and run basic statistical tests, which is often enough for calibration lines, shelf-life plots and process trials.
Many routine dairy calculations are well suited to spreadsheets. In standardisation, the masses of whole milk and skim milk needed to reach a target fat content are calculated from a mass balance and a fat balance; a sheet with input cells for the fat contents and batch size and formula cells for the answers gives results instantly and avoids arithmetic errors. Ice cream mix calculations solve for the masses of cream, milk, skim milk powder, sugar and stabiliser that give the target composition. Milk payment sheets multiply each farmer's delivered quantity by the price calculated from the composition. Laboratory sheets convert titration volumes to acidity, calculate fat by Gerber or by instrument, apply dilution factors to plate counts, and compute means, standard deviations and control-chart limits.
A worked example shows the structure. Cells B1, B2 and B3 hold the fat percentages of whole milk (4.5), skim milk (0.05) and the target (3.0), and B4 holds the batch mass (1,000 kg). The mass of whole milk is calculated in B5 as =B4*(B3-B2)/(B1-B2), giving 662.9 kg, and the skim milk in B6 as =B4-B5, giving 337.1 kg. A check cell, =(B5*B1+B6*B2)/B4, should return exactly the target, 3.0, and turns red through conditional formatting if it does not.
Such sheets save time only if they are correct. Every input should have a stated unit, every formula should be traceable, and every sheet should be tested with a case whose answer is known before it is used.
Spreadsheets are easy to build and easy to get wrong. Audits of operational spreadsheets in industry have repeatedly found that a large share of them contain errors, many of them material: a formula that refers to the wrong row, a range that misses the last record, a constant typed into a formula and never updated, a unit mismatch between litres and kilograms, or an accidental overwrite of a formula with a number. Errors in payment, formulation or release calculations can cost money, cause product to be out of specification, or wrongly release product.
Good practice reduces this risk. Separate inputs, calculations and outputs, ideally on different sheets, and colour-code input cells. Keep every constant in a labelled cell, never inside formulas. Use data validation to restrict inputs to sensible ranges, such as fat content between 0 and 10%. Protect formula cells against accidental change. Add check totals and cross-checks, such as a mass balance that must close. Document the purpose, author, version and date of each sheet, and keep previous versions. Test each new or changed sheet against hand calculations or known cases, and have a second person review it.
Where spreadsheets are used for records required by regulations or customers, such as release decisions or payment calculations, data integrity rules apply: records must be attributable, legible, contemporaneous, original and accurate, and complete, consistent, enduring and available, a set of principles often summarised as ALCOA+. A spreadsheet without access control and an audit trail cannot easily meet them, which is why many companies move such calculations into validated software.
A database is an organised collection of data managed by software, a database management system, that stores, retrieves and protects it. Most business systems use relational databases, in which data are held in tables. Each table describes one kind of thing, such as farmers, deliveries, products or laboratory samples; each row is a record, one farmer or one delivery; and each column is a field, an attribute such as the farmer's name or the delivery's fat content. Every table has a primary key, a field or combination of fields whose value uniquely identifies each row, such as a farmer code. A foreign key is a field in one table that refers to the primary key of another, linking them: each delivery record carries the farmer code of the farmer who made it. This builds a one-to-many relationship, since one farmer makes many deliveries.
Normalisation organises tables to avoid redundancy and inconsistency. In first normal form each field holds a single value and there are no repeating groups; in second normal form every non-key field depends on the whole primary key; and in third normal form non-key fields depend only on the key and not on each other. In practice it means storing each fact once: a farmer's bank details are kept in the farmer table, not repeated in every delivery, so that a change is made in one place.
Data are retrieved with Structured Query Language (SQL). A query such as SELECT farmer_code, SUM(kg_fat) FROM deliveries WHERE delivery_date BETWEEN a start and an end date GROUP BY farmer_code returns the total fat supplied by each farmer in a period; a JOIN combines rows from related tables, such as deliveries with farmer names. Indexes speed up searches on frequently used fields.
Not all data suit relational tables. Process plants generate enormous streams of time-stamped measurements: temperatures, pressures, flows, valve positions, recorded every second from thousands of sensors. These are stored in process historians or time-series databases, designed to compress and retrieve such data quickly by tag and time. A pasteuriser's holding-tube temperature over a year can be called up in seconds to investigate a complaint.
Document and other non-relational databases store flexible records whose structure varies, such as equipment manuals, images of defects or sensor messages. Data warehouses gather data from several operational systems, such as procurement, production, sales and laboratory, into one structure designed for analysis and reporting, and data lakes store large volumes of raw data of many kinds for later analysis. Each type has a place; the essential skill is to know where a piece of information is held, who owns it, and how reliable it is.
Small dairies and laboratories often keep data in spreadsheets used as databases. This works for small volumes but breaks down as records multiply: duplicates appear, different people keep different versions, and links between tables are maintained by hand. Moving such records into a proper database, even a simple one, is a common early improvement.
Data management is the set of practices that keeps data accurate, available and secure throughout its life. Master data, the core reference records of a business, such as farmers, customers, products, recipes, specifications and equipment, must be created and changed under control, because errors in them spread to every transaction. Transaction data, such as deliveries, batches, test results and invoices, record what happened.
Data quality has several dimensions: accuracy, completeness, consistency between systems, timeliness, validity against rules, and uniqueness. Poor data quality has real costs in a dairy: a wrong bank account pays the wrong farmer; a missing test result holds a lot; inconsistent product codes between production and sales make traceability fail. Validation at entry, such as range checks on fat tests and mandatory fields for lot numbers, prevents many problems.
Data must be protected against loss. Regular backups, tested by restoring them, are essential; a common rule is to keep at least three copies on two different types of media, with one copy off-site or offline, so that a fire, hardware failure or ransomware attack does not destroy all of them. Retention periods are set by law, customer requirements and the shelf life of products. Access is controlled so that people see and change only what their role needs.
Personal data, such as farmers' names, addresses, identity numbers, bank details and mobile numbers, are protected by data-protection laws in many countries, which commonly require a lawful purpose, minimal collection, security, and rights for the people concerned to see and correct their data. A dairy that collects such data must know which rules apply to it.
A program is a sequence of instructions that a computer executes. Before writing one, a programmer designs an algorithm, a precise, finite sequence of steps that solves the problem, often sketched as a flowchart, with standard symbols for start and end, process, decision and input or output, or written as pseudocode in plain language.
All common programming languages share a few building blocks. Variables are named storage locations holding values of a data type: integers, floating-point (decimal) numbers, text strings, true-or-false Boolean values, and collections such as lists and tables. Operators perform arithmetic and comparisons. Conditional statements choose between actions: if the temperature of a delivery exceeds the limit, reject it; otherwise accept it. Loops repeat actions, for each farmer in a list or while a condition holds. Functions package a calculation under a name, with inputs and outputs, so that it can be reused and tested, such as a function that returns the price for given fat and protein contents.
Computers store numbers in binary. A bit is a single binary digit and a byte is eight bits. Integers are stored exactly within a range, but most decimal fractions cannot be stored exactly in binary floating-point form, so 0.1 + 0.2 may not equal exactly 0.3; money calculations therefore use rounding rules or decimal types to avoid small errors that accumulate over thousands of farmer payments.
Many languages are in use. General-purpose scripting languages such as Python are widely taught because they are readable and have libraries for data analysis, statistics, charts and machine learning; R is a language built for statistics and graphics, much used in research. Database work uses SQL. Older engineering syllabi taught FORTRAN and BASIC; the concepts carry over.
Industrial control uses its own languages. Programmable logic controllers (PLCs), which run pasteurisers, CIP systems and fillers, are programmed in the languages of the international standard IEC 61131-3: ladder diagram, which looks like relay wiring and is easy for electricians to read; function block diagram; structured text, a high-level textual language; sequential function chart, for step-by-step sequences such as the phases of a CIP cycle; and instruction list, now deprecated. A dairy technologist rarely writes PLC code but should be able to read a sequence description and understand what the controller will do in each step.
Scripting automates repetitive tasks: a short script can read the day's collection-centre files, check them for missing or impossible values, calculate payments and produce a summary; another can pull a week of pasteuriser temperatures from the historian and plot them. Compared with manual spreadsheet work, a script performs the same steps in the same way every time and leaves a record of exactly what was done.
Programs fail in three ways. Syntax errors break the rules of the language and stop the program from running; runtime errors occur while it runs, such as dividing by zero or opening a file that does not exist; and logic errors produce wrong answers without any warning, such as using percentage fat where the formula expects a fraction. Logic errors are the most dangerous, which is why programs must be tested.
Testing means running the code on cases with known answers, including edge cases such as a delivery of zero litres, a missing test result, or a fat content at exactly the boundary of a price band. Automated tests, small programs that check a function against expected results, can be rerun after every change. Code should be readable: meaningful names, short functions, comments that explain why, and consistent layout. Version control systems keep the full history of every change, who made it and why, and let teams work on the same code safely. For calculations that affect payments, product release or regulatory records, the software must be validated: its requirements written down, its correct performance demonstrated by documented tests, and changes controlled.
Reproducibility is a final virtue. An analysis written as a script, with its input data and software versions recorded, can be rerun by a colleague or an auditor months later and give the same result, which is far harder to guarantee with a series of manual spreadsheet operations.
In many dairy regions of South Asia, East Africa, Latin America and South-East Asia, milk comes from very large numbers of small farms, each delivering a few litres once or twice a day to a village collection centre run by a cooperative, a processor or an agent. Testing every delivery by hand for fat with the Gerber method and recording results in ledgers was slow and open to error and dispute. Automatic milk collection units (AMCUs), also called data processor-based milk collection systems, replaced this in many places from the 1990s onward.
A typical unit combines an electronic weighing scale, or a volumetric meter, to measure the quantity; an electronic milk analyser, usually ultrasonic or infrared, that measures fat, solids-not-fat, density and an estimate of added water in about a minute from a small sample; a data processor or computer running collection software; a display and slip printer; and a means of identifying the farmer, such as a code, card or tag. The farmer's identity, quantity, composition, price and amount due are recorded and printed on a slip at once, and often sent to the farmer's mobile phone by text message. The day's data are transmitted to the processor's or union's server over mobile networks.
The benefits are transparency, speed and fewer disputes, because farmers see their results immediately; payment on quality, which rewards better milk; and a database of every delivery that supports procurement planning, quality improvement and fraud detection. The units depend on reliable electricity, often backed by batteries or solar panels, on regular cleaning and calibration of the analysers, and on trained operators.
Milk is paid for by quantity and composition, because fat and protein determine the value of the products made from it. Systems differ between regions. In fat-and-SNF systems, common where milk is collected from small farms, the price per litre or kilogram is read from a two-axis rate chart with fat percentage on one axis and solids-not-fat on the other, or calculated as a price per kilogram of fat plus a price per kilogram of SNF. In component pricing systems, common in Europe, North America and Oceania, farmers are paid per kilogram of fat and per kilogram of protein, sometimes with a value for other solids, and a deduction or charge for volume to reflect transport and processing costs, as in some Oceanian systems that pay mainly on milk solids. Quality bonuses and penalties are added for somatic cell count, bacterial count, temperature, residues and other factors.
A worked component example: a farmer delivers 1,000 kg of milk containing 4.0% fat and 3.3% protein, paid at 5.00 currency units per kilogram of fat and 8.00 per kilogram of protein. The milk contains 40 kg of fat and 33 kg of protein, so the payment is 40 × 5.00 + 33 × 8.00 = 200 + 264 = 464 currency units, or 0.464 per kilogram of milk. If the milk was measured in litres, it must first be converted to kilograms with its density, about 1.030 kg/L for whole milk, because composition is expressed as a percentage by mass.
Payment cycles are often ten days, two weeks or a month, and money is increasingly paid by bank transfer or mobile money rather than cash, which reduces handling risk and builds farmers' financial records. Payment software applies the price rules, deductions for feed, loans or services, and taxes, and produces statements.
Because payment depends on measurement, the instruments must be accurate and seen to be fair. Weighing scales and meters are verified by legal metrology authorities in many countries and checked daily with test weights. Electronic milk analysers estimate composition indirectly, from the speed of sound or from infrared absorption, so they must be calibrated against reference methods, such as the Gerber or Röse–Gottlieb method for fat, using milk samples typical of the region and season, and checked daily with control samples. A drift of 0.1 percentage point of fat, multiplied over thousands of litres a day, is a large sum of money, and farmers notice discrepancies quickly.
Sampling must be representative: milk must be mixed before sampling, because fat rises within minutes. Adulteration is a constant risk where payment rewards quantity and composition: added water, which lowers solids and freezing point; added skim milk powder, sugar, starch, salt or urea to restore density or SNF; and fat from other sources. Analysers flag suspicious density or freezing-point estimates, and collection data can be screened by software for unusual patterns, such as a farmer whose fat content jumps or a centre whose average composition differs from its neighbours'. Suspicious samples are sent to the laboratory for confirmatory tests.
Data integrity at the collection centre also matters: user log-ins, locked rate charts that only the head office can change, records that cannot be edited after printing without an audit trail, and reconciliation of the quantity collected with the quantity received at the chilling centre or plant. Large gaps between collected and received quantities indicate losses, errors or theft.
Enterprise resource planning (ERP) systems are integrated software suites that run the core business processes of a company on one shared database. Typical modules cover finance and accounting; procurement and supplier management; inventory and warehouse management; production planning and execution; sales, distribution and customer service; quality management; plant maintenance; and human resources and payroll. Because all modules use the same master data and transactions, a sale reduces inventory, triggers replenishment, updates the accounts and appears in management reports without re-entry.
ERP systems grew from material requirements planning (MRP), which calculates the raw materials and components needed to make a production plan, from bills of materials and stock levels, and from manufacturing resource planning, which added capacity and finance. In process industries such as dairying, the bill of materials is replaced by a recipe or formula, and production is planned in batches or continuous runs on lines with limited capacity.
ERP sits at the business level of the manufacturing systems hierarchy described by the international standard ISA-95 (IEC 62264): level 4 is business planning and logistics (ERP); level 3 is manufacturing operations management, the manufacturing execution system (MES) that manages orders, batches, recipes and records on the shop floor; level 2 is supervisory control (SCADA and HMI); level 1 is basic control (PLCs) and level 0 is the physical process with its sensors and actuators. Clear interfaces between these levels let a production order created in ERP be executed by MES and the control system, with actual quantities and quality results returned.
Dairying places unusual demands on ERP. The main raw material arrives every day from thousands of suppliers, in variable quantities and composition, and must be paid for on composition; many ERP systems therefore have, or are linked to, a milk procurement module that manages farmer master data, collection routes, quality results, price rules and payments.
Milk is not used as a fixed ingredient but split and recombined. Separation produces cream and skim milk; standardisation recombines them to target fat and protein; and many products produce co-products and by-products, such as buttermilk from butter and whey from cheese. The system must therefore track components, kilograms of fat and protein, rather than only kilograms of product, and allocate costs between joint products. A component mass balance for the whole plant, comparing the fat and protein received with that leaving in products and losses, is a key control.
Dairy products are perishable and have use-by or best-before dates, so inventory is managed by lot and expiry date, issued first-expired-first-out (FEFO), with remaining shelf life checked against each customer's requirements. Many products, such as cheese blocks, are sold by actual weight (catch weight). Distribution of fresh milk involves daily routes, returns of unsold product in some markets, and returnable crates. Products are made to tight specifications, so quality results must block or release lots automatically.
ERP projects are large, costly and risky. Many fail to deliver their expected benefits on time, usually for organisational rather than technical reasons: unclear requirements, poor master data, excessive customisation, insufficient training, and resistance to new ways of working. Successful projects define business processes first, keep the software close to its standard functions, cleanse and migrate master data carefully, test thoroughly with real scenarios, train users, and plan the switch-over to avoid disrupting milk collection and payment, which cannot stop for a day.
Systems are installed on the company's own servers or used as cloud services; cloud systems reduce the need for in-house infrastructure but depend on reliable connectivity and raise questions of data location and access. Smaller dairies use simpler packages or accounting software with add-on modules.
For a dairy technologist, ERP is a daily tool: production planners schedule lines and calculate ingredient needs; production staff confirm batches and record consumption; quality staff enter results and release lots; procurement staff manage milk supply; and managers read reports on yields, losses, costs and stock. Understanding how a transaction in one module affects others prevents errors, such as consuming the wrong lot, and helps people use the data the system holds to improve operations.
A laboratory information management system (LIMS) manages the samples, tests, results and records of a laboratory. In a dairy, the laboratory may handle hundreds or thousands of samples a day: milk from every tanker and many farmers, in-process samples from silos and lines, finished products, environmental swabs, water and ingredients. A LIMS brings order to this flow.
Its core functions follow the life of a sample. Samples are registered, either manually or automatically from production or procurement systems, and given unique identifiers with barcode labels. The system assigns the tests required for each sample type from its specification, schedules them, and sends work lists to analysts and instruments. Results are captured, ideally directly from instruments such as mid-infrared analysers, flow cytometers and chromatographs, avoiding transcription errors, or entered by analysts with checks on format and range. Calculations, such as converting colony counts to counts per gram, are done by the system. Each result is compared automatically with its specification, and out-of-specification results are flagged. Results are reviewed and approved by authorised staff, and certificates of analysis are generated for customers.
A LIMS also manages the laboratory itself: instrument calibration and maintenance, reagent and media lots and expiry, staff training and authorisation, reference materials, quality-control samples and control charts, proficiency testing, and environmental monitoring programmes with their sampling plans and trend reports. Many systems include modules for stability and shelf-life studies.
Laboratory results decide whether products are released, whether farmers are paid, and whether a plant meets the law, so their integrity is critical. A LIMS supports integrity by requiring each user to log in with a unique identity, recording who did what and when in an audit trail that cannot be switched off or edited, controlling which roles may enter, change, review or approve results, and preserving original data and every change with its reason.
Laboratories accredited to ISO/IEC 17025 must control their data and information management systems: validate them for their intended use, protect data against unauthorised access and loss, and keep records that allow tests to be reconstructed. Regulators in several jurisdictions set specific rules for electronic records and signatures in regulated industries, such as those of the United States for electronic records and the European guidance on computerised systems; dairies producing infant formula or pharmaceutical-grade lactose often follow them, and other dairies use them as models of good practice.
Computerised systems are validated by a risk-based approach: user requirements are defined; the system is configured; tests show that it performs as required, including calculations, specification checks, audit trails and security; and changes after go-live are controlled and retested. A widely used industry guide for this risk-based approach is known as GAMP. Validation is proportionate: a system that releases infant formula needs more rigorous evidence than one that logs canteen water tests.
A dairy laboratory chooses a LIMS by its needs: the number and types of samples, the instruments to be connected, links to ERP, procurement and production systems, regulatory requirements, the number of sites, and budget. Commercial systems range from large configurable platforms to smaller packages for single laboratories; some are installed locally and some are offered as cloud services.
Integration multiplies the value. When the procurement system sends tanker identities and farmer codes to the LIMS, and the LIMS returns fat, protein, somatic cell and bacterial results, payment calculations run without re-entry. When the ERP creates a production lot, the LIMS creates the samples required for it, and lot release in the ERP waits for approval in the LIMS. Positive pathogen results can trigger holds automatically.
Good use depends on good configuration. Specifications must be correct and up to date for every product and market; test methods, units, decimal places and rounding must match the method; and sample types and sampling points must reflect the real plant. Analysts need training not only in operating the system but in understanding why its controls exist. Common problems include parallel paper records that contradict the system, shared log-ins, results entered in batches long after the test, and specifications changed without approval. Each undermines the trust that a LIMS is meant to create.
Traceability is the ability to follow a food through specified stages of production, processing and distribution. The Codex Alimentarius principles for traceability and product tracing describe its basic form: each business should be able to identify the immediate supplier of each input and the immediate customer of each output, one step back and one step forward. Many national laws and food-safety certification schemes require at least this, and many businesses go further with internal traceability, linking each output lot to the input lots and process conditions that produced it.
Traceability rests on identification and records. Each lot or batch of raw material, ingredient, packaging material and product must have a unique identifier; each movement and transformation, receipt, storage, mixing, processing, packing, dispatch, must be recorded with the lots involved, quantities and times. Tracing backward from a complaint or a positive test finds the inputs and suppliers involved; tracking forward from a suspect input finds every product and customer that may be affected.
Dairy traceability has particular difficulties. Milk from many farms is pooled in tankers, then in silos, then blended continuously into products, so a single carton may contain milk from hundreds of farms collected over two days. Lots are therefore defined by time windows and equipment, such as a silo fill or a production run between cleanings, and the records must show which tankers fed which silo and which silo fed which run. Rework, the return of product into the process, and the use of milk powders and ingredients across many products, widen the links. The narrower the lot definition, the smaller a recall, but the greater the record-keeping burden.
Common identification standards allow trading partners to read each other's codes. The GS1 system, used worldwide in retail and supply chains, assigns a Global Trade Item Number (GTIN) to each product type, a Global Location Number (GLN) to each location and business, and a Serial Shipping Container Code (SSCC) to each pallet or logistic unit. Additional data such as lot number, production date and best-before date are encoded with application identifiers in barcodes such as GS1-128 on cases and pallets, or in two-dimensional codes such as GS1 DataMatrix and QR codes, which can carry more data in less space. Retail point-of-sale barcodes traditionally carry only the GTIN, but two-dimensional codes on consumer packs are being introduced in many countries to carry lot and date information as well.
Traceability software records events: what was observed or done, which items were involved, where, when and why. The GS1 EPCIS standard defines a common format for such events so that they can be shared between companies. Within a plant, traceability data come from ERP, MES, warehouse systems, scanners and printers; the software links them into a genealogy of each lot.
Claims are sometimes made that new technologies, such as distributed ledgers (blockchain), guarantee traceability. They can make shared records harder to alter, but they do not make the original data correct: if a lot is mislabelled at the plant, the ledger faithfully records the wrong information. Accurate identification and recording at each step remain the foundation.
A recall is the removal of unsafe or non-compliant product from the market, including from consumers; a withdrawal removes product from the distribution chain before it reaches consumers, or for reasons that are not a safety issue. Recalls in dairies are caused by pathogens such as Listeria monocytogenes or Salmonella, undeclared allergens, foreign bodies, chemical residues, under-processing, packaging faults and labelling errors. National food authorities set rules for when and how a recall must be notified and carried out; many publish recall guidance and lists of recalled products.
Recall software, often part of ERP or traceability systems, supports the process. It identifies the affected lots, calculates the quantities produced, in stock, shipped and still in the market, lists the customers and locations that received them with contact details, produces notifications, and records the quantities returned or destroyed so that effectiveness can be shown. Speed matters because perishable dairy products are consumed within days.
A business tests its system with mock recalls: a lot is chosen, and the team traces it backward to its raw materials and forward to every customer, reconciling quantities produced with quantities accounted for. Food-safety certification schemes require such tests at regular intervals, often at least annually, and many set expectations for how quickly the trace must be completed, commonly within a few hours. A mass balance that does not close, for example 5% of a lot unaccounted for, is a finding that must be investigated. Lessons from mock recalls, such as missing customer contacts or gaps in silo records, are used to improve the system.
Dairies collect far more data than they use. Analysis turns them into decisions: whether yields have fallen, which line wastes most product, which farmers need help with hygiene, whether energy use per tonne is improving. The process has recognisable steps: define the question; gather the relevant data; clean them; explore them with summaries and charts; analyse them with suitable methods; and communicate the result so that someone can act on it.
Cleaning usually takes most of the effort. Real data have missing values, duplicates, impossible entries such as a fat content of 45% caused by a misplaced decimal point, inconsistent units, time stamps from clocks that disagree, and codes that changed partway through a year. Each problem must be found and handled explicitly, with a record of what was done; silently deleting awkward values can bias the result. Outliers deserve investigation rather than automatic removal, because the unusual delivery may be the adulterated one.
Key performance indicators summarise operations for management. In a dairy they include milk intake and its composition, product yield against theoretical yield, losses of fat and protein in effluent, energy and water use per tonne of milk, overall equipment effectiveness of packing lines, laboratory turnaround time, and complaint and rejection rates. Good indicators are clearly defined, measured the same way over time, and tied to actions; too many indicators dilute attention.
The right chart depends on the question. Line charts show change over time, such as daily milk intake or the fat content of a route; bar charts compare categories, such as losses by product; histograms show the distribution of one variable, such as fill weights; scatter plots show the relation between two variables, such as the temperature of milk at collection and its bacterial count; box plots compare distributions across groups, such as somatic cell counts by village; Pareto charts rank causes of a problem by frequency or cost; and control charts show whether a process is stable over time. Maps show the geography of supply.
Good charts are honest and easy to read. Axes are labelled with quantities and units; scales are chosen so that differences are neither exaggerated nor hidden, and a bar chart's value axis should start at zero; three-dimensional effects, unnecessary colours and decorations are avoided because they distort or distract; and colour is used consistently, with attention to readers who cannot distinguish red from green. A chart should make one point clearly, stated in its title.
Dashboards bring several indicators together on one screen, often updated automatically from ERP, LIMS and historian data. They are useful for daily management meetings and control rooms, but they invite over-reaction to random variation; displaying limits or targets and using control-chart logic helps people tell signals from noise.
Most analysis needs only descriptive statistics, comparisons of means, correlation and regression, and control charts, all of which are covered in the statistics subject. Several further methods are now common in dairy data work.
Multivariate methods handle many variables at once. Principal component analysis reduces many correlated measurements, such as dozens of spectral readings or process variables, to a few components that capture most of the variation, revealing patterns and outliers. Partial least squares regression relates many predictor variables to a response and is the standard method for calibrating near-infrared and mid-infrared analysers, which predict fat, protein, lactose, total solids and even fatty acid profiles from spectra.
Machine learning builds predictive models from data. In supervised learning the model learns from examples with known outcomes: regression predicts a number, such as cheese yield from milk composition and process conditions, and classification predicts a category, such as whether a delivery is likely to be adulterated. In unsupervised learning the model finds structure without labels, such as clusters of farms with similar quality patterns. The central danger is overfitting: a complex model can fit the training data closely but predict new data poorly. Models are therefore tested on data not used to build them, by splitting the data into training and test sets or by cross-validation, and their predictions are checked for sense. A model is only as good as its data, and it may reproduce biases or fail when conditions change, such as a new feed season or a new analyser.
Sensors convert physical quantities into electrical signals that control systems and data systems can use. Temperature is measured mostly with resistance temperature detectors, especially the platinum Pt100 type, whose resistance rises predictably with temperature and which are accurate and stable, and with thermocouples, which generate a small voltage at the junction of two metals and suit higher temperatures. Pasteuriser holding-tube sensors are among the most critical and are checked against a reference thermometer at set intervals.
Pressure transmitters measure line and vessel pressures, including the pressure difference that keeps pasteurised milk at higher pressure than raw milk in a regenerative heat exchanger. Flow is measured with electromagnetic flowmeters, which work on conductive liquids such as milk and have no moving parts or obstructions, and with Coriolis meters, which measure mass flow directly and also density. Level is measured by pressure, radar or ultrasonic sensors. Conductivity sensors distinguish water, milk and cleaning solutions, and are used to detect phase changes in CIP and product–water interfaces. Inline analysers measure fat, protein and total solids by near-infrared spectroscopy, and pH, turbidity and dissolved oxygen sensors are used in specific applications.
Sensor signals are traditionally transmitted as 4 to 20 mA current loops, in which 4 mA represents the bottom of the range and 20 mA the top, so that a reading of 0 mA reveals a broken wire; digital fieldbus and industrial Ethernet networks increasingly carry the values together with diagnostic information. Every measurement that matters must be calibrated against a traceable standard at a defined interval, and the result recorded.
The Internet of Things (IoT) connects sensors and devices to networks so that their data can be collected, stored and used remotely. A typical IoT system has four layers: devices with sensors at the edge; a gateway that collects their readings and connects to a network; the network itself, which may be cellular, low-power wide-area radio such as LoRaWAN or narrowband IoT, Wi-Fi or wired; and a cloud or server platform that stores the data, runs analysis and presents it through dashboards, alerts and applications. Lightweight messaging protocols such as MQTT carry data from small devices, and OPC UA is the common standard for exchanging data with industrial control systems.
Dairying uses IoT from farm to shop. On farms, bulk tank monitors send milk temperature, volume and agitation status to the farmer's phone and the processor, raising an alarm if cooling fails; milking robots and parlour systems record each cow's yield and milk conductivity, an early sign of mastitis; and activity collars detect heat and illness. At village collection and chilling centres, remote monitoring of milk temperature and cooler operation helps keep milk cold where power is unreliable, often with solar-powered coolers. In tankers and the distribution chain, temperature loggers and location trackers record the cold chain. In plants, energy meters on major equipment and wireless sensors on motors for vibration and temperature support energy management and predictive maintenance.
Edge computing processes data close to where they are produced, reducing the volume sent over networks and allowing quick local responses even when the connection fails.
IoT data are only as useful as they are reliable. Sensors drift and fail, batteries run down, connections drop and clocks go out of step, so systems need health monitoring, calibration schedules, and rules for handling gaps; a missing temperature record for a night may be a sensor failure or a cooling failure, and the system must tell which. Time synchronisation across devices is essential when data from several sources are combined.
The volume of data can be large. A plant with ten thousand signals recorded every second produces hundreds of millions of values a day. Historians compress data by storing changes rather than every reading, and analytics must be designed to answer specific questions rather than collect data for its own sake.
Ownership and access raise questions. Data from a farmer's bulk tank, cows and fields may be collected by the processor, an equipment supplier and a cooperative; agreements should state who owns the data, who may use them and for what, and how the farmer can obtain them. Codes of practice on agricultural data have been published by farm and industry organisations in several regions. Connectivity, cost and skills remain barriers on small farms and in rural collection systems, where simpler solutions, such as text-message alerts, may deliver more benefit than sophisticated platforms. Each connected device is also a possible entry point for attackers, which links IoT to cybersecurity.
Operational technology (OT) is the hardware and software that monitors and controls physical processes: programmable logic controllers, distributed control systems, supervisory control and data acquisition (SCADA) systems, human–machine interfaces, historians and the networks that link them. In a dairy, OT runs reception, pasteurisation, UHT, evaporators, dryers, CIP, refrigeration, fillers and utilities. Information technology (IT) handles business data: e-mail, ERP, finance and office systems.
Security priorities differ. In IT, the classic priorities are confidentiality, integrity and availability, roughly in that order. In OT, availability and safety come first, then integrity, then confidentiality: a stopped pasteuriser or an ammonia refrigeration system out of control matters far more than a leaked document. OT systems also differ technically. They often run for fifteen to twenty-five years; many use old operating systems that can no longer be updated; they cannot be rebooted or patched at will during production; their communication protocols were designed without authentication; and they are maintained by vendors who may need remote access.
Once isolated, OT networks are now connected to business networks and the internet, for production reporting, remote support, energy management and IoT. This connection has brought them within reach of attackers. Well-known incidents include malware designed to damage industrial equipment and, more commonly, ransomware attacks on food and beverage companies that encrypted business systems and forced production to stop, sometimes for days, because production could not be planned, tracked or shipped without them.
The commonest attacks begin with people. Phishing e-mails trick staff into opening malicious attachments or entering passwords on fake sites; stolen or weak passwords, and default passwords never changed on devices, let attackers log in; and remote-access connections left open for vendors are a frequent route into control networks. Malware can also arrive on USB drives and laptops brought to the plant by contractors. Insiders, whether careless or malicious, are another source of risk.
Once inside a business network, attackers look for paths into the OT network: shared accounts, flat networks with no separation, engineering workstations used for e-mail, or historians that bridge the two. In the control network they can stop processes, change setpoints, disable alarms or encrypt servers. Ransomware criminals usually seek payment rather than physical damage, but the result for a dairy, with perishable milk arriving daily from farms, can be severe: milk that cannot be processed must be diverted or discarded, and farmers cannot be paid if procurement systems are down.
Vulnerabilities in control systems are common because many devices were designed for isolated networks. Protocols widely used in industry, for example, often accept commands from any device on the network without checking its identity. Public databases list known vulnerabilities in industrial products, and search engines that index internet-connected devices regularly find control systems exposed directly to the internet, a basic and avoidable failing.
Defence in depth uses several layers of protection so that no single failure lets an attacker succeed. The foundations are an inventory of every OT asset, its software version and its connections, because what is not known cannot be protected; and segmentation of networks into zones with controlled conduits between them, following the reference architecture often called the Purdue model and the zones-and-conduits approach of the international standard series IEC 62443. Business and control networks are separated by firewalls, with a demilitarised zone holding servers, such as historians and patch servers, that both sides need, so that no direct connection runs from the internet or office network to controllers.
Access control limits who can do what. Each person has an individual account with the least privilege needed; default passwords are changed; multi-factor authentication protects remote access and administrator accounts; vendor remote access is granted only when needed, through a controlled, monitored connection, and removed afterwards. Removable media are scanned or restricted. Patches are applied after testing and during planned stops, and where patching is impossible, compensating controls such as stricter network rules are used.
Preparation limits the damage of an attack that succeeds. Offline, tested backups of PLC programs, HMI configurations, recipes and servers allow recovery; manual procedures allow critical operations, such as receiving and cooling milk, to continue safely for a time without IT systems; and an incident response plan, rehearsed with production, IT, management and suppliers, sets out who does what. Staff training in recognising phishing and following security rules is among the most effective measures. Frameworks such as IEC 62443, ISO/IEC 27001 and national cybersecurity frameworks guide these programmes, and food-safety and food-defence plans increasingly include cyber risks.
Build your own spreadsheets for standardisation, ice cream mix, plate counts and milk payment, test them against hand calculations, and then rebuild one as a short script. Learn enough SQL to answer simple questions from a table of deliveries. During in-plant training, follow one tanker of milk through every system that records it, from the collection unit to the LIMS, the ERP and the traceability report, and note where data are re-entered by hand; those are the places where errors arise and improvements are possible.
A quality, production or procurement officer five years into the job lives in these systems: checking collection-centre data for anomalies, approving laboratory results in the LIMS, releasing lots in the ERP, running mock recalls, building dashboards for daily meetings, and specifying requirements when a system is upgraded. Typical decisions are whether an unusual pattern in a centre's data is a calibration problem or adulteration, whether a spreadsheet used for release needs to be replaced by validated software, and how to define lots so that traceability is both practical and narrow.
A plant head or chief information officer with 25 years' experience decides the digital architecture of the business: which ERP, MES, LIMS and procurement systems to run, how they connect, which data to collect from farms and plants, and how to protect them. These are large investments with long lives, and their success depends more on processes, master data and people than on software features. Cyber risk is now a board-level issue in food businesses, and senior managers must ensure that control systems are segmented, backed up and covered by rehearsed incident plans.
A professor or research leader uses data science to study dairy systems: predictive models of milk quality and yield, spectral calibration, sensor-based monitoring of animal health and process performance, and data-driven optimisation of plants and supply chains. Research must handle real data honestly, with reproducible code, proper validation of models on independent data, and attention to data ownership and privacy. Teaching students to think critically about data, models and dashboards is as important as teaching the tools themselves, which change every few years.
This subject describes software by type and function only; it does not teach any product. Programming is introduced at concept level, without a full course in any language. Data-protection, electronic-records and recall laws differ between countries and are described in general terms. Detailed control-system engineering is left to instrumentation-process-control-dairy, statistics to mathematics-statistics-dairy and dairy-quality-assurance-sqc, and milk pricing economics to dairy-economics-cooperatives. Specific cyber incidents in the dairy sector are not described in detail because public accounts are incomplete.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.