Research methods · HydroSight

Research design

The project follows design science research: a useful artifact is built to solve a real problem, then evaluated against that problem. The artifact here is HydroSight, an analytics and alerting system for one real plant. The work moved through six phases, similar to CRISP-DM.

Understand the plant

Study the 49.9 kW site, its sensors and its operating context on an agricultural channel.

Prepare the data

Profile 590,674 records, fix sentinels and dead channels, fill gaps and flag every change.

Model

Virtual flow sensor, bearing health model, Isolation Forest and forecasts.

Translate

Alert rules and natural-language messages for operators, managers and government.

Build

Responsive web app with a replay simulator, PHP API and assistant.

Evaluate

Backtest alarms on five years, test forecasts on unseen 2026 data, review insights.

System architecture

Data flows from the PLC to the cloud collector, through the Python analytics pipeline, and out to the people who act on it. In this project the 2026 data is replayed in the browser to show the real-time path.

Data and cleaning

The source is the plant's own log: 590,674 five-minute records from 11 March 2021 to 30 October 2026 with 22 columns. Profiling found six kinds of problem, and each was fixed with a rule that leaves a trace in a quality flag.

Missing intervals. 2,132 records were missing in 14 gaps (longest 24 hours). The timeline was rebuilt on a complete 5-minute grid and gaps were filled by time-based linear interpolation.
Sentinel values. The "currently generating power" channel reads about 654 kW whenever the plant is offline (56,977 records), and generator power overflows to about 6,550 kW in 144 records. These were treated as invalid. When stopped, power was set to zero; when running, grid power was rebuilt from generator power using a fitted inverter relationship.
Frozen inverter data. In 12 periods (82 hours) power, voltage, speed and the energy counter all stopped changing while pressure and temperature kept updating. The energy counter barely moved, so these are hidden stops, not running time. They are classed as HELD, with power set to zero and voltage marked unavailable, and they raise their own alert.
Dead channels. Frequency is always 60 Hz, and water level, flow rate and efficiency are always zero. They were dropped and replaced with physics-based estimates.
Sensors not yet installed. Bearing temperatures read zero until 24 June 2021 and were marked as unavailable.
Counters. Daily energy was recomputed from the cumulative counter so it stays consistent across gaps.

$$x(t)=x(t_a)+\frac{t-t_a}{t_b-t_a}\,\bigl[x(t_b)-x(t_a)\bigr],\qquad t_a\lt t\lt t_b$$

Linear interpolation across a gap between the last good record at $t_a$ and the next at $t_b$.

$$H=\frac{p}{\rho g}$$

Net head $H$ (m) from penstock pressure $p$ (Pa), with water density $\rho$ and gravity $g$.

Physics-informed virtual sensing

Without a working flow meter the plant cannot report how much water it uses or how efficiently. Penstock friction solves this. The more water flows, the more head is lost before the turbine, so the measured head falls as output rises. A curve was fitted to the median head in each 1 kW power bin.

$$H(P)=H_0-a\,P^{\,b}\qquad H_0=87.36\ \text{m},\ a=0.0048,\ b=1.672$$

$H_0$ is the static head at zero flow and $P$ is grid power in kW.

The head loss grows with the square of flow, $H_0-H=K\,Q^2$. Calibrating $K$ so that design flow (0.11 m³/s) gives the measured loss at rated output (3.31 m) lets flow and efficiency be estimated from every record:

$$Q=\sqrt{\frac{H_0-H}{K}},\qquad P_h=\rho\,g\,Q\,H_0,\qquad \eta_{\text{w2w}}=\frac{P_{\text{grid}}}{\rho\,g\,Q\,H_0}$$

$K=273.6\ \text{s}^2/\text{m}^5$. Flow is only estimated while running with head of at least 80 m, using a one-hour rolling median of head to smooth sensor resolution.

Performance and loss accounting

$$\text{CF}=\frac{E}{P_{\text{rated}}\,T},\qquad A=\frac{T_{\text{run}}}{T_{\text{period}}}$$

Capacity factor $\text{CF}$ for energy $E$ over period $T$, and availability $A$ as the share of time running.

$$E_{\text{lost}}=\sum_{i\in\text{stopped}}\bar P_{m(i)}\,\Delta t,\qquad \text{Revenue}_{\text{lost}}=E_{\text{lost}}\times 34\ \text{JPY/kWh}$$

$\bar P_{m(i)}$ is the median running output in the calendar month of interval $i$, and $\Delta t$ is 5 minutes. 34 JPY/kWh is the feed-in tariff for hydropower under 200 kW.

Bearing health model

A gradient-boosting regressor learned how bearing 1 temperature normally depends on bearing 2 temperature, current and 6-hour load and time of day during 2021-22, before the typhoon. It was trained to 30 June 2022 and checked on July to mid-September 2022 (mean absolute error 2.3 °C). Applied to later years, the difference between actual and expected temperature becomes a health indicator.

$$F_m(\mathbf x)=F_{m-1}(\mathbf x)+\nu\,h_m(\mathbf x),\qquad \text{excess}(t)=T_1(t)-\hat T_1(t)$$

Each new tree $h_m$ corrects the remaining error, scaled by learning rate $\nu=0.05$. Absolute-error loss keeps occasional spikes from distorting the baseline.

Multivariate anomaly detection

An Isolation Forest looks at eleven signals together. Unusual combinations are easier to separate from the rest with random splits, so they end up with short paths in the trees. Signals were first scaled with the median and median absolute deviation so outliers do not distort the scale.

$$z=\frac{x-\operatorname{median}(x)}{1.4826\,\operatorname{MAD}(x)},\qquad s(\mathbf x,n)=2^{-\,\mathbb E[h(\mathbf x)]/c(n)}$$

$h(\mathbf x)$ is the path length to isolate a record, and $c(n)=2H(n-1)-2(n-1)/n$ normalizes it. 200 trees were trained on 120,000 running records from 2023-25, and the alert threshold is the 99.5th percentile of training scores.

Forecasting

Gradient-boosting models predict the change from the current output at 1, 6 and 24 hours ahead, using recent lags, rolling means, head, guide vane and calendar features. Quantile models give the 10th and 90th percentiles for an 80% band. Training used 2021-2025 and testing used only 2026.

$$\text{MAE}=\frac1n\sum|y-\hat y|,\quad \text{RMSE}=\sqrt{\frac1n\sum(y-\hat y)^2},\quad \text{SS}=1-\frac{\text{RMSE}_{\text{model}}}{\text{RMSE}_{\text{persistence}}}$$

A positive skill score $\text{SS}$ means the model beats persistence, which assumes output stays at its current level.

Alarm management

Fourteen rules turn signals into alerts. Following the ANSI/ISA-18.2 alarm-management standard, each rule needs the condition to hold for a minimum time (on-delay), must cross a separate clear level to reset (deadband), and cannot re-alarm within a cooldown period. Readings that are frozen are never alarmed on. Alerts that depend on the unit running close quietly when it stops, and a lower-priority alert on the same signal is suppressed while a higher one is active.

$$\text{raise at }t\iff \bigwedge_{k=0}^{d-1}\bigl[x(t-k)\ \text{op}\ \theta\bigr]\ \wedge\ t-t_{\text{last}}\ge T_c,\qquad \text{clear at }t\iff \bigwedge_{k=0}^{d_c-1}\bigl[x(t-k)\ \overline{\text{op}}\ \theta_c\bigr]$$

Threshold $\theta$, clear level $\theta_c$, on-delay $d$ and clear delay $d_c$ in samples, cooldown $T_c$. Over five years this cut 4,582 raw threshold crossings to 509 notifications.

Natural-language generation

Messages are produced with the classic data-to-text pipeline. Content selection picks the alert, the reading and the context. Aggregation combines alerts raised in the same five-minute step into one SMS. Lexical choice adapts wording to the reader: short imperative SMS for staff (single 160-character segment where possible), structured e-mail with causes and actions from the knowledge base for managers, and formal notices for the government energy office. Realization fills templates with formatted values and units. Daily briefs at 07:00 and weekly reports on Monday mornings are generated from the day's statistics.

Language-model assistant

The "Ask the plant" box and the Explain button always give a deterministic answer first, built from live values and the closest knowledge-base entries (keyword scoring over titles, tags and summaries). When the PHP server has an API key, the same question, the telemetry snapshot and the retrieved entries are sent to a large language model through its HTTP API. A system prompt limits answers to the supplied data, asks for plain English under 160 words with numbered actions, and puts safety first. Requests are rate-limited per client, and the page falls back to the knowledge-base answer if the model is unavailable.

Visualization design

The interface follows overview first, zoom and filter, then details on demand. Key numbers sit in tiles at the top, trends and limits sit in small charts, and details live in tooltips, the alert log and the e-mail view. The visual language is a light adaptation of the Linear design system: white cards with hairline borders, Inter type, one lavender accent for data, and red and amber kept for limits and alerts so they stand out.

Limitations and ethics

There is no measured flow, so the virtual sensor depends on the nameplate calibration. A prefectural case sheet lists a larger intake flow (0.186 m³/s), which would change absolute flow and efficiency values but not the trends.
No maintenance log was available, so causes of outages are inferred from data and public weather records.
Forecasts are only as good as persistence until rainfall and channel-level data are added.
SMS and e-mails are simulated. Phone numbers are masked and addresses use reserved example domains.
The telemetry belongs to the plant operator and is used here only for academic analysis.

How the system works, step by step