A wrong character flooded the birch: diary of an autonomous irrigation system

One night the system opened a zone for an hour instead of seventeen minutes and flooded the birch bed — the plant I care about most. I found out from the security camera, not from a log. A wrong character in the way I was communicating with the controller. That bug is the reason this project got serious.

The idea

A timer-based system is dumb: it waters at 6 whether it’s raining or shining. I wanted something that would reason zone by zone. My garden is 2,500 m², 10 zones, 3 wireless controllers. Every night the system computes how much water each one needs, an AI agent reviews the plan, and the controllers execute on their own.

The physical model before the AI

The heart isn’t the AI: it’s the agronomy. I use ET0 per FAO-56 (reference evapotranspiration) plus the soil moisture sensors. The per-zone demand is pure FAO-56 water balance — no numbers hardcoded by hand, every parameter has a value, a confidence and a provenance:

@property
def taw_mm(self):   # Total Available Water: acqua totale disponibile
    return max(0.0, (self.field_capacity_pct/100 - self.wilting_pct/100)
                     * self.root_depth_mm)
@property
def raw_mm(self):   # Readily Available Water = p · TAW
    return self.depletion_fraction * self.taw_mm
@property
def max_session_mm(self):   # mai riempire troppo in un colpo
    ceiling = 8.0 if self.soil_texture == "peat" else 12.0
    return min(0.6 * self.raw_mm, ceiling)

The plant coefficients (Kc, root depth) come from the FAO-56 tables, the soil properties from the USDA textural class. The sensor, though, beats the math: if a real probe reads below the stress threshold, the zone becomes critical and the urgency shoots to 80-100, regardless of the forecasts. With one trick: after heavy rain, a probe under the tree canopy (where the rain didn’t reach) reads “dry” — and it must be ignored, otherwise I water for nothing.

Where the AI sits

The physical model computes the millimeters, the liters, the urgency. The AI decides the strategy: one or two cycles (halving the durations helps infiltration on clay/peat soil), the delays between zones to let the cistern recharge, the time window for fungal risk. It receives all the context and responds in strict JSON. Hard rules constrain it: it can never reduce a zone in critical stress, and only a critical Pythium risk can skip the sprinklers — but if a zone is in heat stress, a deterministic override waters it anyway, because “a dead lawn is irreversible, Pythium can be cured”.

I changed the model twice. The last jump happened when the previous model, in a 38°C heat wave, kept putting “skip the humid night, it’s reversible anyway” over a lawn that was drying out. Wrong agronomic priority. Current cost: one call per night, ~$0.025, about 9 dollars a year.

The hard part: decoding the controller

The controllers are proprietary, designed to be commanded only from their app. The flooding bug came from there: I was sending the duration as MM:SS, but the firmware expects HH:MM. A 17-minute zone (1008 seconds) started as "16:48" → interpreted as 16 hours and 48 minutes.

To figure it out, one morning: a rooted Android phone with the official app, mitmproxy intercepting the HTTPS, decompiled APK alongside. 246 API flows captured, every “Save Program” tap decoded. Key discoveries:

• The modules are LoRa children of a hub; every call uses the hub’s serial, never the child’s ID.
• Three different units on the same wire: start times in minutes-from-midnight, durations in seconds, the manual command as a "HH:MM" string.
• The server responds 200 even to malformed bodies and then delivers nothing. You have to poll the controller’s real state until it confirms.

The real lesson wasn’t the right format: it was to stop commanding zone by zone and switch to the firmware’s native programming, injected once and executed autonomously. And to discover that LoRa has its own timing: from injection to execution 19-24 minutes pass, variable. I set a fixed margin of 25.

The deploy and the network gotcha

It runs on a self-hosted Kubernetes cluster. Three CronJobs share the same image: the planner (starts in the evening, computes, injects, arms the watchdog, then the pod dies in ~90 seconds — it’s the firmware that executes, no need to keep anything alive), the watchdog (every 5 minutes within the window, checks for anomalies and at the end suspends its own CronJob by itself), and a daytime anti-heat pass.

The gotcha that stole days from me: with the Cilium CNI, a standard NetworkPolicy can’t whitelist the Kubernetes API server — Cilium gives it a special identity after kube-proxy’s NAT. Six lines fixed silent 5-minute timeouts:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: { name: allow-kube-apiserver, namespace: my-garden }
spec:
  endpointSelector: {}            # tutti i pod del namespace
  egress:
    - toEntities: [ kube-apiserver ]

What I learned

1. Smart automation isn’t putting an LLM everywhere. The right physical model (FAO-56 has existed for decades) does the math; the AI only decides where judgment is needed. Nine dollars a year.
2. HTTP 200 lies. Never trust the status code with proprietary hardware: verify the real state.
3. Silent degradation is the enemy. For entire nights an invisible fallback masked a truncated JSON. Now everything fails out loud.

Next chapter: a small bit of vision — a camera watching the lawn — as an extra signal, never as absolute truth.