# SCTAK incident, weather, wildfire and water collection

Built and checked September 28, 2026. This is a working local collector and saved-data viewer. It is not deployed to the public/member service, and no persistent collector is currently running.

## Verified snapshot

At 12:29 UTC, all 11 configured source checks succeeded, producing 563 records:

| Collection | Records | Meaning |
|---|---:|---|
| NWS | 2 | Actual, unexpired SC weather alerts |
| SCDOT incidents | 2 | Website-listed road incidents |
| SCDOT construction | 3 | Website-listed construction events |
| SCDOT road weather / other events | 0 / 0 | Successful empty responses, not collection errors |
| USGS stage | 279 | Latest time-series observations, not necessarily unique stations or recent readings |
| USGS streamflow | 263 | Latest time-series observations; older readings retained and flagged |
| NIFC current SC fire incidents | 0 | No matching records in this source; not a claim of zero SC wildfires |
| NOAA coastal water levels | 2 | Charleston and Springmaid Pier, meters relative to MLLW |
| NASA FIRMS | 12 | NOAA-20 VIIRS near-real-time thermal detections in an approximate SC rectangle over one day |

USGS returned 42 stage and 126 streamflow observations more than two hours old. These remain explicitly stale even though the API request succeeded. Multiple time series can exist at one station. No flood classification is inferred from a river measurement.

## Use

From the website directory, with Python 3 and network access:

```powershell
python tools/collect_live_feeds.py
python tools/collect_live_feeds.py --watch
python tools/test_live_feeds.py
```

The default runs once. `--watch` stays in the foreground and waits five minutes after each collection cycle; stop it with Ctrl+C. It is not installed as a Windows service or scheduled task. Avoid overlapping collector processes.

Set `FIRMS_MAP_KEY` in the collector's server-side environment to enable NASA. Optional `USGS_API_KEY` is sent only to api.waterdata.usgs.gov. The NASA key from the user's signup email was used successfully in a process-scoped environment variable for validation; it was not saved to a public file or permanently configured. A future run without the key marks NASA unavailable and preserves the last snapshot as stale. Configure secrets in the deployment's secret store before installing a recurring service.

Open `live-feeds.html` through the SCTAK web server. It loads `status.json` and `live.geojson`, displays per-source collection status and lets users filter records. It recalculates age and alert expiry while open; reload the page to fetch a newly saved collection. Viewing this page does not run the collector.

## Data contract and failures

Each normalized GeoJSON feature includes stable source/provider IDs, source attribution, category, optional geometry, provider observation time, collection time, expiry when supplied and original units. Coordinates from SCDOT are converted from strings to numbers. Unknown geometry remains null; NWS affected-zone URLs and geocodes are retained when no polygon is provided.

Successful empty responses clear previous records for that source. Failed requests preserve the previous successful records and mark collection status stale. Expired alerts are removed even if the provider request fails. Missing credentials are distinct from an empty response. Exceptions record only their class, avoiding key-containing request URLs in public error output. Files are written atomically; the combined snapshot is written after a cycle completes.

The viewer marks collection snapshots older than 15 minutes stale. Observation thresholds are two hours for water data and 24 hours for fire/thermal records. NWS validity follows provider expiry instead of treating issuance age as an outage. Road metadata has no verified observation timestamps and explicitly states that limitation.

Eight offline tests passed: stale-cache retention, successful-empty replacement, expiry during an outage, exclusion of test/cancelled/expired alerts, old-observation detection, valid-alert age semantics, pagination and cross-host pagination rejection. Real network collection validated every configured source. Browser layout/interaction testing of the new viewer remains outstanding.

## Source references

- [NWS API documentation](https://www.weather.gov/documentation/services-web-api)
- [SCDOT 511SC](https://www.511sc.org/): its public website publishes incident, construction, weather and other-event GeoJSON under `https://sc.cdn.iteris-atis.com/geojson/icons/metadata/`. This is website metadata, not a promised stable developer API; reuse/redistribution terms remain to be established.
- [USGS latest continuous observations](https://api.waterdata.usgs.gov/ogcapi/v1/collections/latest-continuous): state_code=45, parameter_code=00065 (stage) / 00060 (streamflow); follows pagination.
- [NIFC current incident service](https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Incident_Locations_Current/FeatureServer/0): POOState=US-SC; separates wildfire, prescribed fire and incident complex.
- [NOAA CO-OPS API](https://api.tidesandcurrents.noaa.gov/api/prod/): latest water_level, metric, GMT, MLLW.
- [NASA FIRMS area API](https://firms.modaps.eosdis.nasa.gov/api/area/): NOAA-20 VIIRS NRT; rectangle -83.5,32.0,-78.4,35.3. This includes adjacent states and coastal water; it is not state-boundary clipping. A thermal detection does not establish a wildfire.

## Remaining integration

SC Forestry fiResponse direct incident access, fire perimeters, radar overlays, SCDPS WebCAD, partner feeds and strict SC boundary clipping are not implemented here. The existing SC Forestry public viewer remains a separate source for verification. NIFC is not a substitute for complete local fire reporting.

No CoT is sent to TAK Server, no push notifications are issued, and no member account changes are made. Deployment needs a private collector runtime, secret configuration, scheduling, monitoring and tested AE/TAK adapters. Only the generated public snapshots should be served to browsers; never serve secret files. Recheck provider terms before public/commercial redistribution.