aric/fajr-watch
1
0
Fork 0
mirror of https://github.com/acamarata/fajr-watch.git synced 2026-06-30 18:54:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Initial scaffold: turnkey Raspberry Pi twilight observation station

Browse source 
Complete project structure for an automated Fajr/Isha observation appliance:

Hardware:
- BOM for 3 build tiers ($270, $465, $1000)
- 20-unit bulk order spec (~$9,900 for fleet)
- Solar power, weatherproof enclosure, GPS timing

Software:
- Detection engine: multi-channel color analysis + temporal derivative
  tracking on horizon ROI, no fixed brightness threshold
- Capture scheduler: computes twilight windows from coordinates,
  captures every 10s during window, runs detection, uploads results
- Solar position via PyEphem for depression angle at each frame
- Upload sync: hourly cron to central server + CSV export for offline

Provisioning:
- first-boot.sh: one-shot setup script installs all deps, configures
  systemd service, sets up WiFi from station.yaml on boot partition
- Flash SD card, edit station.yaml, plug in, forget

Docs:
- Volunteer host guide: 10-minute installation, plug-and-play
- Hardware BOM with sourcing links
This commit is contained in:
Aric Camarata 2026-03-23 05:58:38 -04:00
parent 66389d1e29
commit b62a361c9a
16 changed files with 1761 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

123
.github/docs/hardware/BOM.md vendored Normal file
View file

@ -0,0 +1,123 @@
# Bill of Materials
Three build tiers depending on budget and deployment needs.
## Tier 1: Budget Build ($270)
Best for WiFi-accessible sites, pilot testing.
| # | Component | Model | Qty | Unit Cost | Source |
|---|---|---|---|---|---|
| 1 | Computer | Raspberry Pi Zero 2W | 1 | $15 | Adafruit, PiShop |
| 2 | Camera | Raspberry Pi HQ Camera (IMX477) | 1 | $50 | Raspberry Pi official |
| 3 | Lens | 1.8mm C-mount fisheye 180 FOV f/2.0 | 1 | $35 | eBay (generic Chinese) |
| 4 | CS-to-C adapter | CS to C mount ring | 1 | $5 | Amazon |
| 5 | SD card | 32GB Class 10 A2 | 1 | $8 | Amazon |
| 6 | Solar panel | 20W monocrystalline 12V | 1 | $25 | Amazon |
| 7 | Battery | 20Ah 12V LiFePO4 | 1 | $45 | Amazon |
| 8 | Charge controller | 10A PWM with LVD | 1 | $15 | Amazon |
| 9 | Buck converter | 12V to 5V/3A USB-C | 1 | $8 | Amazon |
| 10 | Enclosure | 4" PVC cap + 6" acrylic dome | 1 | $30 | Hardware store + eBay |
| 11 | Desiccant | Silica gel packets (reusable) | 4 | $5 | Amazon |
| 12 | Sealant | Clear silicone (GE Silicone II) | 1 | $8 | Hardware store |
| 13 | Cables | Micro-USB, CSI ribbon 300mm | 1 | $6 | Amazon |
| 14 | Mounting | L-bracket + U-bolts (pole mount) | 1 | $15 | Hardware store |
| | **Total** | | | **~$270** | |
Limitation: Pi Zero 2W has 512MB RAM. On-device detection works but is slower.
## Tier 2: Recommended Build ($465)
Best for most deployments. Balanced performance and cost.
| # | Component | Model | Qty | Unit Cost | Source |
|---|---|---|---|---|---|
| 1 | Computer | Raspberry Pi 4B (2GB RAM) | 1 | $45 | Adafruit, PiShop |
| 2 | Camera | ZWO ASI224MC | 1 | $180 | ZWO direct, eBay |
| 3 | Lens | 1.8mm C-mount fisheye 180 FOV f/2.0 | 1 | $35 | eBay |
| 4 | GPS module | U-blox NEO-6M with antenna | 1 | $10 | Amazon |
| 5 | SD card | 64GB Class 10 A2 | 1 | $10 | Amazon |
| 6 | Solar panel | 30W monocrystalline 12V | 1 | $35 | Amazon |
| 7 | Battery | 30Ah 12V LiFePO4 | 1 | $65 | Amazon |
| 8 | Charge controller | 10A PWM with LVD | 1 | $15 | Amazon |
| 9 | Buck converter | 12V to 5V/5A USB-C (PD) | 1 | $12 | Amazon |
| 10 | Enclosure | 6" PVC cap + 9" acrylic dome | 1 | $45 | Hardware store + eBay |
| 11 | Dew heater | Nichrome wire strip (5W) | 1 | $8 | Amazon |
| 12 | Desiccant | Silica gel packets (reusable) | 6 | $5 | Amazon |
| 13 | Sealant | Clear silicone | 1 | $8 | Hardware store |
| 14 | Cables | USB-C, USB 3.0 (camera), GPS UART | 1 | $12 | Amazon |
| 15 | Mounting | L-bracket + U-bolts (pole mount) | 1 | $15 | Hardware store |
| | **Total** | | | **~$505** | |
The ZWO ASI224MC has 0.8e read noise at Gain 60. It resolves faint sky gradients that are invisible to consumer cameras. 1000-second maximum exposure covers the full twilight range.
## Tier 2b: Recommended + Cellular ($590)
Same as Tier 2 plus cellular connectivity for remote dark-sky sites.
| # | Component | Model | Qty | Unit Cost | Source |
|---|---|---|---|---|---|
| | All items from Tier 2 | | | $505 | |
| 16 | 4G HAT | Waveshare SIM7600G-H (includes GPS) | 1 | $83 | Waveshare |
| 17 | SIM card | Hologram Global IoT SIM | 1 | $3 | Hologram.io |
| | **Total** | | | **~$590** | |
The Waveshare 4G HAT includes built-in GNSS (GPS/BeiDou/GLONASS), so the separate GPS module can be dropped ($10 savings). Monthly data cost: ~$2-5 depending on upload frequency.
## Tier 3: Research Grade ($1,000+)
For primary anchor stations where precision is the top priority.
| # | Component | Model | Qty | Unit Cost | Source |
|---|---|---|---|---|---|
| 1 | Computer | Raspberry Pi 5 (4GB) | 1 | $60 | Raspberry Pi official |
| 2 | Camera | ZWO ASI462MC (IMX462 Starvis 2) | 1 | $270 | ZWO direct |
| 3 | Lens | Fujinon 2.7mm f/1.8 fisheye | 1 | $250 | eBay (used) |
| 4 | 4G + GPS | Waveshare SIM7600G-H | 1 | $83 | Waveshare |
| 5 | SD card | 128GB Class 10 A2 | 1 | $15 | Amazon |
| 6 | Solar panel | 40W monocrystalline 12V | 1 | $45 | Amazon |
| 7 | Battery | 40Ah 12V LiFePO4 | 1 | $80 | Amazon |
| 8 | Charge controller | 20A MPPT | 1 | $35 | Amazon |
| 9 | Buck converter | 12V to 5V/5A USB-C PD | 1 | $12 | Amazon |
| 10 | Enclosure | IP67 rated + 9" UV-stabilized dome | 1 | $100 | Pelican + eBay |
| 11 | Dew heater | 10W nichrome strip with PWM controller | 1 | $15 | Amazon |
| 12 | Desiccant + hygro | Silica gel + BME280 humidity sensor | 1 | $12 | Amazon |
| 13 | Cables + misc | | | $25 | |
| | **Total** | | | **~$1,000** | |
## Bulk Order for 20 Units (Tier 2)
| Item | 20x Unit Cost | 20x Total |
|---|---|---|
| Pi 4B 2GB | $45 | $900 |
| ZWO ASI224MC | $180 | $3,600 |
| Lens + GPS + SD + cables | $67 | $1,340 |
| Solar + battery + controller + buck | $127 | $2,540 |
| Enclosure + dew heater + sealant + mount | $76 | $1,520 |
| **Grand total (20 units)** | | **$9,900** |
At 20 units, you may get volume pricing on the ZWO cameras (contact ZWO directly) and solar panels. Realistic total: $8,000-10,000 for 20 complete stations.
## Recommended Starter Kit
For someone who wants to buy a ready-made kit and just flash an SD card:
**CanaKit Raspberry Pi 4 Starter Kit** ($90-120)
- Includes: Pi 4B 4GB, 32GB SD card, case, power supply, HDMI cable
- You still need: camera, lens, outdoor enclosure, solar power, GPS
- The CanaKit case and power supply are for indoor bench testing only. The outdoor deployment uses the solar power system and weatherproof enclosure.
Then add:
- ZWO ASI224MC ($180)
- 1.8mm fisheye lens ($35)
- U-blox GPS ($10)
- Outdoor power + enclosure (build from Tier 2 BOM)
## Tools Needed for Assembly
- Phillips screwdriver
- Wire strippers
- Soldering iron (for dew heater nichrome wire only)
- Drill with 1/2" bit (for cable pass-through in enclosure)
- Silicone caulk gun
- Multimeter (for verifying solar voltage)

131
.github/docs/hosting/HOST-GUIDE.md vendored Normal file
View file

@ -0,0 +1,131 @@
# Volunteer Host Guide
Thank you for helping calibrate Islamic prayer times with real observations. This guide covers everything you need to host a fajr-watch station.
## What You Receive
A complete, pre-configured package:
1. Raspberry Pi 4B computer (pre-flashed SD card)
2. ZWO ASI224MC astronomical camera with fisheye lens
3. Weatherproof dome enclosure (assembled)
4. Solar panel + battery pack + cables
5. Pole mounting bracket
6. This guide
Everything is plug-and-play. You provide a mounting location and WiFi.
## What You Need
### Required
- **A clear view of the eastern horizon** (for Fajr/dawn detection). The camera needs to see the sky from roughly 0 to 30 degrees elevation in the east. Trees, buildings, or hills that block the low eastern sky will reduce data quality.
- **WiFi within range** (or you can run an ethernet cable). The station uploads small data files (a few KB per night). If you have no WiFi at the mounting location, let us know and we can provide a cellular-equipped unit.
- **A mounting point** at least 2 meters (6 feet) above ground level. A fence post, roof edge, deck railing, or pole works. The station comes with a universal bracket.
### Nice to Have (Not Required)
- A clear western horizon too (for Isha/dusk detection). If you only have one clear direction, that's fine. One-direction data is still valuable.
- A dark sky. Urban and suburban sites produce useful data too. The light pollution offset is itself a measurement we need.
- A flat horizon (ocean, lake, prairie). Horizon obstructions below ~3 degrees elevation are common and acceptable.
## Site Selection
### Best Locations
1. **Rooftop with open eastern sky.** Flat commercial roofs are ideal. Residential roofs work if you have a safe, permanent mounting point.
2. **Lake or ocean shoreline.** Water gives the flattest possible horizon. Conneaut OH on Lake Erie, Florida coasts, etc.
3. **Open field or farmland.** Rural properties with no eastern obstructions.
4. **Mosque rooftop or minaret.** Many mosques have flat roofs with clear sky views. The imam may be interested in supporting prayer time research.
5. **Backyard on a tall pole.** A 10-foot pole (fence post, antenna mast) in the yard clears most fences and shrubs.
### Avoid
- Under tree canopy or heavy vegetation
- Inside a building (windows block UV and distort brightness)
- Next to bright security lights or street lamps that face the camera
- Locations with frequent vibration (on top of HVAC units, etc.)
## Installation (10 minutes)
### Step 1: Mount the enclosure
Attach the mounting bracket to your chosen location using the included hardware. The dome should face UP (skyward). The cable exits from the bottom.
The camera inside is pre-aimed. For an all-sky (fisheye) setup, orientation does not matter as long as the dome faces straight up. For a horizon-pointed setup, aim the cable exit toward the south (so the camera looks north-east).
### Step 2: Connect the solar panel
Place the solar panel where it gets direct sunlight for at least 4-5 hours per day. South-facing is ideal in the Northern Hemisphere. Connect the panel to the charge controller (pre-wired in the battery box). Connect the battery box to the enclosure via the USB-C cable.
If you have outdoor AC power nearby, you can skip the solar panel entirely and plug the Pi into a weatherproof USB-C adapter.
### Step 3: Configure WiFi
Before powering on, insert the SD card into a computer and edit the file `fajr-watch/station.yaml` on the boot partition:
```yaml
network:
wifi_ssid: "YourNetworkName"
wifi_password: "YourPassword"
```
Save and eject. Insert the SD card into the Pi.
Your station ID, coordinates, and camera settings are pre-configured.
### Step 4: Power on
Connect power. The status LED will:
- Blink rapidly (booting, ~30 seconds)
- Solid green (connected and healthy)
- Blink amber (capturing twilight frames)
- Solid red (error, see troubleshooting)
That's it. The station runs itself from here.
## What Happens Each Night
1. **Evening:** Station wakes up ~30 minutes before sunset. Captures one frame every 10 seconds through dusk until full darkness. Detects the moment Shafaq al-Abyad (white twilight glow) disappears. Records the solar depression angle.
2. **Night:** Station sleeps (conserves power).
3. **Morning:** Station wakes up ~2 hours before sunrise. Captures frames through dawn. Detects the moment of Fajr Sadiq (first visible white light on the eastern horizon). Records the solar depression angle.
4. **Upload:** Once per hour, the station uploads its results to our central server. Each result is ~2 KB. A month of data uses less than 1 MB of bandwidth.
## Data and Privacy
- The station does NOT capture identifiable images of people or property. The camera points at the sky.
- Raw sky images are processed on-device and deleted. Only numerical brightness measurements and computed angles are uploaded.
- Your name and location are used only for data attribution and station health monitoring. They are not shared publicly without your consent.
- You can opt out at any time. Unplug the station and mail it back (we cover return shipping).
## Troubleshooting
**Status LED is red:**
- Check that the SD card is fully inserted
- Check that the USB-C power cable is connected
- Try unplugging and re-plugging power (30-second wait between)
- If the LED stays red after 3 reboots, contact us
**No data uploading (check at fajr.watch/stations):**
- Verify WiFi credentials in station.yaml
- Move the station closer to the WiFi router
- Check that your router allows new devices
**Camera not detected:**
- Ensure the USB cable (ZWO) or ribbon cable (Pi camera) is firmly seated
- Try a different USB port on the Pi
**Solar panel not charging:**
- Panel must face the sun with no shadows
- Check charge controller LED (should show charging during daylight)
- In winter at high latitudes, you may need AC power backup
## Contact
Questions, problems, or want to adjust your station setup:
- Email: fajr-watch@acamarata.com
- GitHub: https://github.com/acamarata/fajr-watch/issues

48
.gitignore vendored Normal file
View file

@ -0,0 +1,48 @@
# Python
__pycache__/
*.py[cod]
*.egg-info/
.eggs/
dist/
build/
.venv/
venv/
*.log
# OS image build artifacts
build/
*.img
*.img.xz
*.img.gz
# Captured data (too large for git)
data/captures/
data/processed/
data/upload-queue/
# System
.DS_Store
*.swp
*.swo
nohup.out
# IDE
.vscode/
.idea/
# AI agent working directories
.claude/
.codex/
.cursor/
.aider/
.continue/
.windsurf/
.gemini/
.codeium/
# Secrets
.env
.env.*
config/station.yaml
.vscode/*
.aider.chat.history.md

171
README.md Normal file
View file

@ -0,0 +1,171 @@
# fajr-watch
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
A turnkey Raspberry Pi appliance that observes dawn and dusk, measures the exact solar depression angle at the moment of true Fajr and Isha, and uploads the data for prayer time calibration research.
Flash an SD card, connect a camera, plug in power. The station runs unattended, captures the twilight horizon every 10 seconds, detects when dawn begins and dusk ends using multi-channel brightness analysis, computes the solar depression angle at that moment, and uploads the result.
## Why
Every Islamic prayer time algorithm uses a fixed depression angle (15, 18, or 20 degrees depending on convention). Nobody has measured whether those numbers are actually correct across different latitudes, seasons, and atmospheric conditions. This project collects that data.
The output feeds [pray-calc-ml](https://github.com/acamarata/pray-calc-ml), which trains the Dynamic Prayer Calculation algorithm used by [pray-calc](https://github.com/acamarata/pray-calc).
## Hardware
| Component | Model | Cost |
|---|---|---|
| Computer | Raspberry Pi 4B (2GB+) | $45 |
| Camera | ZWO ASI224MC (or Pi HQ Camera) | $50-180 |
| Lens | 1.8mm C-mount fisheye 180 FOV | $35 |
| Power | 30W solar + 30Ah LiFePO4 + controller | $120 |
| Enclosure | PVC housing + 9" acrylic dome | $60 |
| GPS | U-blox NEO-6M (for precise timestamps) | $10 |
Total: ~$320-465 per station. Full bill of materials and assembly guide in [docs/hardware](https://github.com/acamarata/fajr-watch/wiki/Hardware).
## Quick Start
### 1. Flash the SD card
Download the latest fajr-watch image from [Releases](https://github.com/acamarata/fajr-watch/releases), or build your own:
```bash
# On any Linux/macOS machine
git clone https://github.com/acamarata/fajr-watch.git
cd fajr-watch
./scripts/provision/build-image.sh
```
Flash the resulting `.img` to a 32GB+ SD card using [Raspberry Pi Imager](https://www.raspberrypi.com/software/) or `dd`.
### 2. Configure your station
Before first boot, edit `config/station.yaml` on the SD card's boot partition:
```yaml
station:
id: "conneaut-oh-01"
lat: 41.95
lng: -80.55
elevation_m: 175
horizon: "lake" # lake, ocean, flat, hills, mountain
environment: "suburban" # dark, rural, suburban, urban
host: "Aric Camarata"
contact: "email@example.com"
camera:
type: "zwo_asi224" # zwo_asi224, zwo_asi462, pi_hq, pi_cam3
lens_mm: 1.8
orientation: "east" # east, west, allsky
network:
wifi_ssid: "YourNetwork"
wifi_password: "YourPassword"
upload_url: "https://api.fajr.watch/upload"
capture:
interval_s: 10 # seconds between frames during twilight
twilight_margin_deg: 5 # start capturing at sun depression + this margin
raw_format: true # capture RAW (recommended) or JPEG
```
### 3. Boot and forget
Insert the SD card, connect the camera, apply power. The station:
1. Connects to WiFi
2. Syncs time via GPS (or NTP fallback)
3. Computes tonight's twilight windows from its coordinates
4. Sleeps until the twilight window begins
5. Captures frames every 10 seconds during the window
6. Runs the detection algorithm on the captured sequence
7. Uploads the result: `(date, lat, lng, elevation, depression_angle, confidence, metadata)`
8. Sleeps until the next twilight window
Status LED blinks green when healthy, amber when capturing, red on error.
## How Detection Works
The station does not use a fixed brightness threshold. It detects the physical signature of Fajr Sadiq (true dawn):
1. **Horizon ROI extraction.** Isolates the eastern horizon band (azimuth centered on true east, elevation -2 to +15 degrees).
2. **Multi-channel tracking.** Measures R, G, B brightness separately in the horizon ROI every 10 seconds.
3. **Color ratio analysis.** Computes the color index `(R-B)/(R+B)` over time. Before dawn, scattered zodiacal light is warm (positive ratio). True dawn produces a neutral white band (ratio approaches zero).
4. **Temporal derivative.** The rate of brightness change `dB/dt` in the east ROI peaks at a specific moment during the twilight transition. The inflection point marks the onset of sustained brightening.
5. **Solar depression lookup.** At the detected moment, computes the exact solar depression angle using PyEphem with the station's GPS coordinates and UTC timestamp.
6. **Confidence scoring.** Rejects nights with clouds (detected via spatial variance in the ROI), moon interference, or insufficient data.
The same process runs in reverse for Isha (western horizon, brightness decreasing, white glow disappearing).
## Output Format
Each twilight event produces one record:
```json
{
"station_id": "conneaut-oh-01",
"date": "2026-06-21",
"prayer": "fajr",
"utc_time": "2026-06-21T09:12:34Z",
"solar_depression_deg": 14.23,
"confidence": 0.92,
"lat": 41.95,
"lng": -80.55,
"elevation_m": 175,
"environment": "suburban",
"horizon": "lake",
"camera": "zwo_asi224",
"sky_quality_mpsas": 19.4,
"moon_alt_deg": -12.3,
"cloud_score": 0.08,
"color_index_at_detection": 0.02,
"brightness_curve_hash": "sha256:..."
}
```
## Project Structure
```
fajr-watch/
├── src/
│ ├── capture/ # Camera control, frame acquisition
│ ├── detect/ # Dawn/dusk detection algorithm
│ ├── upload/ # Data upload to pray-calc-ml
│ └── calibrate/ # Flat-field, dark frame, photometric calibration
├── config/
│ └── station.example.yaml
├── scripts/
│ └── provision/ # OS image build, first-boot setup
├── .github/
│ ├── docs/
│ │ ├── hardware/ # BOM, assembly, wiring diagrams
│ │ └── hosting/ # Volunteer host guide, site selection
│ └── workflows/ # CI
├── .gitignore
├── README.md
└── LICENSE
```
## Contributing Data
Want to host a station? We provide the hardware (camera + Pi + enclosure + solar panel) and you provide a mounting location with a clear eastern or western horizon. Dark sky sites are ideal, but suburban and urban sites are also valuable for measuring the light pollution offset.
See the [Host Guide](https://github.com/acamarata/fajr-watch/wiki/Host-Guide) for requirements and how to sign up.
## Related Projects
- [pray-calc](https://github.com/acamarata/pray-calc) - Islamic prayer time calculator (npm)
- [pray-calc-ml](https://github.com/acamarata/pray-calc-ml) - ML dataset for prayer angle calibration
- [nrel-spa](https://github.com/acamarata/nrel-spa) - Solar position algorithm
- [moon-sighting](https://github.com/acamarata/moon-sighting) - Lunar crescent visibility
## License
MIT

97
config/station.example.yaml Normal file
View file

@ -0,0 +1,97 @@
# fajr-watch station configuration
# Copy this file to station.yaml and fill in your values.
# On the Pi SD card, place station.yaml in /boot/fajr-watch/station.yaml
station:
# Unique station identifier (lowercase, hyphens ok)
id: "my-station-01"
# GPS coordinates (decimal degrees)
# South latitudes and west longitudes are negative
lat: 41.95
lng: -80.55
# Elevation above sea level in metres
elevation_m: 175
# Horizon type at camera location
# Options: ocean, lake, flat, hills, mountain
horizon: "lake"
# Light pollution environment
# Options: dark (Bortle 1-2), rural (3-4), suburban (5-6), urban (7-9)
environment: "suburban"
# Your name (for data attribution)
host: "Your Name"
# Contact email (not published, used for station health alerts)
contact: "you@example.com"
camera:
# Camera model connected to the Pi
# Options: zwo_asi224, zwo_asi462, zwo_asi662, pi_hq, pi_cam3, pi_cam3_wide
type: "zwo_asi224"
# Lens focal length in mm (for FOV calculation)
lens_mm: 1.8
# Which horizon the camera points at
# Options: east (Fajr only), west (Isha only), allsky (both)
orientation: "allsky"
# Azimuth offset in degrees (0 = camera north is true north)
# Measured clockwise. If your camera's "up" points 30 degrees east of north,
# set this to 30.
azimuth_offset: 0
network:
# WiFi credentials (WPA2)
wifi_ssid: ""
wifi_password: ""
# Optional: additional WiFi networks (tried in order)
# wifi_networks:
# - ssid: "BackupNetwork"
# password: "password2"
# Data upload endpoint
upload_url: "https://api.fajr.watch/v1/upload"
# Upload API key (provided when you register your station)
api_key: ""
capture:
# Seconds between frames during twilight window
interval_s: 10
# How many degrees before/after the expected twilight to start/stop capturing
# Larger margin = more data but more storage. 5 degrees is ~20-30 min extra.
twilight_margin_deg: 5
# Capture RAW images (recommended for scientific quality)
# Set to false for JPEG-only (smaller files, less accurate)
raw_format: true
# Maximum exposure time in seconds
# ZWO cameras support up to 1000s. Pi cameras typically max at 30-60s.
max_exposure_s: 30
# Target mean brightness for auto-exposure (0-255 for 8-bit)
# Lower = darker frames that preserve faint gradients
target_brightness: 80
processing:
# Run detection on-device (true) or upload raw frames for server processing (false)
detect_on_device: true
# Minimum confidence score to accept a detection (0.0 - 1.0)
min_confidence: 0.5
# Keep raw frames after processing
# true = keep all frames (uses more storage, useful for debugging)
# false = keep only the 10 frames around each detection event
keep_raw_frames: false
# Maximum days of local data to retain before cleanup
retention_days: 30

7
requirements.txt Normal file
View file

@ -0,0 +1,7 @@
ephem>=4.1
numpy>=1.25
opencv-python-headless>=4.8
scipy>=1.11
Pillow>=10.0
pyyaml>=6.0
requests>=2.31

190
scripts/provision/first-boot.sh Executable file
View file

@ -0,0 +1,190 @@
#!/bin/bash
# fajr-watch first-boot provisioning script
#
# This runs once on first boot of a fresh Raspberry Pi OS image.
# It installs all dependencies, configures the system, and enables
# the fajr-watch service.
#
# Usage: placed in /boot/fajr-watch/first-boot.sh by the image builder,
# triggered by a systemd oneshot service on first boot.
set -euo pipefail
LOG="/var/log/fajr-watch-provision.log"
exec &> >(tee -a "$LOG")
echo "=== fajr-watch first-boot provisioning ==="
echo "Date: $(date -u)"
# ── System setup ──
echo "[1/8] Updating system packages..."
apt-get update -qq
apt-get upgrade -y -qq
echo "[2/8] Installing system dependencies..."
apt-get install -y -qq \
python3-pip python3-venv python3-numpy python3-opencv \
python3-yaml python3-scipy python3-astropy \
gpsd gpsd-clients libgps-dev \
libusb-1.0-0-dev libudev-dev \
ntp ntpdate \
jq curl
# ── GPS setup ──
echo "[3/8] Configuring GPS..."
if [ -e /dev/ttyUSB0 ] || [ -e /dev/ttyACM0 ]; then
systemctl enable gpsd
systemctl start gpsd
echo "GPS device detected"
else
echo "No GPS device found. Using NTP for time sync."
fi
# ── Python environment ──
echo "[4/8] Setting up Python environment..."
VENV="/opt/fajr-watch/venv"
python3 -m venv --system-site-packages "$VENV"
source "$VENV/bin/activate"
pip install --quiet \
ephem \
pyyaml \
requests \
Pillow
# Install ZWO ASI SDK if a ZWO camera is configured
STATION_CONFIG="/boot/fajr-watch/station.yaml"
if [ -f "$STATION_CONFIG" ]; then
CAMERA_TYPE=$(python3 -c "
import yaml
with open('$STATION_CONFIG') as f:
c = yaml.safe_load(f)
print(c.get('camera', {}).get('type', ''))
")
if [[ "$CAMERA_TYPE" == zwo_* ]]; then
echo "ZWO camera configured. Installing ASI SDK..."
pip install --quiet zwoasi
# Download ZWO SDK library
if [ ! -f /usr/lib/libASICamera2.so ]; then
ARCH=$(uname -m)
if [ "$ARCH" = "aarch64" ]; then
SDK_URL="https://github.com/stevemarple/python-zwoasi/raw/master/lib/armv8/libASICamera2.so"
else
SDK_URL="https://github.com/stevemarple/python-zwoasi/raw/master/lib/armv7/libASICamera2.so"
fi
curl -sL "$SDK_URL" -o /usr/lib/libASICamera2.so
chmod 644 /usr/lib/libASICamera2.so
ldconfig
fi
fi
if [[ "$CAMERA_TYPE" == pi_* ]]; then
echo "Pi camera configured. Installing picamera2..."
pip install --quiet picamera2
fi
fi
# ── Install fajr-watch ──
echo "[5/8] Installing fajr-watch software..."
INSTALL_DIR="/opt/fajr-watch/app"
if [ -d /boot/fajr-watch/src ]; then
cp -r /boot/fajr-watch/src "$INSTALL_DIR/src"
cp -r /boot/fajr-watch/config "$INSTALL_DIR/config"
else
# Clone from GitHub if not bundled on the SD card
git clone --depth 1 https://github.com/acamarata/fajr-watch.git "$INSTALL_DIR"
fi
# ── Data directories ──
echo "[6/8] Creating data directories..."
mkdir -p /var/lib/fajr-watch/data/{captures,results,upload-queue}
chown -R pi:pi /var/lib/fajr-watch
# ── Copy station config ──
if [ -f "$STATION_CONFIG" ]; then
cp "$STATION_CONFIG" "$INSTALL_DIR/config/station.yaml"
echo "Station config loaded from boot partition"
fi
# ── WiFi setup from station config ──
if [ -f "$STATION_CONFIG" ]; then
WIFI_SSID=$(python3 -c "
import yaml
with open('$STATION_CONFIG') as f:
c = yaml.safe_load(f)
print(c.get('network', {}).get('wifi_ssid', ''))
")
WIFI_PASS=$(python3 -c "
import yaml
with open('$STATION_CONFIG') as f:
c = yaml.safe_load(f)
print(c.get('network', {}).get('wifi_password', ''))
")
if [ -n "$WIFI_SSID" ] && [ -n "$WIFI_PASS" ]; then
echo "[6b] Configuring WiFi: $WIFI_SSID"
nmcli device wifi connect "$WIFI_SSID" password "$WIFI_PASS" || true
fi
fi
# ── Systemd service ──
echo "[7/8] Installing systemd service..."
cat > /etc/systemd/system/fajr-watch.service << 'UNIT'
[Unit]
Description=fajr-watch twilight observation station
After=network-online.target gpsd.service
Wants=network-online.target
[Service]
Type=simple
User=pi
Group=pi
Environment=PYTHONPATH=/opt/fajr-watch/app
ExecStart=/opt/fajr-watch/venv/bin/python -m src.capture.scheduler
WorkingDirectory=/opt/fajr-watch/app
Restart=always
RestartSec=60
StandardOutput=append:/var/log/fajr-watch.log
StandardError=append:/var/log/fajr-watch.log
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable fajr-watch.service
systemctl start fajr-watch.service
# ── Upload cron ──
echo "[8/8] Setting up data upload cron..."
cat > /etc/cron.d/fajr-watch-upload << 'CRON'
# Upload completed detection results every hour
0 * * * * pi /opt/fajr-watch/venv/bin/python -m src.upload.sync 2>> /var/log/fajr-watch-upload.log
CRON
# ── Disable first-boot trigger ──
rm -f /boot/fajr-watch/first-boot-trigger
systemctl disable fajr-watch-provision.service 2>/dev/null || true
echo ""
echo "=== fajr-watch provisioning complete ==="
echo "Station ID: $(python3 -c "
import yaml
with open('$STATION_CONFIG') as f:
c = yaml.safe_load(f)
print(c.get('station', {}).get('id', 'unknown'))
" 2>/dev/null || echo 'unknown')"
echo "Service status:"
systemctl status fajr-watch.service --no-pager || true
echo ""
echo "Logs: journalctl -u fajr-watch -f"
echo "Data: /var/lib/fajr-watch/data/results/"

0
src/__init__.py Normal file
View file

0
src/calibrate/__init__.py Normal file
View file

0
src/capture/__init__.py Normal file
View file

302
src/capture/scheduler.py Normal file
View file

@ -0,0 +1,302 @@
"""
Capture scheduler for fajr-watch.
Runs the main loop: sleep until twilight, capture frames, run detection,
upload results, repeat.
"""
import json
import logging
import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
from ..detect.solar import twilight_window, solar_depression
from ..detect.twilight import (
TwilightEvent,
FrameData,
detect_fajr,
detect_isha,
extract_roi_data,
)
log = logging.getLogger(__name__)
DATA_DIR = Path("/var/lib/fajr-watch/data")
RESULTS_DIR = DATA_DIR / "results"
CAPTURES_DIR = DATA_DIR / "captures"
def load_config(path: str = "/boot/fajr-watch/station.yaml") -> dict:
"""Load station configuration from YAML."""
import yaml
with open(path) as f:
return yaml.safe_load(f)
def capture_frame(camera, config: dict) -> tuple:
"""
Capture a single frame from the camera.
Returns (image_array, utc_timestamp).
"""
camera_type = config["camera"]["type"]
if camera_type.startswith("zwo_"):
return _capture_zwo(camera, config)
elif camera_type.startswith("pi_"):
return _capture_picamera(camera, config)
else:
raise ValueError(f"Unknown camera type: {camera_type}")
def _capture_zwo(camera, config: dict) -> tuple:
"""Capture from ZWO ASI camera via the ZWO SDK."""
import zwoasi
utc_now = datetime.now(timezone.utc)
# Auto-exposure: adjust to target brightness
target = config["capture"].get("target_brightness", 80)
max_exp = config["capture"].get("max_exposure_s", 30) * 1_000_000 # microseconds
camera.set_control_value(zwoasi.ASI_EXPOSURE, min(int(max_exp), 30_000_000))
camera.set_control_value(zwoasi.ASI_GAIN, 200)
image = camera.capture_video_frame()
return image, utc_now
def _capture_picamera(camera, config: dict) -> tuple:
"""Capture from Raspberry Pi camera via picamera2."""
import numpy as np
utc_now = datetime.now(timezone.utc)
# camera is a Picamera2 instance, already configured
image = camera.capture_array("main")
return image, utc_now
def init_camera(config: dict):
"""Initialize the camera based on config."""
camera_type = config["camera"]["type"]
if camera_type.startswith("zwo_"):
import zwoasi
zwoasi.init("/usr/lib/libASICamera2.so")
cameras = zwoasi.list_cameras()
if not cameras:
raise RuntimeError("No ZWO camera detected")
camera = zwoasi.Camera(0)
camera.set_control_value(zwoasi.ASI_BANDWIDTHOVERLOAD, 40)
camera.set_image_type(zwoasi.ASI_IMG_RGB24)
camera.start_video_capture()
log.info("ZWO camera initialized: %s", cameras[0])
return camera
elif camera_type.startswith("pi_"):
from picamera2 import Picamera2
camera = Picamera2()
raw_mode = config["capture"].get("raw_format", True)
if raw_mode:
cam_config = camera.create_still_configuration(
main={"format": "RGB888"},
raw={"format": camera.sensor_modes[0]["format"]},
)
else:
cam_config = camera.create_still_configuration(
main={"format": "RGB888"},
)
camera.configure(cam_config)
camera.start()
log.info("Pi camera initialized")
return camera
else:
raise ValueError(f"Unknown camera type: {camera_type}")
def save_event(event: TwilightEvent, config: dict):
"""Save a detection event to disk as JSON."""
RESULTS_DIR.mkdir(parents=True, exist_ok=True)
station = config["station"]
filename = f"{event.prayer}_{event.utc_time.strftime('%Y-%m-%d_%H%M%S')}.json"
record = {
"station_id": station["id"],
"date": event.utc_time.strftime("%Y-%m-%d"),
"prayer": event.prayer,
"utc_time": event.utc_time.isoformat(),
"solar_depression_deg": round(event.solar_depression_deg, 4),
"confidence": event.confidence,
"lat": station["lat"],
"lng": station["lng"],
"elevation_m": station["elevation_m"],
"environment": station.get("environment", "unknown"),
"horizon": station.get("horizon", "unknown"),
"camera": config["camera"]["type"],
"sky_quality_mpsas": event.sky_quality_mpsas,
"moon_alt_deg": event.moon_alt_deg,
"cloud_score": event.cloud_score,
"color_index_at_detection": event.color_index,
"n_frames": event.n_frames,
}
path = RESULTS_DIR / filename
with open(path, "w") as f:
json.dump(record, f, indent=2)
log.info("Saved %s detection: %.2f deg (confidence %.2f) -> %s",
event.prayer, event.solar_depression_deg, event.confidence, path)
return path
def run_capture_session(
camera,
config: dict,
window_start: datetime,
window_end: datetime,
prayer: str,
) -> list[FrameData]:
"""
Capture frames during a twilight window.
Returns list of FrameData for detection.
"""
station = config["station"]
interval = config["capture"].get("interval_s", 10)
lens_fov = config["camera"].get("lens_fov_deg", 180.0)
az_offset = config["camera"].get("azimuth_offset", 0.0)
frames = []
log.info("Starting %s capture session: %s to %s (interval %ds)",
prayer, window_start.isoformat(), window_end.isoformat(), interval)
# Wait until window starts
now = datetime.now(timezone.utc)
if now < window_start:
wait = (window_start - now).total_seconds()
log.info("Sleeping %.0f seconds until %s window", wait, prayer)
time.sleep(wait)
while datetime.now(timezone.utc) < window_end:
try:
image, utc_time = capture_frame(camera, config)
frame = extract_roi_data(
image, utc_time,
station["lat"], station["lng"], station["elevation_m"],
lens_fov, az_offset,
)
frames.append(frame)
dep = frame.solar_dep
log.debug("Frame %s: dep=%.2f, east_mean=%.1f",
utc_time.strftime("%H:%M:%S"), dep,
float(frame.east_roi_rgb.mean()))
except Exception as e:
log.warning("Frame capture error: %s", e)
time.sleep(interval)
log.info("Capture session complete: %d frames", len(frames))
return frames
def run_night(config: dict, camera):
"""
Run one complete night: Isha capture + detection, sleep, Fajr capture + detection.
"""
station = config["station"]
lat = station["lat"]
lng = station["lng"]
elev = station["elevation_m"]
margin = config["capture"].get("twilight_margin_deg", 5)
now = datetime.now(timezone.utc)
windows = twilight_window(now, lat, lng, elev, margin)
if windows is None:
log.warning("No twilight tonight (polar conditions). Sleeping 12 hours.")
time.sleep(43200)
return
# Isha session (evening)
if datetime.now(timezone.utc) < windows["isha_end"]:
frames = run_capture_session(
camera, config,
windows["isha_start"], windows["isha_end"],
"isha",
)
if frames:
event = detect_isha(frames, lat, lng)
if event and event.confidence >= config["processing"].get("min_confidence", 0.5):
save_event(event, config)
elif event:
log.info("Isha detection below confidence threshold: %.2f", event.confidence)
else:
log.info("No Isha detection from %d frames", len(frames))
# Sleep until Fajr window
now = datetime.now(timezone.utc)
if now < windows["fajr_start"]:
wait = (windows["fajr_start"] - now).total_seconds()
log.info("Sleeping %.0f seconds until Fajr window", wait)
time.sleep(wait)
# Fajr session (morning)
frames = run_capture_session(
camera, config,
windows["fajr_start"], windows["fajr_end"],
"fajr",
)
if frames:
event = detect_fajr(frames, lat, lng)
if event and event.confidence >= config["processing"].get("min_confidence", 0.5):
save_event(event, config)
elif event:
log.info("Fajr detection below confidence threshold: %.2f", event.confidence)
else:
log.info("No Fajr detection from %d frames", len(frames))
def main():
"""Main entry point. Runs forever."""
import sys
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
handlers=[
logging.StreamHandler(sys.stdout),
logging.FileHandler("/var/log/fajr-watch.log"),
],
)
config_path = "/boot/fajr-watch/station.yaml"
if not Path(config_path).exists():
config_path = "config/station.yaml"
config = load_config(config_path)
log.info("Station %s starting at %.4f, %.4f",
config["station"]["id"],
config["station"]["lat"],
config["station"]["lng"])
camera = init_camera(config)
while True:
try:
run_night(config, camera)
except KeyboardInterrupt:
log.info("Shutdown requested")
break
except Exception as e:
log.error("Night session error: %s", e, exc_info=True)
time.sleep(300) # wait 5 min on error, then retry
if __name__ == "__main__":
main()

0
src/detect/__init__.py Normal file
View file

134
src/detect/solar.py Normal file
View file

@ -0,0 +1,134 @@
"""
Solar position utilities for fajr-watch.
Computes solar depression angle, twilight windows, and sun azimuth
for a given location and time. Uses PyEphem for accuracy.
"""
import math
from datetime import datetime, timedelta, timezone
import ephem
def solar_depression(utc_dt: datetime, lat: float, lng: float, elevation_m: float = 0) -> float:
"""
Solar depression angle in degrees at the given UTC time and location.
Returns positive values when the sun is below the horizon.
Returns negative values when the sun is above the horizon.
"""
obs = ephem.Observer()
obs.lat = str(lat)
obs.lon = str(lng)
obs.elevation = elevation_m
obs.pressure = 1013.25
obs.temp = 15.0
if utc_dt.tzinfo is not None:
utc_dt = utc_dt.replace(tzinfo=None)
obs.date = ephem.Date(utc_dt)
sun = ephem.Sun(obs)
altitude_deg = math.degrees(float(sun.alt))
return -altitude_deg
def sun_azimuth(utc_dt: datetime, lat: float, lng: float, elevation_m: float = 0) -> float:
"""
Sun azimuth in degrees (0=N, 90=E, 180=S, 270=W) at the given UTC time.
"""
obs = ephem.Observer()
obs.lat = str(lat)
obs.lon = str(lng)
obs.elevation = elevation_m
obs.pressure = 1013.25
obs.temp = 15.0
if utc_dt.tzinfo is not None:
utc_dt = utc_dt.replace(tzinfo=None)
obs.date = ephem.Date(utc_dt)
sun = ephem.Sun(obs)
return math.degrees(float(sun.az))
def twilight_window(
date: datetime,
lat: float,
lng: float,
elevation_m: float = 0,
margin_deg: float = 5.0,
) -> dict:
"""
Compute the UTC time windows for Fajr and Isha observation tonight.
Returns a dict with keys:
fajr_start: UTC datetime to begin Fajr capture
fajr_end: UTC datetime to stop Fajr capture
isha_start: UTC datetime to begin Isha capture
isha_end: UTC datetime to stop Isha capture
The window is defined as when solar depression is between
(target_angle + margin) and (target_angle - margin), roughly
22+margin to 7-margin degrees, covering the full possible range.
"""
obs = ephem.Observer()
obs.lat = str(lat)
obs.lon = str(lng)
obs.elevation = elevation_m
obs.pressure = 0 # no refraction for horizon crossing
obs.date = ephem.Date(date.replace(tzinfo=None))
# Find sunset and sunrise to anchor the windows
sun = ephem.Sun()
try:
sunset = obs.next_setting(sun).datetime().replace(tzinfo=timezone.utc)
except ephem.NeverUpError:
# Polar night: no sunset
return None
except ephem.AlwaysUpError:
# Midnight sun: no sunset
return None
try:
obs.date = ephem.Date(sunset + timedelta(hours=1))
sunrise = obs.next_rising(sun).datetime().replace(tzinfo=timezone.utc)
except (ephem.NeverUpError, ephem.AlwaysUpError):
return None
# Isha window: starts at sunset, ends when depression exceeds 22+margin
isha_start = sunset - timedelta(minutes=10)
isha_end = sunset + timedelta(hours=2, minutes=30)
# Fajr window: starts when depression is 22+margin before sunrise
fajr_start = sunrise - timedelta(hours=2, minutes=30)
fajr_end = sunrise + timedelta(minutes=10)
return {
"isha_start": isha_start,
"isha_end": isha_end,
"fajr_start": fajr_start,
"fajr_end": fajr_end,
"sunset": sunset,
"sunrise": sunrise,
}
def moon_altitude(utc_dt: datetime, lat: float, lng: float) -> float:
"""
Moon altitude in degrees at the given time and location.
Used to flag moonlit conditions that may affect detection.
"""
obs = ephem.Observer()
obs.lat = str(lat)
obs.lon = str(lng)
obs.pressure = 1013.25
if utc_dt.tzinfo is not None:
utc_dt = utc_dt.replace(tzinfo=None)
obs.date = ephem.Date(utc_dt)
moon = ephem.Moon(obs)
return math.degrees(float(moon.alt))

416
src/detect/twilight.py Normal file
View file

@ -0,0 +1,416 @@
"""
Twilight detection algorithm for fajr-watch.
Detects the exact moment of Fajr Sadiq (true dawn) and Shafaq al-Abyad
disappearance (Isha) from a sequence of captured horizon images.
The algorithm does NOT use a fixed brightness threshold. It detects the
physical signature of dawn/dusk using multi-channel color analysis and
temporal derivative tracking.
"""
from dataclasses import dataclass
from datetime import datetime
from typing import Optional
import numpy as np
from .solar import solar_depression, sun_azimuth, moon_altitude
@dataclass
class TwilightEvent:
"""Result of a single twilight detection."""
prayer: str # "fajr" or "isha"
utc_time: datetime # UTC time of detected event
solar_depression_deg: float # depression angle at detection moment
confidence: float # 0.0 to 1.0
color_index: float # (R-B)/(R+B) at detection
brightness_east: float # mean brightness of east ROI
brightness_west: float # mean brightness of west ROI
sky_quality_mpsas: Optional[float] # estimated sky quality
moon_alt_deg: float # moon altitude at detection
cloud_score: float # 0.0 = clear, 1.0 = overcast
n_frames: int # frames in the sequence
@dataclass
class FrameData:
"""Extracted data from a single captured frame."""
utc_time: datetime
east_roi_rgb: np.ndarray # shape (3,) mean R, G, B in east ROI
west_roi_rgb: np.ndarray # shape (3,) mean R, G, B in west ROI
ref_roi_rgb: np.ndarray # shape (3,) mean R, G, B in reference ROI
east_roi_std: float # spatial std dev in east ROI (cloud proxy)
solar_dep: float # solar depression at this frame
sun_az: float # sun azimuth at this frame
def extract_roi_data(
image: np.ndarray,
utc_time: datetime,
lat: float,
lng: float,
elevation_m: float,
lens_fov_deg: float = 180.0,
azimuth_offset: float = 0.0,
) -> FrameData:
"""
Extract horizon ROI brightness data from a captured frame.
For an all-sky fisheye image, the ROIs are defined as:
- East ROI: 30-degree azimuth band centered on the sun's rising azimuth,
elevation -2 to +15 degrees
- West ROI: same band centered on the sun's setting azimuth
- Reference ROI: north sky, elevation 30-60 degrees (away from twilight)
For a horizon-pointed camera, the east ROI covers the central 60% of the frame
horizontally and the lower 40% vertically.
"""
h, w = image.shape[:2]
dep = solar_depression(utc_time, lat, lng, elevation_m)
az = sun_azimuth(utc_time, lat, lng, elevation_m)
if lens_fov_deg >= 150:
# All-sky fisheye: map azimuth/elevation to pixel coordinates
east_roi, west_roi, ref_roi = _extract_fisheye_rois(
image, az, azimuth_offset, h, w
)
else:
# Horizon-pointed: simple rectangular ROIs
east_roi, west_roi, ref_roi = _extract_horizon_rois(image, h, w)
east_rgb = np.mean(east_roi, axis=(0, 1)).astype(float)
west_rgb = np.mean(west_roi, axis=(0, 1)).astype(float)
ref_rgb = np.mean(ref_roi, axis=(0, 1)).astype(float)
east_std = float(np.std(east_roi))
return FrameData(
utc_time=utc_time,
east_roi_rgb=east_rgb,
west_roi_rgb=west_rgb,
ref_roi_rgb=ref_rgb,
east_roi_std=east_std,
solar_dep=dep,
sun_az=az,
)
def _extract_fisheye_rois(
image: np.ndarray,
sun_az: float,
az_offset: float,
h: int,
w: int,
) -> tuple:
"""
Extract ROIs from an all-sky fisheye image.
In a standard all-sky image:
- Center of image = zenith
- Edge of image = horizon (radius = min(h,w)/2)
- Azimuth maps to angle from top (north=0, east=90 clockwise)
"""
cx, cy = w // 2, h // 2
radius = min(h, w) // 2
# Sun azimuth (east at dawn, west at dusk)
east_az = sun_az - az_offset
west_az = (east_az + 180) % 360
north_az = 0 - az_offset
# Define ROI masks using polar coordinates
y_grid, x_grid = np.ogrid[:h, :w]
dx = x_grid - cx
dy = y_grid - cy
r = np.sqrt(dx**2 + dy**2)
theta = np.degrees(np.arctan2(dx, -dy)) % 360 # 0=N, 90=E
# Elevation: center=90deg, edge=0deg
elev = 90.0 * (1.0 - r / radius)
# East ROI: azimuth within 30deg of sun, elevation -2 to 15
east_az_dist = np.minimum(
np.abs(theta - east_az),
360 - np.abs(theta - east_az)
)
east_mask = (east_az_dist < 30) & (elev > -2) & (elev < 15) & (r < radius)
# West ROI: opposite side
west_az_dist = np.minimum(
np.abs(theta - west_az),
360 - np.abs(theta - west_az)
)
west_mask = (west_az_dist < 30) & (elev > -2) & (elev < 15) & (r < radius)
# Reference ROI: north, elevation 30-60
north_az_dist = np.minimum(
np.abs(theta - north_az),
360 - np.abs(theta - north_az)
)
ref_mask = (north_az_dist < 45) & (elev > 30) & (elev < 60) & (r < radius)
# Extract pixels (fall back to full image quadrants if masks are empty)
east_pixels = image[east_mask] if east_mask.any() else image[cy:, cx:]
west_pixels = image[west_mask] if west_mask.any() else image[cy:, :cx]
ref_pixels = image[ref_mask] if ref_mask.any() else image[:cy // 2, :]
return east_pixels, west_pixels, ref_pixels
def _extract_horizon_rois(
image: np.ndarray,
h: int,
w: int,
) -> tuple:
"""
Extract ROIs from a horizon-pointed camera (not fisheye).
East ROI: center-right of frame, lower half (horizon band)
West ROI: center-left of frame, lower half
Reference: top quarter of frame (sky above horizon)
"""
mid_w = w // 2
horizon_top = h * 3 // 5 # horizon in lower 40%
east_roi = image[horizon_top:, mid_w:]
west_roi = image[horizon_top:, :mid_w]
ref_roi = image[:h // 4, :]
return east_roi, west_roi, ref_roi
def color_index(rgb: np.ndarray) -> float:
"""
Compute the color index (R-B)/(R+B).
Positive = warm/reddish (scattered light, zodiacal)
Zero = neutral white (Fajr Sadiq signature)
Negative = bluish (post-dawn sky)
Returns 0.0 if both channels are near zero (dark sky).
"""
r, _, b = float(rgb[0]), float(rgb[1]), float(rgb[2])
denom = r + b
if denom < 1.0:
return 0.0
return (r - b) / denom
def detect_fajr(
frames: list[FrameData],
lat: float,
lng: float,
) -> Optional[TwilightEvent]:
"""
Detect Fajr Sadiq from a sequence of morning twilight frames.
Algorithm:
1. Sort frames by time (earliest first)
2. Compute color index time series for east ROI
3. Compute brightness derivative time series for east ROI
4. Find the inflection point where:
a. East ROI brightness begins sustained increase (dB/dt > threshold)
b. Color index transitions from positive toward zero (warm -> white)
c. Solar depression is in the 7-22 degree range
5. Score confidence based on:
- Smoothness of the brightness curve (noisy = clouds)
- Consistency between R, G, B channels
- Moon altitude (moonlit = lower confidence)
"""
if len(frames) < 10:
return None
frames = sorted(frames, key=lambda f: f.utc_time)
# Filter to frames where solar depression is in the relevant range
twilight = [f for f in frames if 5.0 <= f.solar_dep <= 24.0]
if len(twilight) < 5:
return None
# Compute time series
times = np.array([(f.utc_time - twilight[0].utc_time).total_seconds() for f in twilight])
east_brightness = np.array([np.mean(f.east_roi_rgb) for f in twilight])
east_ci = np.array([color_index(f.east_roi_rgb) for f in twilight])
solar_deps = np.array([f.solar_dep for f in twilight])
cloud_scores = np.array([f.east_roi_std for f in twilight])
# Smooth the brightness curve (5-point median)
if len(east_brightness) >= 5:
smoothed = np.convolve(east_brightness, np.ones(5) / 5, mode="same")
else:
smoothed = east_brightness
# Compute temporal derivative
dt = np.diff(times)
dt[dt == 0] = 1 # prevent division by zero
db_dt = np.diff(smoothed) / dt
# Find sustained positive brightness increase
# (at least 3 consecutive frames with increasing brightness)
candidates = []
run_length = 0
for i in range(len(db_dt)):
if db_dt[i] > 0:
run_length += 1
if run_length >= 3:
onset_idx = i - run_length + 1
if 7.0 <= solar_deps[onset_idx] <= 22.0:
candidates.append(onset_idx)
break
else:
run_length = 0
if not candidates:
# Fall back: find the frame with maximum brightness derivative
# in the valid depression range
valid = (solar_deps[:-1] >= 7.0) & (solar_deps[:-1] <= 22.0)
if not valid.any():
return None
valid_db = np.where(valid, db_dt, -np.inf)
onset_idx = int(np.argmax(valid_db))
else:
onset_idx = candidates[0]
# The detection frame
det = twilight[onset_idx]
# Confidence scoring
confidence = 1.0
# Penalize for clouds (high spatial variance)
mean_cloud = float(np.mean(cloud_scores))
if mean_cloud > 50:
confidence *= 0.5
elif mean_cloud > 20:
confidence *= 0.8
# Penalize for moon interference
moon_alt = moon_altitude(det.utc_time, lat, lng)
if moon_alt > 10:
confidence *= 0.7
elif moon_alt > 0:
confidence *= 0.9
# Penalize if color index is far from zero (not white)
ci = color_index(det.east_roi_rgb)
if abs(ci) > 0.3:
confidence *= 0.6
# Penalize for few frames
if len(twilight) < 20:
confidence *= 0.7
return TwilightEvent(
prayer="fajr",
utc_time=det.utc_time,
solar_depression_deg=det.solar_dep,
confidence=round(confidence, 3),
color_index=round(ci, 4),
brightness_east=float(np.mean(det.east_roi_rgb)),
brightness_west=float(np.mean(det.west_roi_rgb)),
sky_quality_mpsas=None, # computed by calibration module
moon_alt_deg=round(moon_alt, 1),
cloud_score=round(mean_cloud / 100, 3),
n_frames=len(twilight),
)
def detect_isha(
frames: list[FrameData],
lat: float,
lng: float,
) -> Optional[TwilightEvent]:
"""
Detect Isha (Shafaq al-Abyad disappearance) from evening twilight frames.
Same algorithm as Fajr but reversed:
- Monitors the WEST ROI instead of east
- Looks for sustained brightness DECREASE (dB/dt < 0)
- Detects when color index of the west horizon returns to zero
(white glow gone, sky fully dark)
"""
if len(frames) < 10:
return None
frames = sorted(frames, key=lambda f: f.utc_time)
twilight = [f for f in frames if 5.0 <= f.solar_dep <= 24.0]
if len(twilight) < 5:
return None
times = np.array([(f.utc_time - twilight[0].utc_time).total_seconds() for f in twilight])
west_brightness = np.array([np.mean(f.west_roi_rgb) for f in twilight])
west_ci = np.array([color_index(f.west_roi_rgb) for f in twilight])
solar_deps = np.array([f.solar_dep for f in twilight])
cloud_scores = np.array([f.east_roi_std for f in twilight])
if len(west_brightness) >= 5:
smoothed = np.convolve(west_brightness, np.ones(5) / 5, mode="same")
else:
smoothed = west_brightness
dt = np.diff(times)
dt[dt == 0] = 1
db_dt = np.diff(smoothed) / dt
# For Isha: find where brightness stabilizes at minimum (dB/dt approaches 0
# from negative) AND the color index has returned to near-zero.
# This marks the moment the white glow is fully gone.
candidates = []
for i in range(3, len(db_dt)):
if solar_deps[i] < 10.0 or solar_deps[i] > 22.0:
continue
# Look for the transition: brightness was decreasing, now stabilized
recent_db = db_dt[max(0, i - 3):i]
if len(recent_db) >= 2 and np.all(recent_db < 0):
# Still decreasing. Check if next frame stabilizes.
if i < len(db_dt) - 1 and abs(db_dt[i]) < abs(np.mean(recent_db)) * 0.3:
candidates.append(i)
break
if not candidates:
# Fall back: find the frame with the most negative derivative
valid = (solar_deps[:-1] >= 10.0) & (solar_deps[:-1] <= 22.0)
if not valid.any():
return None
valid_db = np.where(valid, db_dt, np.inf)
onset_idx = int(np.argmin(valid_db))
else:
onset_idx = candidates[0]
det = twilight[onset_idx]
confidence = 1.0
mean_cloud = float(np.mean(cloud_scores))
if mean_cloud > 50:
confidence *= 0.5
elif mean_cloud > 20:
confidence *= 0.8
moon_alt = moon_altitude(det.utc_time, lat, lng)
if moon_alt > 10:
confidence *= 0.7
elif moon_alt > 0:
confidence *= 0.9
ci = color_index(det.west_roi_rgb)
if abs(ci) > 0.3:
confidence *= 0.6
if len(twilight) < 20:
confidence *= 0.7
return TwilightEvent(
prayer="isha",
utc_time=det.utc_time,
solar_depression_deg=det.solar_dep,
confidence=round(confidence, 3),
color_index=round(ci, 4),
brightness_east=float(np.mean(det.east_roi_rgb)),
brightness_west=float(np.mean(det.west_roi_rgb)),
sky_quality_mpsas=None,
moon_alt_deg=round(moon_alt, 1),
cloud_score=round(mean_cloud / 100, 3),
n_frames=len(twilight),
)

0
src/upload/__init__.py Normal file
View file

142
src/upload/sync.py Normal file
View file

@ -0,0 +1,142 @@
"""
Data upload module for fajr-watch.
Syncs completed detection results to the central pray-calc-ml dataset.
Runs as a cron job (hourly) or can be triggered manually.
"""
import json
import logging
import shutil
from datetime import datetime, timezone
from pathlib import Path
import requests
log = logging.getLogger(__name__)
RESULTS_DIR = Path("/var/lib/fajr-watch/data/results")
UPLOAD_QUEUE = Path("/var/lib/fajr-watch/data/upload-queue")
UPLOADED_DIR = Path("/var/lib/fajr-watch/data/uploaded")
def load_config() -> dict:
"""Load station config."""
import yaml
for path in ["/boot/fajr-watch/station.yaml", "/opt/fajr-watch/app/config/station.yaml"]:
p = Path(path)
if p.exists():
with open(p) as f:
return yaml.safe_load(f)
raise FileNotFoundError("No station.yaml found")
def sync():
"""Upload all pending results to the central server."""
config = load_config()
upload_url = config["network"].get("upload_url", "")
api_key = config["network"].get("api_key", "")
if not upload_url:
log.warning("No upload_url configured. Skipping sync.")
return
UPLOADED_DIR.mkdir(parents=True, exist_ok=True)
# Collect all JSON result files
results = sorted(RESULTS_DIR.glob("*.json"))
if not results:
log.info("No results to upload.")
return
log.info("Uploading %d result(s) to %s", len(results), upload_url)
headers = {"Content-Type": "application/json"}
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
uploaded = 0
failed = 0
for result_path in results:
try:
with open(result_path) as f:
data = json.load(f)
response = requests.post(
upload_url,
json=data,
headers=headers,
timeout=30,
)
if response.status_code in (200, 201, 204):
# Move to uploaded archive
shutil.move(str(result_path), str(UPLOADED_DIR / result_path.name))
uploaded += 1
else:
log.warning("Upload failed for %s: HTTP %d %s",
result_path.name, response.status_code, response.text[:200])
failed += 1
except requests.RequestException as e:
log.warning("Upload error for %s: %s", result_path.name, e)
failed += 1
except Exception as e:
log.error("Unexpected error uploading %s: %s", result_path.name, e)
failed += 1
log.info("Upload complete: %d uploaded, %d failed, %d remaining",
uploaded, failed, len(list(RESULTS_DIR.glob("*.json"))))
def export_csv(output_path: str = None):
"""
Export all local results as a CSV compatible with pray-calc-ml ingest format.
Useful for manual data transfer (USB stick) when the station has no internet.
"""
import csv
if output_path is None:
output_path = f"/var/lib/fajr-watch/data/export_{datetime.now().strftime('%Y%m%d')}.csv"
all_dirs = [RESULTS_DIR, UPLOADED_DIR]
records = []
for d in all_dirs:
if not d.exists():
continue
for f in sorted(d.glob("*.json")):
with open(f) as fh:
data = json.load(fh)
records.append(data)
if not records:
log.info("No records to export.")
return
fieldnames = [
"prayer", "date", "utc_time", "solar_depression_deg", "confidence",
"lat", "lng", "elevation_m", "environment", "horizon",
"camera", "sky_quality_mpsas", "moon_alt_deg", "cloud_score",
"color_index_at_detection", "station_id", "n_frames",
]
with open(output_path, "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames, extrasaction="ignore")
writer.writeheader()
for r in records:
writer.writerow(r)
log.info("Exported %d records to %s", len(records), output_path)
if __name__ == "__main__":
import sys
logging.basicConfig(level=logging.INFO)
if len(sys.argv) > 1 and sys.argv[1] == "export":
export_csv(sys.argv[2] if len(sys.argv) > 2 else None)
else:
sync()