pray-calc-ml/README.md

# pray-calc-ml

A Python data science project that collects and back-calculates solar depression angles
from human-verified Fajr and Isha prayer sightings. The goal is to find the real empirical
patterns in how the solar depression angle at Fajr and Isha varies with latitude, season,
and elevation — then use machine learning to refine the DPC (Dynamic Pray Calc) algorithm
in [pray-calc](https://github.com/acamarata/pray-calc).

## What this is

Most Islamic prayer time calculators use a fixed angle (e.g. 15° or 18°) for Fajr and Isha.
Peer-reviewed observation studies consistently find the real angle is lower and varies with
latitude, season, and atmospheric conditions. This project compiles the most complete
dataset of actual human-verified sightings and back-calculates the solar depression angle
at each observed moment.

The training data comes exclusively from confirmed human sightings with explicit dates,
locations, and times. No aggregated statistics or calculated-angle guesses are used as
ground truth. Each record is back-calculated independently using PyEphem.

## Datasets

Two clean CSV files are generated by the pipeline:

**`data/processed/fajr_angles.csv`** — One confirmed Fajr sighting per row

| Column | Description |
| --- | --- |
| `date` | YYYY-MM-DD (local calendar date) |
| `utc_dt` | ISO 8601 UTC datetime |
| `lat` | Decimal degrees (north positive) |
| `lng` | Decimal degrees (east positive) |
| `elevation_m` | Metres above sea level |
| `day_of_year` | 1-366 (seasonality feature) |
| `fajr_angle` | Solar depression angle at moment of sighting (degrees) |
| `source` | Citation |
| `notes` | Observer notes |

**`data/processed/isha_angles.csv`** — Same schema with `isha_angle`.

### Current dataset size

- **Fajr:** ~4,100 records, 35 unique locations, latitude range -37.8° to 53.7°
- **Isha:** ~43 records, 20+ locations
- **Date range:** 1984 to 2026

The dominant Fajr source is the [OpenFajr Project](https://openfajr.org) — 4,000+ community-reviewed
daily observations from Birmingham, UK. The remaining records are manually compiled from
peer-reviewed studies spanning Egypt, Saudi Arabia, Malaysia, Indonesia, Turkey, Morocco,
and other locations across five continents.

## Setup

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

## Running the pipeline

```bash
python -m src.pipeline
```

This fetches the OpenFajr iCal feed (network required), loads the compiled sighting records,
back-calculates depression angles, and writes both CSVs.

```bash
python -m src.pipeline --no-elevation-lookup
```

Skip the Open-Elevation API calls and use pre-set elevations from the source records.

## Project structure

```text
pray-calc-ml/
├── src/
│   ├── angle_calc.py          Back-calculation: observed time -> depression angle (PyEphem)
│   ├── elevation.py           Open-Elevation API lookup
│   ├── pipeline.py            Master pipeline: collect -> enrich -> filter -> export
│   └── collect/
│       ├── openfajr.py        OpenFajr iCal feed parser
│       └── verified_sightings.py  Manually compiled records from peer-reviewed studies
├── data/
│   ├── raw/sources.md         Full data source documentation
│   └── processed/             Generated CSVs (not committed to git)
├── notebooks/
│   └── 01_exploratory_analysis.ipynb  Latitude, TOY, and elevation pattern analysis
├── research/                  Academic paper summaries (not training data)
└── requirements.txt
```

## Back-calculation method

For each confirmed sighting (date, location, observed local time):

1. Convert observed local time to UTC using the documented UTC offset
2. Set up a PyEphem observer at the sighting location with standard atmosphere (1013.25 hPa, 15°C)
3. Compute solar altitude at the UTC moment, including atmospheric refraction
4. Depression angle = negative altitude (positive when sun is below the horizon)

Records where the depression angle is below 7° (Fajr) or 10° (Isha) are dropped as data
entry errors. This catches DST clock-change artifacts in the OpenFajr feed and a small number
of mis-estimated observation times.

## Key findings so far

The data shows three main patterns:

1. **Latitude matters.** Near-equatorial sites (Malaysia, Indonesia, 2°-7°) show mean Fajr angles
   of 16°-17°. Mid-latitude sites (UK at 52°N) average ~13°. This counter-intuitive result
   occurs because the sun rises at a steeper angle at low latitudes, compressing the twilight
   interval.

2. **Season matters.** At fixed latitude, Fajr angle is lower in summer than winter. Birmingham's
   10-year dataset shows a clear sinusoidal seasonal pattern with a ~3° peak-to-trough range.

3. **Elevation has a smaller but real effect.** High-altitude desert sites (Hail 1020m, Tehran
   1191m, Kottamia 477m) consistently trend toward the high end of the angle distribution.

## Data sources

See [data/raw/sources.md](data/raw/sources.md) for the full source table.

Primary sources:

- [OpenFajr Project](https://openfajr.org) — Birmingham, UK, community astrophotography
- NRIAG Egypt (Hassan et al. 2014, 2016; Rashed et al. 2022, 2025)
- Khalifa 2018, NRIAG J. — Hail, Saudi Arabia
- Kassim Bahali et al. 2018, Sains Malaysia — Malaysia/Indonesia DSLR study
- Saksono 2020, NRIAG J. — Depok, Indonesia (SQM)
- Asim Yusuf 2017 — Exmoor UK (multi-observer)
- Hizbul Ulama UK 1987-1989 — Blackburn, Lancashire
- Moonsighting.com / Khalid Shaukat — global network (Chicago, Buffalo, Toronto, Karachi, Cape Town, Auckland, Trinidad)
- OIF UMSU 2017-2020 — Medan, North Sumatra
- Various national religious body timetables (Turkey, Morocco, Jordan, Iran, UAE, Oman)

## Related packages

- [pray-calc](https://github.com/acamarata/pray-calc) — Islamic prayer times calculator; this project feeds its DPC algorithm
- [nrel-spa](https://github.com/acamarata/nrel-spa) — NREL Solar Position Algorithm used inside pray-calc
- [moon-sighting](https://github.com/acamarata/moon-sighting) — Lunar crescent visibility

## License

MIT. Copyright (c) 2026 Aric Camarata.