pray-calc-ml/research
Aric Camarata ada08e7ec4 data: expand dataset from 5.9k to 91k records via 6 new SQM sources
Add 6 new data collection pipelines and their processed outputs:

Sources added:
- TESS/Stars4All photometer network: 37 months (Jun 2017-Aug 2020),
  ~40k raw events from 100+ European stations via Zenodo archives
- Globe at Night citizen science: 26k twilight observations (2006-2024),
  filtered from 308k total observations for solar depression 6-22 deg
- GaN-MN continuous monitoring: 45 months (Jan 2022-Sep 2025),
  ~12.5k twilight events from 88 stations across 20+ countries
- Galicia SQM network: 14 stations, 1-min resolution, 7.5k events
- Madrid/Majadahonda SQM: multi-year continuous monitoring, 3.1k events
- washetdonker.nl Netherlands: 7 stations, 3.3k morning events
- Academic papers: Jordan (Abed 2015), Fayum Egypt, India photometer

Pipeline changes:
- ingest.py: add all new files to APPROVED_RAW_CSVS allowlist,
  fix filter to use allowlist instead of hardcoded exclusions
- .gitignore: exclude bulk raw data directories (BSRN, TESS, GaN-MN,
  washetdonker, Globe at Night downloads)

Final dataset: 56,668 Fajr + 34,763 Isha = 91,431 total records
Previous: 5,871 Fajr + 46 Isha = 5,917 total records
2026-03-22 16:39:29 -04:00
..
outreach Expand dataset to 5,871 Fajr / 46 Isha across 114 locations 2026-02-28 10:51:01 -05:00
abed-2015-jordan.md data: expand dataset from 5.9k to 91k records via 6 new SQM sources 2026-03-22 16:39:29 -04:00
aggregate_comparison.md Expand dataset to 5,871 Fajr / 46 Isha across 114 locations 2026-02-28 10:51:01 -05:00
aggregate_d0_values.csv Expand dataset to 5,871 Fajr / 46 Isha across 114 locations 2026-02-28 10:51:01 -05:00
baghdad-sky-brightness-2021.md data: expand dataset from 5.9k to 91k records via 6 new SQM sources 2026-03-22 16:39:29 -04:00
birmingham-openfajr.md Rebuild as Python data science project 2026-02-25 19:32:47 -05:00
fayum-egypt-2022.md data: expand dataset from 5.9k to 91k records via 6 new SQM sources 2026-03-22 16:39:29 -04:00
hail-saudi-2018.md Rebuild as Python data science project 2026-02-25 19:32:47 -05:00
malaysia-indonesia-2018.md Rebuild as Python data science project 2026-02-25 19:32:47 -05:00
malaysia-light-pollution-2024.md data: expand dataset from 5.9k to 91k records via 6 new SQM sources 2026-03-22 16:39:29 -04:00
nriag-egypt-4sites-2014.md data: expand dataset from 5.9k to 91k records via 6 new SQM sources 2026-03-22 16:39:29 -04:00
nriag-egypt-1984-2014.md Rebuild as Python data science project 2026-02-25 19:32:47 -05:00
README.md Rebuild as Python data science project 2026-02-25 19:32:47 -05:00
tubruq-libya-2015.md data: expand dataset from 5.9k to 91k records via 6 new SQM sources 2026-03-22 16:39:29 -04:00
uk-observations.md Rebuild as Python data science project 2026-02-25 19:32:47 -05:00

Research Archive

This folder contains summaries of academic and institutional research on Islamic twilight angles. These papers describe other researchers' conclusions about the solar depression angle at Fajr and Isha. They are useful for understanding the scientific landscape but are not used as ML training data — only raw per-date observations with explicit timestamps feed the data/processed/ datasets.

Summary: What the research says

Most peer-reviewed naked-eye studies find Fajr (true dawn / Subh Sadiq) corresponds to a solar depression of roughly 13°16° depending on site, season, and atmospheric conditions. Isha (Shafaq al-Abyad, white dusk twilight) corresponds to roughly 14°18°.

The classic convention of 18° Fajr (used by ISNA, MWL, and others) is based on astronomical twilight (the sky becoming fully dark), not the first appearance of dawn light. Observations consistently show true dawn appears while the sun is 12°15° below the horizon, not 18°.

Papers Summarized

File Authors Year Site Finding
nriag-egypt-1984-2014.md Hassan et al. 1984-2014 Egypt (6 sites) 13°-15° Fajr
nriag-egypt-2022-2025.md Rashed et al. 2018-2025 Egypt (Fayum, Alex) 13°-14° Fajr
hail-saudi-2018.md Khalifa 2014-2015 Hail, Saudi Arabia 14° mean Fajr
malaysia-indonesia-2018.md Kassim Bahali 2017 KL + Indonesia 16.67° Fajr
depok-indonesia-2020.md Saksono 2015 Depok, Java ~16° Fajr
uk-observations.md Yusuf, Hizbul Ulama 1987-2017 UK (3 sites) 12°-14° Fajr
birmingham-openfajr.md OpenFajr project 2016-2026 Birmingham 12.5°-14° Fajr
moonsighting-global.md Khalid Shaukat 2000s Multiple global 15°-18° Fajr