chore: superclean compliance pass

- Migrate .wiki/ to .github/wiki/ (GCI standard for public repos)
- Add _Sidebar.md for GitHub Wiki navigation
- Update wiki-sync.yml to reference .github/wiki/ path
- Remove .markdownlintignore (covered by .vscode/settings.json)
- Migrate .allow-ai-terms to ALLOW_AI_TERMS_REPOS in pre-commit hook
- Expand .gitignore with full IDE and AI agent directory list
- Update README project structure reference

.allow-ai-terms

@@ -1,4 +0,0 @@
-# .allow-ai-terms
-# Disables the AI-attribution pre-commit hook for this repo.
-# .markdownlintignore and .vscode/settings.json reference ".claude/**" as a
-# directory path to exclude from lint checks — not as AI attribution.

.github/wiki/_Sidebar.md

@@ -0,0 +1,20 @@
+**pray-calc-ml**
+
+- [Home](Home)
+
+**Dataset**
+- [Data](Data)
+- [Data Sources](Data-Sources)
+- [Data Collection](Data-Collection)
+
+**Analysis**
+- [Architecture](Architecture)
+- [ML Crunching](ML-Crunching)
+
+**Research**
+- [Research](Research)
+- [Research Notes](Research-Notes)
+
+---
+
+*Part of [acamarata](https://github.com/acamarata)*

.github/workflows/wiki-sync.yml

@@ -4,7 +4,10 @@ on:
   push:
     branches: [main]
     paths:
-      - ".wiki/**"
+      - ".github/wiki/**"
+
+permissions:
+  contents: write
 
 jobs:
   sync:
@@ -12,10 +15,10 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      - name: Sync .wiki/ to GitHub Wiki
+      - name: Sync .github/wiki/ to GitHub Wiki
         uses: newrelic/wiki-sync-action@v1.0.1
         with:
-          source: .wiki
+          source: .github/wiki
           destination: wiki
           token: ${{ secrets.GITHUB_TOKEN }}
           gitAuthorName: github-actions[bot]

.gitignore

@@ -12,7 +12,24 @@ env/
 .DS_Store
 .ipynb_checkpoints/
 .jupyter/
+# IDE directories
+.vscode/
+.idea/
+*.suo
+*.user
+*.swp
+*.swo
+
+# AI agent working directories
 .claude/
+.codex/
+.cursor/
+.aider/
+.aider.chat.history.md
+.continue/
+.windsurf/
+.gemini/
+.codeium/
 
 # Raw scraped/downloaded files
 data/raw/*.pdf

.markdownlintignore

@@ -1,4 +0,0 @@
-**/.claude/**
-.claude/**
-**/node_modules/**
-node_modules/**

README.md

@@ -40,14 +40,15 @@ Two clean CSV files are generated by the pipeline:
 
 ### Current dataset size
 
-- **Fajr:** ~4,100 records, 35 unique locations, latitude range -37.8° to 53.7°
+- **Fajr:** 5,871 records, 110 unique locations, latitude range -9.6° to 53.8°
-- **Isha:** ~43 records, 20+ locations
+- **Isha:** 46 records, 5 unique locations, latitude range -33.9° to 53.8°
-- **Date range:** 1984 to 2026
+- **Date range:** 1970 to 2026
 
-The dominant Fajr source is the [OpenFajr Project](https://openfajr.org) — 4,000+ community-reviewed
+The dominant Fajr source is the [OpenFajr Project](https://openfajr.org), with 4,000+
-daily observations from Birmingham, UK. The remaining records are manually compiled from
+community-reviewed daily observations from Birmingham, UK. The second-largest source is
-peer-reviewed studies spanning Egypt, Saudi Arabia, Malaysia, Indonesia, Turkey, Morocco,
+Basthoni's 2022 PhD dissertation (UIN Walisongo), with 1,621 per-night SQM records across
-and other locations across five continents.
+46 Indonesian sites. The remaining records are manually compiled from peer-reviewed studies
+spanning Egypt, Saudi Arabia, Malaysia, Indonesia, Mauritania, and other locations.
 
 ## Setup
 
@@ -77,18 +78,23 @@ Skip the Open-Elevation API calls and use pre-set elevations from the source rec
 ```text
 pray-calc-ml/
 ├── src/
-│   ├── angle_calc.py          Back-calculation: observed time -> depression angle (PyEphem)
+│   ├── angle_calc.py              Back-calculation: observed time -> depression angle (PyEphem)
-│   ├── elevation.py           Open-Elevation API lookup
+│   ├── elevation.py               Open-Topo-Data / Open-Elevation API lookup
-│   ├── pipeline.py            Master pipeline: collect -> enrich -> filter -> export
+│   ├── ingest.py                  Standardize and validate raw CSV files
+│   ├── pipeline.py                Master pipeline: collect -> enrich -> filter -> export
 │   └── collect/
-│       ├── openfajr.py        OpenFajr iCal feed parser
+│       ├── openfajr.py            OpenFajr iCal feed parser (~4,018 Fajr records)
-│       └── verified_sightings.py  Manually compiled records from peer-reviewed studies
+│       ├── verified_sightings.py  Manually compiled records from peer-reviewed studies
+│       ├── precomputed_angles.py  1,621 Basthoni 2022 SQM records (46 Indonesian sites)
+│       ├── brin_multistation_sqm.py   BRIN multistation SQM processor
+│       ├── brin_timau_sqm.py      BRIN Mount Timau SQM processor
+│       ├── paper_extractor.py     PDF/HTML table extractor for academic papers
+│       └── pdf_extractor.py       PDF text extraction via PyMuPDF + pdfminer
 ├── data/
-│   ├── raw/sources.md         Full data source documentation
+│   ├── raw/raw_sightings/         Per-source raw CSV files
-│   └── processed/             Generated CSVs (not committed to git)
+│   └── processed/                 Generated CSVs (fajr_angles.csv, isha_angles.csv)
-├── notebooks/
+├── research/                      Academic paper summaries, aggregate D0 database
-│   └── 01_exploratory_analysis.ipynb  Latitude, TOY, and elevation pattern analysis
+├── .github/wiki/                  GitHub Wiki pages (synced via Actions)
-├── research/                  Academic paper summaries (not training data)
 └── requirements.txt
 ```
 
@@ -122,20 +128,20 @@ The data shows three main patterns:
 
 ## Data sources
 
-See [data/raw/sources.md](data/raw/sources.md) for the full source table.
+See the [wiki](https://github.com/acamarata/pray-calc-ml/wiki/Data-Sources) for the full
+citation table.
 
 Primary sources:
 
-- [OpenFajr Project](https://openfajr.org) — Birmingham, UK, community astrophotography
+- [OpenFajr Project](https://openfajr.org) -- Birmingham, UK, community astrophotography (~4,018 records)
-- NRIAG Egypt (Hassan et al. 2014, 2016; Rashed et al. 2022, 2025)
+- Basthoni 2022 PhD, UIN Walisongo -- 46 Indonesian SQM sites (1,621 records)
-- Khalifa 2018, NRIAG J. — Hail, Saudi Arabia
+- BRIN Mount Timau SQM -- NTT, Indonesia (59 Fajr + 577 Isha)
-- Kassim Bahali et al. 2018, Sains Malaysia — Malaysia/Indonesia DSLR study
+- NRIAG Egypt (Hassan et al. 2014, Semeida & Hassan 2018, Marzouk et al. 2025)
-- Saksono 2020, NRIAG J. — Depok, Indonesia (SQM)
+- Taha et al. 2025, EJSAS -- Riyadh, Saudi Arabia + Mauritania
-- Asim Yusuf 2017 — Exmoor UK (multi-observer)
+- Khalifa 2018, NRIAG J. -- Hail, Saudi Arabia
-- Hizbul Ulama UK 1987-1989 — Blackburn, Lancashire
+- Kassim Bahali et al. 2018, 2019 -- Malaysia/Indonesia DSLR + SQM studies
-- Moonsighting.com / Khalid Shaukat — global network (Chicago, Buffalo, Toronto, Karachi, Cape Town, Auckland, Trinidad)
+- Miftahi/Shaukat 2015 -- Blackburn, Lancashire UK (29 Fajr + 32 Isha)
-- OIF UMSU 2017-2020 — Medan, North Sumatra
+- Asim Yusuf 2017 -- Exmoor UK (multi-observer)
-- Various national religious body timetables (Turkey, Morocco, Jordan, Iran, UAE, Oman)
 
 ## Related packages