pray-calc-ml/data/raw/raw_sightings/india_twilight_photometer.csv
Aric Camarata ada08e7ec4 data: expand dataset from 5.9k to 91k records via 6 new SQM sources
Add 6 new data collection pipelines and their processed outputs:

Sources added:
- TESS/Stars4All photometer network: 37 months (Jun 2017-Aug 2020),
  ~40k raw events from 100+ European stations via Zenodo archives
- Globe at Night citizen science: 26k twilight observations (2006-2024),
  filtered from 308k total observations for solar depression 6-22 deg
- GaN-MN continuous monitoring: 45 months (Jan 2022-Sep 2025),
  ~12.5k twilight events from 88 stations across 20+ countries
- Galicia SQM network: 14 stations, 1-min resolution, 7.5k events
- Madrid/Majadahonda SQM: multi-year continuous monitoring, 3.1k events
- washetdonker.nl Netherlands: 7 stations, 3.3k morning events
- Academic papers: Jordan (Abed 2015), Fayum Egypt, India photometer

Pipeline changes:
- ingest.py: add all new files to APPROVED_RAW_CSVS allowlist,
  fix filter to use allowlist instead of hardcoded exclusions
- .gitignore: exclude bulk raw data directories (BSRN, TESS, GaN-MN,
  washetdonker, Globe at Night downloads)

Final dataset: 56,668 Fajr + 34,763 Isha = 91,431 total records
Previous: 5,871 Fajr + 46 Isha = 5,917 total records
2026-03-22 16:39:29 -04:00

34 lines
4.9 KiB
Text

prayer,date_local,time_local,utc_offset,lat,lng,elevation_m,source,notes
fajr,2009-01-04,06:21:38,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=10.077,solar_dep=10.03deg,inflection_method"
fajr,2009-01-13,06:23:30,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.496,solar_dep=10.16deg,inflection_method"
fajr,2009-01-22,06:25:07,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=7.770,solar_dep=10.01deg,inflection_method"
fajr,2009-01-25,06:25:03,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=11.490,solar_dep=10.02deg,inflection_method"
fajr,2009-02-01,06:24:00,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=7.430,solar_dep=10.10deg,inflection_method"
isha,2009-02-01,19:09:32,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.056,solar_dep=10.05deg,inflection_method"
isha,2009-02-09,19:22:30,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.174,solar_dep=12.32deg,inflection_method"
isha,2009-02-10,19:15:03,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.588,solar_dep=10.49deg,inflection_method"
isha,2009-02-15,19:15:00,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=5.926,solar_dep=10.08deg,inflection_method"
isha,2009-03-18,19:23:09,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=4.559,solar_dep=10.26deg,inflection_method"
isha,2009-03-27,19:23:59,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.361,solar_dep=10.06deg,inflection_method"
isha,2009-04-06,19:26:08,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=5.755,solar_dep=10.08deg,inflection_method"
isha,2009-04-28,19:32:19,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.749,solar_dep=10.12deg,inflection_method"
isha,2009-04-29,19:32:37,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.378,solar_dep=10.11deg,inflection_method"
isha,2009-05-03,19:36:31,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=6.099,solar_dep=10.67deg,inflection_method"
fajr,2009-12-21,06:15:25,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=0.448,solar_dep=10.05deg,inflection_method"
isha,2010-01-29,19:08:11,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=0.565,solar_dep=10.08deg,inflection_method"
isha,2010-02-13,19:19:43,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=7.482,solar_dep=11.37deg,inflection_method"
isha,2010-02-26,19:18:08,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.330,solar_dep=10.09deg,inflection_method"
isha,2010-03-04,19:20:44,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=0.938,solar_dep=10.37deg,inflection_method"
isha,2010-12-13,18:43:28,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=2.160,solar_dep=10.07deg,inflection_method"
isha,2010-12-19,18:47:12,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.366,solar_dep=10.32deg,inflection_method"
isha,2011-01-07,18:56:14,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.516,solar_dep=10.05deg,inflection_method"
isha,2011-01-08,18:57:00,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.521,solar_dep=10.09deg,inflection_method"
isha,2011-01-21,19:04:09,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.492,solar_dep=10.09deg,inflection_method"
isha,2011-01-22,19:05:00,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.276,solar_dep=10.17deg,inflection_method"
isha,2011-01-24,19:06:26,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.390,solar_dep=10.26deg,inflection_method"
isha,2011-01-27,19:08:47,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.037,solar_dep=10.47deg,inflection_method"
isha,2011-01-28,19:07:45,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.492,solar_dep=10.12deg,inflection_method"
isha,2011-01-29,19:08:02,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.576,solar_dep=10.07deg,inflection_method"
isha,2011-01-30,19:08:28,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.445,solar_dep=10.07deg,inflection_method"
isha,2011-02-01,19:09:31,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.591,solar_dep=10.10deg,inflection_method"
isha,2011-04-02,19:25:11,5.5,16.7,74.2,570.0,India_Twilight_Photometer_Zenodo5541367,"photometer_intensity=1.197,solar_dep=10.08deg,inflection_method"