pray-calc-ml/data/raw/raw_sightings/india_twilight_photometer.csv at 9ea537d26d0f37096a9bdf73f8d60581deaa07ea

mirror of https://github.com/acamarata/pray-calc-ml.git synced 2026-07-01 11:24:26 +00:00

Aric Camarata ada08e7ec4 data: expand dataset from 5.9k to 91k records via 6 new SQM sources

Add 6 new data collection pipelines and their processed outputs:

Sources added:
- TESS/Stars4All photometer network: 37 months (Jun 2017-Aug 2020),
  ~40k raw events from 100+ European stations via Zenodo archives
- Globe at Night citizen science: 26k twilight observations (2006-2024),
  filtered from 308k total observations for solar depression 6-22 deg
- GaN-MN continuous monitoring: 45 months (Jan 2022-Sep 2025),
  ~12.5k twilight events from 88 stations across 20+ countries
- Galicia SQM network: 14 stations, 1-min resolution, 7.5k events
- Madrid/Majadahonda SQM: multi-year continuous monitoring, 3.1k events
- washetdonker.nl Netherlands: 7 stations, 3.3k morning events
- Academic papers: Jordan (Abed 2015), Fayum Egypt, India photometer

Pipeline changes:
- ingest.py: add all new files to APPROVED_RAW_CSVS allowlist,
  fix filter to use allowlist instead of hardcoded exclusions
- .gitignore: exclude bulk raw data directories (BSRN, TESS, GaN-MN,
  washetdonker, Globe at Night downloads)

Final dataset: 56,668 Fajr + 34,763 Isha = 91,431 total records
Previous: 5,871 Fajr + 46 Isha = 5,917 total records

2026-03-22 16:39:29 -04:00

4.9 KiB

Raw Blame History

1	prayer	date_local	time_local	utc_offset	lat	lng	elevation_m	source	notes
2	fajr	2009-01-04	06:21:38	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=10.077,solar_dep=10.03deg,inflection_method
3	fajr	2009-01-13	06:23:30	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.496,solar_dep=10.16deg,inflection_method
4	fajr	2009-01-22	06:25:07	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=7.770,solar_dep=10.01deg,inflection_method
5	fajr	2009-01-25	06:25:03	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=11.490,solar_dep=10.02deg,inflection_method
6	fajr	2009-02-01	06:24:00	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=7.430,solar_dep=10.10deg,inflection_method
7	isha	2009-02-01	19:09:32	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.056,solar_dep=10.05deg,inflection_method
8	isha	2009-02-09	19:22:30	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.174,solar_dep=12.32deg,inflection_method
9	isha	2009-02-10	19:15:03	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.588,solar_dep=10.49deg,inflection_method
10	isha	2009-02-15	19:15:00	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=5.926,solar_dep=10.08deg,inflection_method
11	isha	2009-03-18	19:23:09	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=4.559,solar_dep=10.26deg,inflection_method
12	isha	2009-03-27	19:23:59	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.361,solar_dep=10.06deg,inflection_method
13	isha	2009-04-06	19:26:08	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=5.755,solar_dep=10.08deg,inflection_method
14	isha	2009-04-28	19:32:19	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.749,solar_dep=10.12deg,inflection_method
15	isha	2009-04-29	19:32:37	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.378,solar_dep=10.11deg,inflection_method
16	isha	2009-05-03	19:36:31	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=6.099,solar_dep=10.67deg,inflection_method
17	fajr	2009-12-21	06:15:25	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=0.448,solar_dep=10.05deg,inflection_method
18	isha	2010-01-29	19:08:11	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=0.565,solar_dep=10.08deg,inflection_method
19	isha	2010-02-13	19:19:43	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=7.482,solar_dep=11.37deg,inflection_method
20	isha	2010-02-26	19:18:08	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.330,solar_dep=10.09deg,inflection_method
21	isha	2010-03-04	19:20:44	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=0.938,solar_dep=10.37deg,inflection_method
22	isha	2010-12-13	18:43:28	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=2.160,solar_dep=10.07deg,inflection_method
23	isha	2010-12-19	18:47:12	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.366,solar_dep=10.32deg,inflection_method
24	isha	2011-01-07	18:56:14	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.516,solar_dep=10.05deg,inflection_method
25	isha	2011-01-08	18:57:00	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.521,solar_dep=10.09deg,inflection_method
26	isha	2011-01-21	19:04:09	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.492,solar_dep=10.09deg,inflection_method
27	isha	2011-01-22	19:05:00	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.276,solar_dep=10.17deg,inflection_method
28	isha	2011-01-24	19:06:26	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.390,solar_dep=10.26deg,inflection_method
29	isha	2011-01-27	19:08:47	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.037,solar_dep=10.47deg,inflection_method
30	isha	2011-01-28	19:07:45	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.492,solar_dep=10.12deg,inflection_method
31	isha	2011-01-29	19:08:02	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.576,solar_dep=10.07deg,inflection_method
32	isha	2011-01-30	19:08:28	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.445,solar_dep=10.07deg,inflection_method
33	isha	2011-02-01	19:09:31	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.591,solar_dep=10.10deg,inflection_method
34	isha	2011-04-02	19:25:11	5.5	16.7	74.2	570.0	India_Twilight_Photometer_Zenodo5541367	photometer_intensity=1.197,solar_dep=10.08deg,inflection_method

4.9 KiB Raw Blame History

4.9 KiB

Raw Blame History