NCNYC Civil Title Viewer
Profiles of NYC civil-service titles — descriptions, positions, roles and pay. Began as a standalone experiment and now lives as Databook’s titles section.
DA
activeopen government
Databook
An award-winning transparency tool that combines over 50 official NYC open datasets — agencies, people, jobs, capital projects, schools, districts and procurement — into one searchable picture of how city government works.
DASubprojects
In the open
Activity
Jul 27This update resolves API connection timeout errors by routing heavy DuckDB scans to a dedicated thread executor, preventing Postgres connection pool starvation. It also configures the Postgres pool to open fully on startup and adds Sentry error alerts to prevent silent failures in weekly data refresh scripts.wegovnyc/databook-private ↗Jul 27Added vendor and contract profile data enrichment and implemented a dedicated DuckDB executor to prevent API event loop blocking and connection timeouts.wegovnyc/databook-private ↗Jul 25Added Sentry Cron monitoring for payroll and version-controlled the curated crosswalk. Updated the NYCHA curated crosswalk to support reviewed rejections, and documented crosswalk normalization and false-positive fixes. Warmed the Postgres database pool baseline to prevent API cold-start connection storms.wegovnyc/databook-private ↗Jul 25This pull request adds Sentry Cron monitoring to the monthly payroll refresh script to alert on failures or runs exceeding 12 hours. It also version-controls the curated NYCHA crosswalk CSV file to prevent data loss and adds one newly approved vendor mapping.wegovnyc/databook-private ↗Jul 25This update adds support for durable fuzzy-match rejections in NYCHA curated crosswalks by storing rejected pairs with a NULL ID so they are not re-proposed. It also sets the curated file path to a default location for automatic weekly runs and updates the logs to report confirmations and rejections separately.wegovnyc/databook-private ↗Jul 25This pull request fixes API timeout bursts by increasing the minimum Postgres connection pool size from 2 to 10. It also adds startup logging to track the pool's state.wegovnyc/databook-private ↗Knowledge
No knowledge items yet.
Repositories
wegovnyc/databook_interfaceJan 26, 2022