Databook
activeopen governmentAn award-winning transparency tool that combines over 50 official NYC open datasets — agencies, people, jobs, capital projects, schools, districts and procurement — into one searchable picture of how city government works.
The update also removes password interpolation in connection strings to prevent exposure in logs, while maintaining env.yaml as a local fallback.
Updated documentation to record the official decommissioning of the staging environment on July 29, 2026.
Implemented support for composite source fields (e.g., "A+B"), which joins multiple column values with a space. This fixes a bug where community district enrichment silently failed for datasets relying on combined columns, and ensures consistent token generation across both streaming and non-streaming data transformation paths.
Fixed a bug where output CSV headers depended on the first row's data, which caused missing enrichment columns when the first row was unmatched. The streaming transformer now derives the complete header from configuration before writing, ensuring consistent columns regardless of row order.
Fixed an issue where an empty data source was incorrectly treated as a pipeline failure. The ingestion process now handles empty sources gracefully without failing.
Deactivated a duplicate CPDB registry class and added safeguards to prevent future duplicates. Implemented a dataset staleness monitoring feature that alerts when source data falls behind or empties. Published a blog post on open data reliability and AI-readiness.