Monday 2:30 p.m.–3 p.m. in Hall 2
ETL-ing the Israeli Government (and living to tell the tale)
Adam Kariv
- Audience level:
- Intermediate
Abstract
In the past 8 years we've been collecting, scraping and processing government data - overcoming technical, legal and other hurdles - to create Israel's richest database of fiscal data (and growing!). In this talk I will talk about how we're doing this (we - at the Public Knowledge Workshop), and present the unique software frameworks and tools we've built to accomplish this goal. In particular, I will talk about the data-package pipelines framework, which combines a suite of standards and tools for 'packaging' data as well as a versatile engine for processing data streams.
