Three responsibilities
DataNepal’s public architecture assigns separate responsibilities to ingestion, transformation, and export. This note explains those documented choices; it does not describe an internal enterprise implementation.
Ingestion handles external sources through dlt connectors. Transformation uses DuckDB and dbt to move through staging, intermediate models, and marts. Publication exports Parquet and JSON together with the dataset catalog.
GitHub Actions runs this sequence. Published files are committed so the web deployment can build independently of the Python pipeline.
A publication contract
A successful transformation alone is not enough to publish a table. The exporter also requires catalog metadata: source, licence, vintage, and caveats.
That requirement makes provenance a condition of publication. It gives consumers context with the download and makes a missing catalog entry visible at the export boundary.
The shared geographic spine provides a second form of consistency. Canonical OCHA identifiers and tested crosswalks connect data across sources without depending on variable place names.
A deliberate serving constraint
Published Parquet and JSON are delivered as static files. DuckDB-WASM supports queries in the browser. For the documented footprint, this keeps the serving model small.
The consequence is a snapshot-based product. Changes become visible after ingestion, validation, and export. This architecture should be reconsidered as data volumes, freshness requirements, or query needs change.
This is a design tradeoff inferred from the public architecture. It is not a claim of measured savings or a recommendation for every enterprise workload.
Further reading
- DataNepal architecture and project documentation
- The complete DataNepal case study
- Explore the published data
This technical note adapts the existing portfolio case study and public project documentation. Source material reviewed October 2, 2026.