FinSight Architecture

End-to-End Analytics Engineering Architecture

FinSight is a portfolio analytics engineering platform that simulates fintech customers, accounts, merchants, transactions, and product events, then operationalizes the data through ingestion, transformation, orchestration, and activation workflows.

Back to FinSight Hub

Visual proof point

Full architecture view

The diagram appears first so reviewers can quickly understand the end-to-end platform before reading implementation notes.

FinSight Analytics full architecture diagram — Hover or focus the diagram to zoom on desktop, then click or tap it for a full-screen, scrollable details view on mobile or desktop.

Project details

Portfolio analytics engineering platform

FinSight Analytics is a portfolio-grade analytics engineering project for a synthetic fintech company. It simulates customers, accounts, merchants, transactions, and product event data, then operationalizes that data across ingestion, transformation, orchestration, and activation workflows.

Current platform stack

Python generators and loaders, Cloudflare R2, Neon Postgres, dbt, Apache Superset, Mixpanel, Airflow, Dagster, GitHub Actions, Caddy, and portfolio documentation.

Validated workflow

GitHub Actions runs active ingestion workflows, raw fintech batches land in Cloudflare R2, raw data loads into Neon Postgres, and dbt runs in the dbt_fs schema.

Orchestration surfaces

Airflow runs VM-hosted operational workflows and Mixpanel syncs with retries, logs, and email alerting. Dagster orchestrates dbt assets, lineage, source freshness, snapshots, model runs, and tests.

Protected access

VM-hosted Airflow and Dagster surfaces are fronted by Caddy with protected access controls, while public proof points remain available through GitHub Actions, dbt documentation, Superset, Mixpanel, and this portfolio page.

1 Generate & land

GitHub Actions and Python generate source-like fintech micro-batches and land raw files in Cloudflare R2.

2 Load & model

Raw files load into Neon Postgres, and dbt owns the dbt_fs transformation layer with staging, intermediate, and reporting models.

3 Activate & observe

Superset and Mixpanel provide analytics activation, while Airflow and Dagster provide protected orchestration, lineage, logs, and alerting context.

Implementation notes

Modeling, validation, and repository structure

Repository modules: data_generator/, object_storage/, loaders/, product_analytics/, scripts/, airflow/, dagster_project/, dbt_fintech/, and docs/.
dbt contract: raw is ingestion-owned; dbt_fs is dbt-owned; staging models are views, intermediate models are tables, and mart/reporting models are views.
Reporting layers: dbt uses stg_* source-cleaned views, int_* trusted business truth tables, and mrt_rp_ reporting/dashboard views.
Observability: Airflow includes retry behavior, task logs, and email alerts; Dagster provides dbt asset visibility, lineage, scheduled execution, and monitoring context.