From Notebooks to a Financial Intelligence Dashboard

April 26, 2026 3 minute read

From Notebooks to a Financial Intelligence Dashboard

Turning raw portfolio tracking into a tested analytics system, AI-powered reporting engine, and interactive dashboard.

🚀 Overview

The Financial Intelligence Lab started as a simple idea:

Track a small investment portfolio and learn how markets behave.

What it became is something much bigger:

a structured data pipeline
a tested analytics system
an AI-powered research layer
and a live Streamlit dashboard

This project is part of the broader DataInsideData™ platform, where I build systems in public to bridge the gap between learning and real-world execution.

🧠 The Core Problem

Most beginner investing workflows look like this:

spreadsheets
manual tracking
no reproducibility
no system-level insight

That approach breaks quickly.

I wanted to answer:

What would a real analytics system for portfolio tracking look like?

⚙️ System Architecture

At a high level, the system evolved into this pipeline:

Portfolio Snapshots (CSV)
        ↓
Data Cleaning + Validation
        ↓
Market Data Backfill (Yahoo Finance)
        ↓
Merged Analysis Dataset
        ↓
Derived Metrics + Aggregations
        ↓
AI Insight Generation
        ↓
Streamlit Dashboard

This is no longer simply a dataset — it has matured into a data system.

🏗️ Phase 1 — From Raw Data to Structured Pipelines

The first step was moving from raw CSVs to structured processing.

Key datasets

portfolio_snapshots.csv
portfolio_summary.csv
market_notes.csv
dividends.csv

What changed

standardized column naming
enforced data types
validation checks (duplicates, nulls)
reproducible load/merge pipeline

This was the turning point:

From “tracking data” → to “engineering data”

📊 Phase 2 — Market Data Integration

To move beyond static snapshots, I added:

Yahoo Finance backfill
daily historical prices
return calculations

New capabilities

price trends over time
volatility observation
normalized performance comparison
sector-level analysis

Now the system could answer:

How are my assets behaving relative to each other?

🔍 Phase 3 — Analysis Layer

The merged dataset became the core analytics engine.

Key features

position weights
unrealized return %
daily price change tracking
sector bucketing
dividend aggregation

Example outputs

portfolio value over time
capital vs market gain
sector allocation trends
normalized performance curves

This is where the project became:

A true analytics platform, not just tracking.

🤖 Phase 4 — AI Insight Generation

Next came the intelligence layer.

Using structured payloads + OpenAI:

daily AI commentary
weekly reports
structured prompts (cost-controlled)
reproducible outputs saved as markdown

Example workflow

Processed Data
    ↓
Structured Payload
    ↓
Prompt Builder
    ↓
LLM Response
    ↓
Markdown Report

This allowed the system to:

interpret trends
explain performance
generate research-style summaries

📈 Phase 5 — Streamlit Dashboard

Finally, everything was surfaced through a dashboard.

Features

portfolio overview metrics
interactive charts
symbol filtering
sector breakdowns
AI report viewer
on-demand AI summary generation

This transformed the system into: A live financial analytics interface

⚠️ This is a demo environment using non-sensitive data.

🧪 Engineering Practices

This project was intentionally built using real development practices:

modular src/ architecture
reusable functions
test coverage with pytest
environment management (conda)
script-based pipelines (python -m ...)
structured directories

Example:

src/
├── load_data.py
├── market_data.py
├── metrics.py
├── charts.py
├── ai_summary.py

This ensures:

reproducibility
scalability
maintainability

🔐 Security Considerations

Because this involves financial data:

no API keys are exposed
.env is protected
no brokerage credentials are used
demo-safe datasets can be used for public versions

Future plans include:

secure deployment layers
access control
private analytics views

🌐 Deployment Strategy

Because the main site is static (Jekyll), the dashboard is:

hosted separately
embedded via iframe

Current approach

Jekyll Site → Streamlit App (embedded)

Future architecture

CloudFront
   ├── Static Site (Jekyll)
   └── App Layer (Streamlit)

📚 What This Project Demonstrates

This project brings together:

data engineering
analytics
AI integration
dashboarding
system design

More importantly:

It shows how small experiments can evolve into real systems.

🔮 What’s Next

improved dashboard interactivity
real-time data ingestion
advanced AI analysis
scenario modeling
potential subscription-based research layer

💡 Final Thoughts

This started with:

“Let me track a small portfolio.”

It became:

“Let me build a financial analytics platform.”

That shift — from task → system → platform — is the core idea behind DataInsideData™.

Data Inside Data™.

Tech Hands, a Science Mind, and a Heart for Community™.

From Notebooks to a Financial Intelligence Dashboard