Data Flow
This document explains how data flows from the merchant's storefront through the entire Store Shield system.
Pixel Pipeline Flow
The core data collection happens through "pixels" - small JavaScript events sent from the storefront.
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Storefront │────▶│ POST /api/pixels │────▶│ pixel-router │
│ (Theme Ext.) │ │ (CF Worker) │ │ │
└──────────────────┘ └──────────────────┘ └────────┬─────────┘
│
┌──────────────────────────────────────┼──────────────────────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│ session-init │ │ bot-detection │ │ basic-security │
│ handler │ │ handler │ │ handler │
└─────────┬─────────┘ └─────────┬─────────┘ └─────────┬─────────┘
│ │ │
▼ ▼ ▼
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│ VisitorIdentity │ │ BotSignal │ │ ProtectionEvent │
│ SessionSnapshot │ │ DailyMetrics │ │ DailyMetrics │
└───────────────────┘ └───────────────────┘ └───────────────────┘
Pixel Types
| Pixel Type | Triggers When | Handler | Data Stored |
|---|---|---|---|
session_init | Page load | session-init-handler | Visitor fingerprint, session, device info |
basic_security | Protection triggered | basic-security-handler | Right-click/copy/devtools blocked |
bot_detection | Bot signal detected | bot-detection-handler | Selenium, headless, honeypot, trap |
spy_detection | Spy tool found | spy-detection-handler | PPSPY, Koala Inspector, etc. |
ip_blocking | IP/country blocked | ip-blocking-handler | Blocked IP, country, reason |
behavior_analytics | Mouse/scroll patterns | behavior-analytics-handler | Behavioral signals |
checkout_session | Checkout reached | checkout-session-handler | Conversion tracking |
Configuration Flow (Heartbeat)
Protection settings flow from the app to the storefront via a "heartbeat" system.
┌──────────────────┐
│ Remix App │
│ (Settings UI) │
└────────┬─────────┘
│ Save settings
▼
┌──────────────────┐
│ Prisma DB │
│ MerchantSettings│
└────────┬─────────┘
│ Webhook/API call
▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ CF Worker │────▶│ KV Cache │────▶│ D1 Cache │
│ heartbeat │ │ (5 min TTL) │ │ (backup) │
└────────┬─────────┘ └──────────────────┘ └──────────────────┘
│
▼
┌──────────────────┐
│ Theme Extension │
│ (Storefront) │
└──────────────────┘
Config Caching Layers
The heartbeat system uses a 4-layer caching strategy to minimize costs:
- Memory cache (Worker isolate) - Fastest, per-request
- KV cache - 5 minute TTL, global
- D1 cache - Persistent, edge-local
- Fly.io API - Source of truth (fallback)
See Config Caching for implementation details.
Analytics Flow
Analytics data flows from D1 aggregates to the dashboard.
┌──────────────────┐
│ D1 Tables │
│ • DailyMetrics │
│ • TopIPsDaily │
│ • TopPagesDaily │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ CF Worker │
│ analytics-handler│
│ GET /api/analytics/*│
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Remix App │
│ app/db/analytics│
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Dashboard UI │
│ Analytics page │
└──────────────────┘
Fraud Order Flow
Fraud analysis happens via Shopify webhooks.
┌──────────────────┐
│ Shopify │
│ New Order │
└────────┬─────────┘
│ Webhook: orders/create
▼
┌──────────────────┐
│ Remix App │
│ webhooks.orders.create │
└────────┬─────────┘
│ Query session data
▼
┌──────────────────┐
│ CF Worker │
│ Session lookup │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Fraud Analysis │
│ • Risk scoring │
│ • Signal matching│
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Prisma DB │
│ FraudOrder │
└──────────────────┘
Threat Scanning Flow
Background scanning for phishing and counterfeit threats.
┌──────────────────┐
│ Product Catalog │
│ (Shopify API) │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ ProductScanJob │
│ (Prisma) │
└────────┬─────────┘
│ Claimed by worker
▼
┌──────────────────────────────────────────┐
│ Background Workers │
├──────────────────┬───────────────────────┤
│ Phishing Scan │ Marketplace Scan │
│ • Domain check │ • SerpAPI queries │
│ • WHOIS lookup │ • Image similarity │
│ • SSL analysis │ • GPT threat rating │
└────────┬─────────┴───────────┬───────────┘
│ │
▼ ▼
┌──────────────────┐ ┌───────────────────┐
│ PhishingDomain │ │ MarketplaceThreat │
│ (Prisma) │ │ (Prisma) │
└──────────────────┘ └───────────────────┘
IP Blocking Flow
IP blocking configuration syncs from the app to the edge.
┌──────────────────┐
│ Remix App │
│ IP Blocking UI │
└────────┬─────────┘
│ Save config
▼
┌──────────────────┐
│ Prisma DB │
│ IPBlockingConfig│
└────────┬─────────┘
│ POST /api/sync-ip-blocking-config
▼
┌──────────────────┐
│ CF Worker │
│ Write to D1 + KV│
└────────┬─────────┘
│
▼
┌──────────────────┐ ┌──────────────────┐
│ Storefront │────▶│ GET /api/ │
│ Page Load │ │ check-ip-access │
└──────────────────┘ └────────┬─────────┘
│
▼
┌──────────────────┐
│ Allow / Block │
└──────────────────┘