← Back to ETL Studio

ETL Studio

A browser-based ETL (Extract, Transform, Load) designer powered by DuckDB-WASM. Build data pipelines visually with a step-based editor, preview results in real-time, and export to CSV, Parquet, Excel, JSON, or JSONL — all in your browser, no server required.

Live demo → pyaek.com/apps/etl/

Features

🧩 Step-Based Pipeline Editor

Build ETL workflows by adding, configuring, and reordering steps visually.

🔧 30+ Transform Steps

Filter, sort, aggregate, pivot, join, change types, formula columns, and more.

👁️ Live Preview

See results instantly as you build your pipeline — no manual refresh needed.

📝 SQL Editor

Write custom SQL with syntax highlighting and AI assistance.

🤖 AI-Powered Generation

Describe what you want in natural language and let AI write the SQL.

🔌 Multiple AI Providers

Ollama (local), Claude, OpenAI, Gemini, Groq, OpenRouter, and local browser models (wllama/WebLLM/Transformers.js).

📦 Export

Download results as CSV, Parquet, Excel, JSON, or JSONL.

🔗 Connections

Link local folders or connect to databases (PostgreSQL, MySQL, SQLite, etc.).

🔀 Cross-Query References

One query can reference another's output for complex workflows.

📲 PWA

Install as a desktop app, works offline after initial load.

🌙 Dark Theme

Easy on the eyes, built for extended use.

Tech Stack

Layer	Technology
Framework	React 18 + TypeScript
Bundler	Vite 6
Database	DuckDB-WASM (in-browser)
State	Zustand 5
Persistence	File System Access API + IndexedDB
UI	Hand-built with CSS custom properties
AI	Ollama, Claude, OpenAI, Gemini, Groq, OpenRouter, Local browser models (wllama/WebLLM/Transformers.js)
PWA	vite-plugin-pwa (Workbox)

Getting Started

Prerequisites

Node.js 18+
npm 9+

Development

# Install dependencies
npm install

# Start dev server
npm run dev

The app runs at http://localhost:5173 (or the next available port).

Build

npm run build

Output goes to dist/. Serve with any static file server.

AI Setup (Optional)

Ollama (local): Install Ollama, pull a model (ollama pull gemma3:4b), start with OLLAMA_ORIGINS=* ollama serve
Cloud providers: Get API keys from Anthropic, OpenAI, Google AI Studio, Groq, or OpenRouter, then configure in the AI panel
Local (browser): Download a small model (wllama, WebLLM, or Transformers.js) and run it entirely in your browser — no server, no API key, fully offline

Project Structure

src/
  main.tsx                    # App entry point
  App.tsx                     # Root component (layout + modals)
  components/
    layout/                   # App shell (Header, LeftPanel, CenterPanel, etc.)
    steps/                    # Step config dialogs
    modals/                   # Modal overlays (AI, Connections, etc.)
    icons/                    # SVG icon components
  steps/                      # Step definitions (buildSql per kind)
  engine/                     # CTE builder, SQL executor, references
  store/                      # Zustand stores
  lib/                        # Utilities (DuckDB, AI, export, file access)
  types/                      # TypeScript type definitions
  styles/                     # CSS (theme + layout)

Contact

Email: hello@pyaek.com

License

MIT — do whatever you want. Attribution appreciated.

← Back to ETL Studio · Technical Documentation