ETL Studio is a browser-based ETL (Extract, Transform, Load) designer that runs entirely in your browser using DuckDB-WASM. Build data pipelines with a visual step editor, preview results in real-time, and export to multiple formats — no server required.
The application follows a pipeline architecture where each step transforms data and passes it to the next step. Steps are compiled into a DuckDB SQL query using Common Table Expressions (CTEs):
WITH step_01 AS (SELECT * FROM source), step_02 AS (SELECT * FROM step_01 WHERE condition), step_03 AS (SELECT col, COUNT(*) FROM step_02 GROUP BY col) SELECT * FROM step_03
| Component | Description |
|---|---|
| Step Definitions | Pure functions that convert a step's config into DuckDB SQL |
| CTE Builder | Chains enabled steps into a single CTE-based SQL query |
| SQL Executor | Runs queries against DuckDB-WASM and returns results |
| Preview System | Auto-executes the pipeline on changes with debounced updates |
| State Management | Zustand stores for queries, steps, connections, UI state |
| File Persistence | File System Access API + IndexedDB for project files |
| Category | Steps | Description |
|---|---|---|
| Get Data | Source File, Source Table | Load data from files or database tables |
| Columns | Set Names, Pick Columns, Change Type, Duplicate | Manipulate column structure and types |
| Rows | Filter, Sort, Remove Duplicates, Fill Null | Filter and organize rows |
| Values | Replace Values, Remove Errors, Fix Errors | Clean and transform cell values |
| Transform | Add Index, Formula, Pivot, Unpivot, Clean Text | Advanced data transformations |
| Combine | Append Tables, Join Tables | Merge data from multiple sources |
| Aggregate | Group & Aggregate | Summarize and group data |
| Advanced | Custom SQL | Write hand-crafted SQL with AI assistance |
| Output | Export to File | Export results to CSV, Parquet, Excel, JSON, JSONL |
ETL Studio supports multiple AI providers for natural language to SQL generation:
AI settings are configured in the AI panel (click the "AI" button in the header). API keys are stored in your browser's localStorage and never sent to our servers.
| Format | Extension | Use Case |
|---|---|---|
| CSV | .csv | Universal spreadsheet format |
| Parquet | .parquet | Columnar storage, efficient for large data |
| Excel | .xlsx | Microsoft Excel workbooks |
| JSON | .json | Full result as a JSON array |
| JSONL | .jsonl | One JSON object per line, streaming-friendly |
The app uses DuckDB's WebAssembly build to run SQL queries entirely in the browser. DuckDB is an in-process SQL OLAP database that excels at analytical queries on large datasets. The WASM build runs at near-native speed using WebAssembly SIMD instructions.
Each enabled step becomes a CTE (Common Table Expression) in a single SQL query.
The CTE builder chains them together, resolving prev references to the
prior step's CTE name. Cross-query references (e.g., FROM other_query)
are resolved by inlining the referenced query's compiled SQL as a subquery.
Zustand stores manage all application state. The project file (etlstudio.json) is serialized/deserialized via the File System Access API and persisted to the user's local file system. Auto-save triggers 800ms after the last change.
The app uses the File System Access API (Chromium browsers) to read and write project files directly to the user's file system. Folder connections use the same API to register directory handles, enabling DuckDB to query files in linked folders.