Marionette

Marionette lets you navigate and control any website using natural language, entirely offline and private.

𝗠𝗔𝗥𝗜𝗢𝗡𝗘𝗧𝗧𝗘 An AI browser automation agent that runs entirely on your device. Control websites using natural language through voice or text. Zero data leaves your machine. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗪𝗛𝗔𝗧 𝗜𝗧 𝗗𝗢𝗘𝗦 Marionette automates web browsing tasks using Chrome's built-in Gemini Nano AI. Tell it to "fill this form", "search for X", "capture this page", or "what's this video about?" and it handles the task autonomously. Everything runs locally on your device after a one-time model download (~2GB). No API keys, no subscriptions, no servers, no tracking. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗣𝗥𝗜𝗩𝗔𝗖𝗬 After initial setup, the extension operates 100% offline. Your conversations, captured pages, and browsing data stay in local browser storage. Nothing is transmitted to external servers. You can verify this by opening Chrome DevTools Network tab during use - you'll see zero outbound requests. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗖𝗔𝗣𝗔𝗕𝗜𝗟𝗜𝗧𝗜𝗘𝗦 𝘉𝘳𝘰𝘸𝘴𝘦𝘳 𝘈𝘶𝘵𝘰𝘮𝘢𝘵𝘪𝘰𝘯 Click buttons, fill forms, scroll pages, navigate tabs. Uses accessibility tree to understand page structure and interact with elements. 𝘝𝘰𝘪𝘤𝘦 & 𝘛𝘦𝘹𝘵 𝘐𝘯𝘱𝘶𝘵 Voice control using Web Speech API (on-device transcription) or text commands. Supports wake word detection. 𝘔𝘶𝘭𝘵𝘪𝘮𝘰𝘥𝘢𝘭 𝘜𝘯𝘥𝘦𝘳𝘴𝘵𝘢𝘯𝘥𝘪𝘯𝘨 Process screenshots ("what do you see?") and audio ("what's playing?"). Gemini Nano handles image and audio analysis locally. 𝘚𝘦𝘮𝘢𝘯𝘵𝘪𝘤 𝘝𝘢𝘶𝘭𝘵 Automatically captures pages you visit. Creates embeddings using Transformers.js for semantic search. Query your browsing history with natural language. 𝘈𝘨𝘦𝘯𝘵𝘪𝘤 𝘓𝘰𝘰𝘱 The AI observes, plans, acts, and repeats until task completion. Supports up to 60 tool iterations for complex workflows. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗨𝗦𝗘 𝗖𝗔𝗦𝗘𝗦 • "Fill this contact form" - Populates fields from stored information • "Capture this article for later" then "find that article about X" - Semantic search through your history • "Open YouTube and search for tutorials" - Multi-step navigation • "Summarize this page" - Content extraction and analysis • "What's this video about?" - Audio transcription and summarization • Voice control for accessibility when mouse/keyboard interaction is difficult ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗦𝗘𝗧𝗨𝗣 1. Install the extension 2. Enable required Chrome flags (interactive guide included) 3. Download models (~2GB one-time) 4. Grant microphone permission if using voice 5. Start using it The onboarding walks through flag setup with direct links to the correct chrome:// pages. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗗𝗔𝗧𝗔 𝗦𝗧𝗢𝗥𝗔𝗚𝗘 All data stored locally: • Conversation history (Chrome Extension storage) • Captured pages and embeddings (IndexedDB) • Optional user info for form autofill (Chrome Extension storage) • Screenshots and audio (temporary, local only) Nothing transmitted to external servers. Check DevTools Network tab to verify. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗧𝗘𝗖𝗛𝗡𝗜𝗖𝗔𝗟 𝗗𝗘𝗧𝗔𝗜𝗟𝗦 𝘈𝘐 𝘚𝘵𝘢𝘤𝘬: • Gemini Nano (Chrome Prompt API) • Transformers.js (all-MiniLM-L6-v2 embeddings) • Web Speech API (voice transcription) • Readability.js (content extraction) 𝘈𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵𝘶𝘳𝘦: • 22 specialized automation tools • Agentic loopback (observe → act → repeat) • Chunk-based RAG for semantic memory • TypeScript + React + Plasmo framework Open source under MIT License. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗥𝗘𝗤𝗨𝗜𝗥𝗘𝗠𝗘𝗡𝗧𝗦 • Chrome browser (latest version) • ~2GB disk space (one-time model download) • Chrome flags enabled (setup guide included) • Microphone for voice control (optional) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗟𝗜𝗠𝗜𝗧𝗔𝗧𝗜𝗢𝗡𝗦 This is an on-device AI system, which comes with trade-offs: • Gemini Nano is smaller than cloud models (3B parameters vs GPT-4's hundreds of billions) • Inference takes 1-3 seconds per response instead of sub-second for cloud APIs • Context window is 9,216 tokens (auto-summarization helps manage this) • Models take ~2GB disk space The benefit: complete privacy. Your data never leaves your device. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 𝗦𝗨𝗣𝗣𝗢𝗥𝗧 Email: [email protected] GitHub: Issues and pull requests welcome License: MIT (free and open source)