VS Code Extension - Multi-Agent AI - Vector Memory
Your IDE never forgets.
SecondCortex is a persistent AI memory layer for VS Code - it captures your workspace context as you code, stores it as searchable vector embeddings, and lets you restore any past session with a natural language command.
IDE events are captured in the background, embedded into a vector store, and made searchable - so any agent or external tool can pull relevant context when you need it.
01
Capture
The VS Code extension monitors every IDE event - open tabs, active files, terminal output, git state, with a debounced snapshot system.
eventCapture.ts
02
Embed
Snapshots are vectorized using text-embedding-3-small into 1536-dimensional space and stored in a persistent ChromaDB instance per user.
vector_db.py
03
Retrieve
When you trigger a session restore or ask a question, the Retriever searches your vector store by semantic similarity - returning the most relevant snapshots, including those from other repos.
retriever.py
04
Execute
After you confirm the Planner's proposed actions, the Executor applies them to your workspace - opening files, switching branches, running the Simulator sub-agent to check for git conflicts first.
executor.py
The agents
Three agents. One pipeline.
A focused multi-agent architecture where each component has a distinct role. The Executor runs a built-in Simulator sub-agent for pre-flight checks before touching your workspace.
PLN
Planner
Task Decomposition
Takes a natural language request and breaks it into a structured, step-by-step action plan. Uses retrieved context from your memory to make decisions relevant to your actual codebase - not guesses.
LLM: GPT-4o via GitHub Models
Output: Structured action plan
Requires explicit user confirmation
RTV
Retriever
Semantic Memory Search
Searches your ChromaDB vector store using cosine similarity to surface relevant past context - open files, git branches, code summaries. Also exposed as an MCP endpoint so Claude and Cursor can query it directly.
Store: ChromaDB (per-user namespace)
Embeddings: text-embedding-3-small
Exposed via MCP SSE endpoint
EXC
Executor
Workspace Restoration
Applies the approved action plan to your VS Code workspace - opening files, switching branches, restoring terminal context. Runs the Simulator sub-agent first to check for unstashed changes or branch conflicts before making any changes.
LLM: Groq Llama-3.1 (fast inference)
Sub-agent: Simulator (git pre-flight)
PowerShell + bash compatible
Live Memory
Query your past work.
ChromaDB - User NamespaceLIVE
auth/jwt_handler.py
Implemented RS256 JWT signing with 24h expiry and refresh token rotation.
feat/auth2h ago
agents/retriever.py
Added cross-workspace semantic search with ChromaDB collection isolation per user_id.
feat/mcp5h ago
security/firewall.ts
Semantic Firewall redacts API keys and secrets locally before upload.
feat/security1d ago
agents/simulator.py
Pre-flight simulator generates conflict Safety Reports from git diff.
feat/simulator2d ago
services/vector_db.py
VectorDB service wrapping ChromaDB with upsert and semantic search.
main3d ago
Ask your second cortex anything about your codebase.
Natural language semantic search across your entire development history - not just grep, but meaning.
Try asking:
JWT token flow
vector search logic
where are secrets handled
git branch conflicts
rate limiting implementation
Retriever - Awaiting Query
Click any memory entry or type a query to see semantic retrieval in action.
Security
Your secrets stay yours.
Credentials never leave your machine unredacted. User data is namespace-isolated, and no workspace change runs without your explicit approval.
Semantic Firewall
Redacts API keys, tokens, and credentials from every snapshot before it leaves your machine. Pattern-matched against common secret formats - env vars, bearer tokens, private keys.
Per-User Isolation
Each user gets a separate ChromaDB collection namespace. JWT-authenticated API endpoints ensure no cross-user data leakage at the storage or query layer.
Confirmation Before Execution
The Executor never runs without your sign-off. Every action plan is displayed as a readable summary in the VS Code sidebar before any workspace change is made.
Architecture
Production-grade from day one.
System Architecture Overview
VS Code Extension (TypeScript)
->
Event Capture + Debouncer
->
Semantic Firewall
->
FastAPI Backend
Data Ingestion Layer
FastAPI Backend
->
LLM Client (GPT-4o / Groq)
->
3-Agent Pipeline
->
ChromaDB (1536d vectors)
Intelligence Layer
MCP SSE Server
->
Claude / Cursor / Any AI
- External AI tools can query your Cortex memory natively via Model Context Protocol
Integration Layer
Azure Web App (Backend)
+
GitHub Actions CI/CD
+
Docker (GHCR)
+
Next.js Web Dashboard
Deployment Layer
Early Access
Build with context.
Install the VS Code extension. SecondCortex starts building your memory immediately.