Observability
40 tools across 4 categories tagged with “observability”
dev tools
(27)Langfuse
Open-source LLM engineering platform — traces, evals, prompt management — 23K+ stars
Arize Phoenix
Open-source LLM observability with tracing, evaluation, and datasets — 8K+ stars
Portkey
AI gateway with guardrails, caching, and observability for 200+ LLMs — 11K+ stars
Helicone
Open-source LLM observability proxy with one-line integration
LangSmith
Observability, tracing, and evaluation platform for LLM applications
AegisUI
Semantic security layer detecting malicious UI behaviors in AI agent systems beyond schema validation
Agent Audit
Security analysis system for LLM agent applications and MCP configurations
AgentBeats
Agent-based evaluation framework where AI judges assess other agents through standardized protocols
AgentLens
Interpretable safety framework for steering coding agent behavior via LLM internal representations
Agent Lifecycle Toolkit (ALTK)
Standardized middleware for production-ready AI agents with built-in failure handling
CausalFlow
Causal debugging framework that converts failed LLM agent traces into minimal counterfactual fixes
ClawTrap
MITM red-teaming framework for testing autonomous web agent security in real-world scenarios
Cognitive Firewall
Hybrid edge-cloud security defense against prompt injection attacks on browser AI agents
CUA-Suite
Large-scale human-annotated video dataset for training computer-use agents
Decoupled Search Grounding (DSG)
Vendor-agnostic architecture decoupling search retrieval from LLM reasoning for portable agents
ElephantBroker
Knowledge-grounded cognitive runtime for trustworthy AI agents with verifiable memory
JetBrains AI Toolkit
In-IDE LLM tracing and evaluation for JetBrains developers
KRCA
Multi-agent AI system for automated root cause analysis in microservice architectures
KYA
Framework-agnostic trust and governance layer for autonomous AI systems with verifiable provenance
MCP-in-SoS
Security risk assessment framework for evaluating open-source MCP server implementations
mcp-sec-audit
Security auditing toolkit for MCP servers with static analysis and sandboxed testing
RuView
WiFi-based spatial intelligence: presence detection and vital signs monitoring using ESP32
RuView
WiFi-based human pose estimation and vital sign monitoring using commodity hardware
SpecOps
Automated testing framework for GUI-based AI agents in real-world environments
TrendRadar
AI-powered trend monitoring with multi-platform aggregation and natural language analysis via MCP
Vera
Automated safety testing framework for LLM agents with risk discovery and verification
worldmonitor
Real-time global intelligence dashboard with AI-powered news and geopolitical monitoring
frameworks
(11)ActiveGraph
Event-sourced graph runtime with time-travel debugging for auditable agent systems
agentdev
Lightweight AI agent framework with built-in debugger for OpenAI, Anthropic & MCP
AOHP
OS-level harness for secure and efficient AI agent execution with native system integration
Fred AI
Effect-based TypeScript framework for type-safe, composable AI agents
Reactive Agents
Effect-TS based agent framework with fine-grained control and MCP support
Reactive Agents
Type-safe AI agent framework with Effect-TS and React Loop patterns for composable agents
Reactive Agents
Effect-TS framework for composable AI agents with React-style loops and MCP support
Reactive Agents
Effect-TS framework for composable AI agents with structured dimensions and MCP integration
TITAN Agent
TypeScript framework for multi-agent orchestration with approval gates and Mission Control UI
TITAN Agent
Full-featured TypeScript agent framework with multi-agent orchestration and Mission Control UI
TITAN Agent
Autonomous AI agent framework with Mission Control GUI and multi-channel communication