Observability
55 tools across 4 categories tagged with “observability”
dev tools
(40)Langfuse
Open-source LLM engineering platform — traces, evals, prompt management — 23K+ stars
Arize Phoenix
Open-source LLM observability with tracing, evaluation, and datasets — 8K+ stars
Portkey
AI gateway with guardrails, caching, and observability for 200+ LLMs — 11K+ stars
Helicone
Open-source LLM observability proxy with one-line integration
LangSmith
Observability, tracing, and evaluation platform for LLM applications
AegisUI
Semantic security layer detecting malicious UI behaviors in AI agent systems beyond schema validation
Agent Audit
Security analysis system for LLM agent applications and MCP configurations
AgentBeats
Agent-based evaluation framework where AI judges assess other agents through standardized protocols
AgentCheck
Debug LLM agents with reproducible recordings and controlled interventions over MCP
AgentDev
Lightweight AI agent framework with built-in debugger for development workflows
AgentLens
Interpretable safety framework for steering coding agent behavior via LLM internal representations
Agent Lifecycle Toolkit (ALTK)
Standardized middleware for production-ready AI agents with built-in failure handling
ATLAS
LLM-powered automata learning for analyzing and explaining agent strategies
CausalFlow
Causal debugging framework that converts failed LLM agent traces into minimal counterfactual fixes
ClawTrap
MITM red-teaming framework for testing autonomous web agent security in real-world scenarios
Cognitive Firewall
Hybrid edge-cloud security defense against prompt injection attacks on browser AI agents
CUA-Suite
Large-scale human-annotated video dataset for training computer-use agents
Decoupled Search Grounding (DSG)
Vendor-agnostic architecture decoupling search retrieval from LLM reasoning for portable agents
ElephantBroker
Knowledge-grounded cognitive runtime for trustworthy AI agents with verifiable memory
FlowArk
Context-aware knowledge reuse system for optimized Android app analysis at scale
Hallucination-Aware Layered Oversight
Zero-hallucination AI framework through architectural separation of generation and validation
HARP
Research platform for systematic HCI studies of human-AI interaction with live LLM agents
JetBrains AI Toolkit
In-IDE LLM tracing and evaluation for JetBrains developers
KRCA
Multi-agent AI system for automated root cause analysis in microservice architectures
KYA
Framework-agnostic trust and governance layer for autonomous AI systems with verifiable provenance
LEDGER
Claim-to-evidence trace graphs for auditing and verifying LLM agent reasoning
Mandato
Cryptographic governance layer for AI agent tool authorization and audit trails
MCP-in-SoS
Security risk assessment framework for evaluating open-source MCP server implementations
mcp-sec-audit
Security auditing toolkit for MCP servers with static analysis and sandboxed testing
PIPES
Secure agent perception through data provenance tracking and authority controls
ProofAgent Index
Governance readiness index for assessing AI agent production safety and compliance
Redakto
Privacy-preserving PII removal for LLMs with GDPR compliance
RuView
WiFi-based spatial intelligence: presence detection and vital signs monitoring using ESP32
RuView
WiFi-based human pose estimation and vital sign monitoring using commodity hardware
SpecOps
Automated testing framework for GUI-based AI agents in real-world environments
StagedWorkspace
Versioned workspace infrastructure ensuring explicit state contracts for knowledge-work agents
TrendRadar
AI-powered trend monitoring with multi-platform aggregation and natural language analysis via MCP
Vera
Automated safety testing framework for LLM agents with risk discovery and verification
WorldMonitor
Real-time global intelligence dashboard with AI-powered news aggregation and geopolitical monitoring
worldmonitor
Real-time global intelligence dashboard with AI-powered news and geopolitical monitoring
frameworks
(12)ActiveGraph
Event-sourced graph runtime with time-travel debugging for auditable agent systems
agentdev
Lightweight AI agent framework with built-in debugger for OpenAI, Anthropic & MCP
AOHP
OS-level harness for secure and efficient AI agent execution with native system integration
DeepCLAW
Enterprise AI agent framework with zero-trust security and hardware-backed governance controls
Fred AI
Effect-based TypeScript framework for type-safe, composable AI agents
Reactive Agents
Effect-TS based agent framework with fine-grained control and MCP support
Reactive Agents
Type-safe AI agent framework with Effect-TS and React Loop patterns for composable agents
Reactive Agents
Effect-TS framework for composable AI agents with React-style loops and MCP support
Reactive Agents
Effect-TS framework for composable AI agents with structured dimensions and MCP integration
TITAN Agent
TypeScript framework for multi-agent orchestration with approval gates and Mission Control UI
TITAN Agent
Full-featured TypeScript agent framework with multi-agent orchestration and Mission Control UI
TITAN Agent
Autonomous AI agent framework with Mission Control GUI and multi-channel communication