Safe Observability: A Framework for Automated PII Redaction from LLM Prompts in OpenTelemetry Pipelines

Abstract

The proliferation of Large Language Models (LLMs) within enterprise applications has introduced a critical conflict between the goals of modern observability and the mandates of data privacy. While observability platforms provide essential, deep visibility into complex distributed systems, they inadvertently become repositories for Personally Identifiable Information (PII) when they ingest the unstructured, information-rich prompts users submit to LLMs. This leakage of sensitive data into telemetry pipelines constitutes a significant security liability and a compliance risk under regulations such as GDPR and CCPA. This paper introduces the concept of "Safe Observability," a paradigm that reconciles the need for comprehensive system insight with robust privacy protection. I propose a novel framework for achieving this through the automated redaction of PII within the OpenTelemetry (OTel) ecosystem. The core of this framework is a custom, configurable PII-Redaction Processor for the OpenTelemetry Collector, designed to act as a strategic control plane for sanitizing telemetry data in-transit. The architecture employs a hybrid PII detection methodology, combining the speed of regular expressions with the contextual accuracy of Named Entity Recognition (NER) models, implemented as a decoupled microservice. This paper details the architectural design, provides a comprehensive implementation guide for building and deploying the custom processor using the OpenTelemetry Collector Builder (OCB) and Go, and presents a rigorous evaluation of its efficacy and performance impact. The findings demonstrate that this approach offers a practically viable and architecturally sound solution for preventing PII leakage, enabling organizations to leverage the power of observability and LLMs without compromising user privacy or regulatory compliance.