Skip to content

Antoine/kor 4806 #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
antoineKorbit wants to merge 5 commits into from
Closed

Antoine/kor 4806 #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c4502b9
Allow to hide llm.prompts
antoineKorbit Jul 14, 2025
04ea366
Fix lint
antoineKorbit Jul 15, 2025
7b20dd2
Fix test
antoineKorbit Jul 15, 2025
7d2e9bb
complete implementation on js side
antoineKorbit Jul 15, 2025
059be9a
fix formatting
antoineKorbit Jul 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions js/packages/openinference-core/src/trace/trace-config/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ export const OPENINFERENCE_HIDE_EMBEDDING_VECTORS =
/** Limits characters of a base64 encoding of an image */
export const OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH =
"OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH";
/** Hides LLM prompts */
export const OPENINFERENCE_HIDE_PROMPTS = "OPENINFERENCE_HIDE_PROMPTS";
Comment on lines +26 to +27

@development-korbit-ai-mentor development-korbit-ai-mentor Bot Jul 14, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Insufficient Configuration Documentation category Readability

Tell me more
What is the issue?

The JSDoc comment is too vague and doesn't explain what 'LLM prompts' are or when this flag should be used.

Why this matters

Developers unfamiliar with LLMs may not understand the purpose and impact of this configuration option, leading to incorrect usage.

Suggested change ∙ Feature Preview
/** 
 * When enabled, hides the input prompts sent to Language Learning Models (LLMs).
 * Use this flag to prevent sensitive prompt data from being exposed in traces.
 */
export const OPENINFERENCE_HIDE_PROMPTS = "OPENINFERENCE_HIDE_PROMPTS";
Provide feedback to improve future suggestions

Nice Catch Incorrect Not in Scope Not in coding standard Other

💬 Looking for more details? Reply to this comment to chat with Korbit.


export const DEFAULT_HIDE_INPUTS = false;
export const DEFAULT_HIDE_OUTPUTS = false;
Expand All @@ -37,6 +39,8 @@ export const DEFAULT_HIDE_OUTPUT_TEXT = false;
export const DEFAULT_HIDE_EMBEDDING_VECTORS = false;
export const DEFAULT_BASE64_IMAGE_MAX_LENGTH = 32000;

export const DEFAULT_HIDE_PROMPTS = false;

/** When a value is hidden, it will be replaced by this redacted value */
export const REDACTED_VALUE = "__REDACTED__";

Expand Down Expand Up @@ -92,6 +96,11 @@ export const traceConfigMetadata: Readonly<
envKey: OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH,
type: "number",
},
hidePrompts: {
default: DEFAULT_HIDE_PROMPTS,
envKey: OPENINFERENCE_HIDE_PROMPTS,
type: "boolean",
},
};

export const DefaultTraceConfig: TraceConfig = {
Expand All @@ -104,4 +113,5 @@ export const DefaultTraceConfig: TraceConfig = {
hideOutputText: DEFAULT_HIDE_OUTPUT_TEXT,
hideEmbeddingVectors: DEFAULT_HIDE_EMBEDDING_VECTORS,
base64ImageMaxLength: DEFAULT_BASE64_IMAGE_MAX_LENGTH,
hidePrompts: DEFAULT_HIDE_PROMPTS,
};
1 change: 1 addition & 0 deletions js/packages/openinference-core/src/trace/trace-config/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ export {
OPENINFERENCE_HIDE_EMBEDDING_VECTORS,
DEFAULT_BASE64_IMAGE_MAX_LENGTH,
DEFAULT_HIDE_EMBEDDING_VECTORS,
OPENINFERENCE_HIDE_PROMPTS,
DEFAULT_HIDE_INPUTS,
DEFAULT_HIDE_INPUT_IMAGES,
DEFAULT_HIDE_INPUT_MESSAGES,
Expand Down
16 changes: 16 additions & 0 deletions js/packages/openinference-core/src/trace/trace-config/maskingRules.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,21 @@ const maskEmbeddingVectorsRule: MaskingRule = {
action: () => undefined,
};

/**
* Masks (redacts) llm.prompts if hidePrompts is true.
* @example
* ```typescript
* maskPromptsRule.condition({
* config: {hidePrompts: true},
* key: "llm.prompts"
* }) // returns true so the rule applies and the value will be redacted
*/
const maskPromptsRule: MaskingRule = {
condition: ({ config, key }) =>
config.hidePrompts && key === SemanticConventions.LLM_PROMPTS,
action: () => REDACTED_VALUE,
};

/**
* A list of {@link MaskingRule}s that are applied to span attributes to either redact or remove sensitive information.
* The order of these rules is important as it can ensure appropriate masking of information
Expand Down Expand Up @@ -185,6 +200,7 @@ const maskingRules: MaskingRule[] = [
maskInputImagesRule,
maskLongBase64ImageRule,
maskEmbeddingVectorsRule,
maskPromptsRule,
];

/**
Expand Down
1 change: 1 addition & 0 deletions js/packages/openinference-core/src/trace/trace-config/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export type TraceConfigOptions = {
hideOutputText?: boolean;
hideEmbeddingVectors?: boolean;
base64ImageMaxLength?: number;
hidePrompts?: boolean;
};

/**
Expand Down
9 changes: 9 additions & 0 deletions js/packages/openinference-core/test/trace-config/maskingRules.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,15 @@ const generateMaskTestCases = (): MaskTestCases[] => {
"data:image/base64,verylongbase64string",
]);
break;
case "hidePrompts":
testCases.push([
`should return ${REDACTED_VALUE} for "llm.prompts" when hidePrompts is set to true`,
{ ...DefaultTraceConfig, hidePrompts: true },
REDACTED_VALUE,
"llm.prompts",
"some prompt",
]);
break;
default:
assertUnreachable(configKey);
}
Expand Down
13 changes: 13 additions & 0 deletions python/openinference-instrumentation/src/openinference/instrumentation/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,13 @@ def __aexit__(
# Hides embedding vectors
OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH = "OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH"
# Limits characters of a base64 encoding of an image
OPENINFERENCE_HIDE_PROMPTS = "OPENINFERENCE_HIDE_PROMPTS"
# Hides LLM prompts
REDACTED_VALUE = "__REDACTED__"
# When a value is hidden, it will be replaced by this redacted value

DEFAULT_HIDE_LLM_INVOCATION_PARAMETERS = False
DEFAULT_HIDE_PROMPTS = False
DEFAULT_HIDE_INPUTS = False
DEFAULT_HIDE_OUTPUTS = False

Expand Down Expand Up @@ -185,6 +188,14 @@ class TraceConfig:
},
)
"""Hides embedding vectors"""
hide_prompts: Optional[bool] = field(
default=None,
metadata={
"env_var": OPENINFERENCE_HIDE_PROMPTS,
"default_value": DEFAULT_HIDE_PROMPTS,
},
)
"""Hides LLM prompts"""
base64_image_max_length: Optional[int] = field(
default=None,
metadata={
Expand Down Expand Up @@ -213,6 +224,8 @@ def mask(
) -> Optional[AttributeValue]:
if self.hide_llm_invocation_parameters and key == SpanAttributes.LLM_INVOCATION_PARAMETERS:
return None
elif self.hide_prompts and key == SpanAttributes.LLM_PROMPTS:
value = REDACTED_VALUE
elif self.hide_inputs and key == SpanAttributes.INPUT_VALUE:
value = REDACTED_VALUE
elif self.hide_inputs and key == SpanAttributes.INPUT_MIME_TYPE:
Expand Down
10 changes: 10 additions & 0 deletions python/openinference-instrumentation/tests/test_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
DEFAULT_HIDE_OUTPUT_MESSAGES,
DEFAULT_HIDE_OUTPUT_TEXT,
DEFAULT_HIDE_OUTPUTS,
DEFAULT_HIDE_PROMPTS,
OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH,
OPENINFERENCE_HIDE_INPUT_IMAGES,
OPENINFERENCE_HIDE_INPUT_MESSAGES,
Expand All @@ -31,6 +32,7 @@
OPENINFERENCE_HIDE_OUTPUT_MESSAGES,
OPENINFERENCE_HIDE_OUTPUT_TEXT,
OPENINFERENCE_HIDE_OUTPUTS,
OPENINFERENCE_HIDE_PROMPTS,
REDACTED_VALUE,
)
from openinference.semconv.trace import SpanAttributes
Expand All @@ -46,6 +48,7 @@ def test_default_settings() -> None:
assert config.hide_input_images == DEFAULT_HIDE_INPUT_IMAGES
assert config.hide_input_text == DEFAULT_HIDE_INPUT_TEXT
assert config.hide_output_text == DEFAULT_HIDE_OUTPUT_TEXT
assert config.hide_prompts == DEFAULT_HIDE_PROMPTS
assert config.base64_image_max_length == DEFAULT_BASE64_IMAGE_MAX_LENGTH


Expand Down Expand Up @@ -117,6 +120,7 @@ def test_attribute_priority(k: str, in_memory_span_exporter: InMemorySpanExporte
@pytest.mark.parametrize("hide_input_images", [False, True])
@pytest.mark.parametrize("hide_input_text", [False, True])
@pytest.mark.parametrize("hide_output_text", [False, True])
@pytest.mark.parametrize("hide_prompts", [False, True])
@pytest.mark.parametrize("base64_image_max_length", [10_000])
def test_settings_from_env_vars_and_code(
hide_inputs: bool,
Expand All @@ -126,6 +130,7 @@ def test_settings_from_env_vars_and_code(
hide_input_images: bool,
hide_input_text: bool,
hide_output_text: bool,
hide_prompts: bool,
base64_image_max_length: int,
monkeypatch: pytest.MonkeyPatch,
) -> None:
Expand All @@ -135,6 +140,7 @@ def test_settings_from_env_vars_and_code(
monkeypatch.setenv(OPENINFERENCE_HIDE_INPUT_MESSAGES, str(hide_input_messages))
monkeypatch.setenv(OPENINFERENCE_HIDE_OUTPUT_MESSAGES, str(hide_output_messages))
monkeypatch.setenv(OPENINFERENCE_HIDE_INPUT_IMAGES, str(hide_input_images))
monkeypatch.setenv(OPENINFERENCE_HIDE_PROMPTS, str(hide_prompts))
monkeypatch.setenv(OPENINFERENCE_HIDE_INPUT_TEXT, str(hide_input_text))
monkeypatch.setenv(OPENINFERENCE_HIDE_OUTPUT_TEXT, str(hide_output_text))
monkeypatch.setenv(OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH, str(base64_image_max_length))
Expand All @@ -147,6 +153,7 @@ def test_settings_from_env_vars_and_code(
assert config.hide_input_images is parse_bool_from_env(OPENINFERENCE_HIDE_INPUT_IMAGES)
assert config.hide_input_text is parse_bool_from_env(OPENINFERENCE_HIDE_INPUT_TEXT)
assert config.hide_output_text is parse_bool_from_env(OPENINFERENCE_HIDE_OUTPUT_TEXT)
assert config.hide_prompts is parse_bool_from_env(OPENINFERENCE_HIDE_PROMPTS)
assert config.base64_image_max_length == int(
os.getenv(OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH, default=-1)
)
Expand All @@ -161,6 +168,7 @@ def test_settings_from_env_vars_and_code(
new_hide_input_images = not hide_input_images
new_hide_input_text = not hide_input_text
new_hide_output_text = not hide_output_text
new_hide_prompts = not hide_prompts
config = TraceConfig(
hide_inputs=new_hide_inputs,
hide_outputs=new_hide_outputs,
Expand All @@ -169,6 +177,7 @@ def test_settings_from_env_vars_and_code(
hide_input_images=new_hide_input_images,
hide_input_text=new_hide_input_text,
hide_output_text=new_hide_output_text,
hide_prompts=new_hide_prompts,
base64_image_max_length=new_base64_image_max_length,
)
assert config.hide_inputs is new_hide_inputs
Expand All @@ -178,6 +187,7 @@ def test_settings_from_env_vars_and_code(
assert config.hide_input_images is new_hide_input_images
assert config.hide_input_text is new_hide_input_text
assert config.hide_output_text is new_hide_output_text
assert config.hide_prompts is new_hide_prompts
assert config.base64_image_max_length == new_base64_image_max_length


Expand Down
4 changes: 3 additions & 1 deletion spec/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ The possible settings are:
| OPENINFERENCE_HIDE_OUTPUTS | Hides output value & all output messages | bool | False |
| OPENINFERENCE_HIDE_INPUT_MESSAGES | Hides all input messages & embedding input text | bool | False |
| OPENINFERENCE_HIDE_OUTPUT_MESSAGES | Hides all output messages | bool | False |
| PENINFERENCE_HIDE_INPUT_IMAGES | Hides images from input messages | bool | False |
| OPENINFERENCE_HIDE_INPUT_IMAGES | Hides images from input messages | bool | False |
| OPENINFERENCE_HIDE_INPUT_TEXT | Hides text from input messages & input embeddings | bool | False |
| OPENINFERENCE_HIDE_PROMPTS | Hides LLM prompts | bool | False |
| OPENINFERENCE_HIDE_OUTPUT_TEXT | Hides text from output messages | bool | False |
| OPENINFERENCE_HIDE_EMBEDDING_VECTORS | Hides returned embedding vectors | bool | False |
| OPENINFERENCE_BASE64_IMAGE_MAX_LENGTH | Limits characters of a base64 encoding of an image | int | 32,000 |
Expand Down Expand Up @@ -44,6 +45,7 @@ If you are working in Python, and want to set up a configuration different than
hide_output_text=...,
hide_embedding_vectors=...,
base64_image_max_length=...,
hide_prompts=...,
)

from openinference.instrumentation.openai import OpenAIInstrumentor
Expand Down
Loading