Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Refactor redaction mechanism to a single interface #929

Closed
lukehinds opened this issue Feb 5, 2025 · 0 comments · Fixed by #1123
Closed

Refactor redaction mechanism to a single interface #929

lukehinds opened this issue Feb 5, 2025 · 0 comments · Fixed by #1123

Comments

@lukehinds
Copy link

Description

Right now we have secrets scanning , which encrypts matched tokens, secrets etc using AES CGM. This in turn has a session manager to track the mapping of secrets to encrypted strings. In hindsight this was over engineered, we really don't need this level of encryption. It's fine to just mask / redact the string, as the LLM can never determine what the original content was. This then leaves the session manager, again encryption does not add much here, as access to the local machine would result in access to the keys within the session manager.

For PII protections I took a different approach, which is to redact using UUIDs. UUIDs work well, they make a nice unique mappable element, have much less processing overhead.

With this in mind, I am repurposing this issue to have a single interface which accepts a string (secret, PII or user nominated string) and returns a UUID. A second interface can then be called, which presents a UUID and is returned with the unredacted metadata (secrets, PII etc). We also then end up with a single session manager.

We can then add context aware sanitation and refactor PII and secrets scanning to using a single interface reaction factory

Additional Context

No response

@lukehinds lukehinds changed the title [Story]: Refactor redaction mechanism to a single interface Feb 5, 2025
@yrobla yrobla self-assigned this Feb 19, 2025
yrobla added a commit that referenced this issue Feb 20, 2025
Instead use an uuid generator as we do for pii, and reuse
same session store mechanism

Closes: #929
yrobla added a commit that referenced this issue Feb 20, 2025
Instead use an uuid generator as we do for pii, and reuse
same session store mechanism

Closes: #929
yrobla added a commit that referenced this issue Feb 20, 2025
Instead use an uuid generator as we do for pii, and reuse
same session store mechanism

Closes: #929
yrobla added a commit that referenced this issue Feb 21, 2025
Instead use an uuid generator as we do for pii, and reuse
same session store mechanism

Closes: #929
yrobla added a commit that referenced this issue Mar 4, 2025
Instead use an uuid generator as we do for pii, and reuse
same session store mechanism

Closes: #929
yrobla added a commit that referenced this issue Mar 4, 2025
* feat: remove not needed encryption of secrets

Instead use an uuid generator as we do for pii, and reuse
same session store mechanism

Closes: #929

* fix tests

* unify interface in sensitive data

* add missing tests

* changes from rebase

* fixes from review

* fixes in tests

* fix tests
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants