glean-data-handling
PII filtering: strip emails, phone numbers, SSNs from document body before indexing. Trigger: "glean data handling", "data-handling".
- License 许可
- MIT
- 最近更新
- 2026-07-13
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- glean-data-handling
- tags
- [ "saas", "enterprise-search", "glean" ]
- author
- Jeremy Longshore <jeremy@intentsolutions.io>
- license
- MIT
- version
- 1.0.0
- description
- PII filtering: strip emails, phone numbers, SSNs from document body before indexing. Trigger: "glean data handling", "data-handling".
- allowed-tools
- Read, Write, Edit, Bash(npm:*), Bash(curl:*), Grep
- compatible-with
- claude-code
03 SKILL.md 正文
Glean Data Handling
Overview
Glean enterprise search ingests documents from dozens of connectors (Google Drive, Confluence, Slack, Jira, Salesforce, etc.) and builds a unified search index with permission-aware access control. Data types include indexed document content, connector metadata, user permission maps, query logs, and search analytics. All document content must be PII-filtered before indexing, permission boundaries must be preserved to prevent data leakage across teams, and retention policies must be enforced to comply with corporate governance and GDPR/CCPA obligations.
Data Classification
| Data Type | Sensitivity | Retention | Encryption |
|---|---|---|---|
| Indexed document content | High (may contain PII) | Per source retention policy | AES-256 at rest |
| User permission maps | High (access control) | Sync lifecycle | TLS + at rest |
| Connector metadata | Medium | Until connector removed | AES-256 at rest |
| Search query logs | Medium (reveals intent) | 90 days default | AES-256 at rest |
| Search analytics/aggregates | Low | 1 year | TLS in transit |
Data Import
interface GleanDocument {
id: string; datasource: string; title: string;
body: string; permissions: { allowedUsers?: string[]; allowAnonymousAccess?: boolean };
updatedAt: string; url: string;
}
async function indexDocuments(docs: GleanDocument[], datasource: string) {
// PII strip before indexing
const sanitized = docs.map(doc => ({
...doc,
body: stripPII(doc.body),
}));
// Batch upload with pagination (max 100 per request)
for (let i = 0; i < sanitized.length; i += 100) {
const batch = sanitized.slice(i, i + 100);
await fetch(`https://customer-be.glean.com/api/index/v1/bulkindexdocuments`, {
method: 'POST',
headers: { Authorization: `Bearer ${process.env.GLEAN_INDEXING_TOKEN}`, 'Content-Type': 'application/json' },
body: JSON.stringify({ datasource, documents: batch }),
});
}
}
function stripPII(text: string): string {
return text
.replace(/\b[\w.+-]+@[\w-]+\.[\w.]+\b/g, '[EMAIL_REDACTED]')
.replace(/\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/g, '[PHONE_REDACTED]')
.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN_REDACTED]');
}
Data Export
async function exportSearchAnalytics(startDate: string, endDate: string) {
const res = await fetch(`https://customer-be.glean.com/api/v1/analytics`, {
method: 'POST',
headers: { Authorization: `Bearer ${process.env.GLEAN_API_TOKEN}`, 'Content-Type': 'application/json' },
body: JSON.stringify({ startDate, endDate, metrics: ['query_count', 'click_through', 'zero_results'] }),
});
const data = await res.json();
// Redact user identifiers from analytics export
return data.results.map((r: any) => ({ ...r, userId: undefined, query: r.query?.length > 3 ? r.query : '[SHORT_QUERY_REDACTED]' }));
}
Data Validation
function validateDocument(doc: GleanDocument): string[] {
const errors: string[] = [];
if (!doc.id || doc.id.length > 512) errors.push('Invalid document ID');
if (!doc.datasource) errors.push('Missing datasource identifier');
if (!doc.title || doc.title.length > 1000) errors.push('Title missing or exceeds 1000 chars');
if (!doc.body || doc.body.length === 0) errors.push('Empty document body');
if (!doc.permissions) errors.push('Missing permissions — defaults to deny-all');
if (doc.updatedAt && isNaN(Date.parse(doc.updatedAt))) errors.push('Invalid updatedAt timestamp');
return errors;
}
Compliance
- [ ] PII stripped from document body before indexing (emails, phones, SSNs)
- [ ] Permission boundaries enforced: allowedUsers scope matches source system ACLs
- [ ] Connector credentials stored in secret manager, rotated quarterly
- [ ] Search query logs retained max 90 days, purged via automated job
- [ ] GDPR right-to-erasure: delete all indexed content referencing a specific user on request
- [ ] CCPA: honor do-not-sell signals for search analytics data
- [ ] SOC 2 Type II audit trail for all indexing and deletion operations
Error Handling
| Issue | Cause | Fix |
|---|---|---|
| 403 on bulk index | Expired or insufficient indexing token | Rotate token, verify datasource permissions |
| Permission mismatch in search | Stale ACL sync from connector | Force re-sync connector permissions via admin API |
| PII detected in indexed content | New PII pattern not in strip regex | Add pattern to stripPII, re-index affected datasource |
| Zero-result queries spike | Connector sync failure, stale index | Check connector health dashboard, trigger manual re-crawl |
| Rate limit 429 on indexing | Batch size too large or too frequent | Reduce batch to 50 docs, add 500ms delay between batches |
Resources
Next Steps
See glean-security-basics.
04 同仓库更多 Skills
zustand-store-creator
Create zustand store creator operations. Auto-activating skill for Frontend Development. Triggers on: zustand store creator, zustand store creator Part of the Frontend Development skill category. Use when working with zustand store creator functionality. Trigger with phrases like "zustand store creator", "zustand creator", "zustand".
zero-trust-config-helper
Configure with zero trust config helper operations. Auto-activating skill for Security Advanced. Triggers on: zero trust config helper, zero trust config helper Part of the Security Advanced skill category. Use when configuring systems or services. Trigger with phrases like "zero trust config helper", "zero helper", "zero".
zapier-integration-helper
Assist with zapier integration helper operations. Auto-activating skill for Business Automation. Triggers on: zapier integration helper, zapier integration helper Part of the Business Automation skill category. Use when working with APIs or building integrations. Trigger with phrases like "zapier integration helper", "zapier helper", "zapier".
yaml-config-validator
Validate yaml config validator operations. Auto-activating skill for DevOps Basics. Triggers on: yaml config validator, yaml config validator Part of the DevOps Basics skill category. Use when configuring systems or services. Trigger with phrases like "yaml config validator", "yaml validator", "yaml".
xvary-stock-research
Thesis-driven equity analysis from public SEC EDGAR and market data; /analyze, /score, /compare workflows with bundled Python tools (Claude Code, Cursor, Codex).
xss-vulnerability-scanner
Scan xss vulnerability scanner operations. Auto-activating skill for Security Fundamentals. Triggers on: xss vulnerability scanner, xss vulnerability scanner Part of the Security Fundamentals skill category. Use when working with xss vulnerability scanner functionality. Trigger with phrases like "xss vulnerability scanner", "xss scanner", "xss".