Text Anonymization

Using Presidio¹ and Faker², we can easily make a simple text anonymization tool or PII (Personally Identifiable Information) removal tool. The (hu)spaCy integration of Presidio can be used to identify and remove personal data, such as names, locations, phone numbers, or even bank details. This tool uses (hu)spaCy's Named Entity Recognition facilities and further pattern matching rules. What is more, an easy-to-use de-identification method is provided by Faker as we show below.

Initializing Presidio with HuSpaCy

Option 1: using only model names

// Here we use the hu_core_news_lg model, but any model supporting NER is a valid option
configuration = {
    "nlp_engine_name": "spacy",
    "models": [{"lang_code": "hu", "model_name": "hu_core_news_lg", }],
}

provider = NlpEngineProvider(nlp_configuration=configuration)
nlp_engine = provider.create_engine()

analyzer = AnalyzerEngine(nlp_engine=nlp_engine,
                            supported_languages=["hu"],)

Option 2: Building on a previously initialized nlp

class HuSpaCyNlpEngine(SpacyNlpEngine):
    def __init__(self, nlp: Language):
        self.nlp = {"hu": nlp}


def process():
    nlp = spacy.load("hu_core_news_trf")

    nlp_engine = HuSpaCyNlpEngine(nlp)

    analyzer = AnalyzerEngine(nlp_engine=nlp_engine, supported_languages=["hu"])

Configuring Faker operators

fake = Faker(locale=["hu_HU"])

fake_operators = {
    "PERSON": OperatorConfig("custom", {"lambda": lambda x: fake.name()}),
    "LOCATION": OperatorConfig("custom", {"lambda": lambda x: fake.address()}),
    "EMAIL_ADDRESS": OperatorConfig("custom", {"lambda": lambda x: fake.email()}),
    "PHONE_NUMBER": OperatorConfig("custom", {"lambda": lambda x: fake.phone_number()}),
    "CRYPTO": OperatorConfig("custom", {"lambda": lambda x: fake.password()}),
    "IP_ADDRESS": OperatorConfig("custom", {"lambda": lambda x: fake.ipv4()}),
    "URL": OperatorConfig("custom", {"lambda": lambda x: fake.url()}),
    "DATE_TIME": OperatorConfig("custom", {"lambda": lambda x: fake.date()}),
    "CREDIT_CARD": OperatorConfig("custom", {"lambda": lambda x: fake.credit_card_number()}),
    "IBAN_CODE": OperatorConfig("custom", {"lambda": lambda x: fake.iban()}),
}

Applying Presidio's anonymizer with Faker

anonymizer = AnonymizerEngine()
anonymized_text = anonymizer.anonymize(text=text, analyzer_results=results, operators=fake_operators)

This example is available on Hugging Face Spaces, while the full source code is on GitHub.