What is an approach to monitor for data leakage from memorized training data?

• A. Differential privacy, regularization, and monitoring for leakage
• B. Increase model size
• C. Data duplication
• D. Ignoring privacy

Answer: (A)

Memorization risk occurs when a model unintentionally reveals sensitive training data in its outputs or through its parameters. To monitor and reduce this risk, a combination of privacy-preserving techniques and checks is most effective. Differential privacy introduces noise in the training process so that the influence of any single training example is limited, which helps prevent exact reproductions of training data and provides a quantifiable privacy guarantee. Regularization methods, such as weight penalties or dropout, discourage the model from fitting too closely to the training data, lowering the chance that memorized phrases or details are reproduced during inference. Monitoring for leakage involves actively testing and auditing the model’s outputs to detect signs that training data could be disclosed, for example by running targeted queries or leakage audits to see if exact training content appears in responses.

The other options don’t address privacy or leakage risk. Increasing model size can increase memorization potential and does not provide leakage safeguards. Data duplication can amplify leakage risk rather than mitigate it. Ignoring privacy offers no protection and leaves sensitive information vulnerable to exposure.