Survey on Deep Neural Network Watermarking techniques
- 1. A survey of deep neural network watermarking techniques Based on: https://arxiv.org/pdf/2103.09274.pdf Original Authors: Yue Lia, Hongxia Wangb, and Mauro Barnic Date Published: 16 Mar 2021
- 2. Introduction What is watermarking in general? In digital watermarking, a low-amplitude, potentially pseudorandom, signal is injected into the original document. Signals in this document are intended to exploit some sort of redundancy in the content of the document and that can be added easily without butchering the content of the document. Why watermark is needed? Watermarks serve to protect content and to claim ownership of an asset. Without watermarks, valuable digital assets are vulnerable to content theft or unauthorized use and distributions.
- 3. Why Deep Neural Network (DNN) watermarking ? - Deep learning models are becoming more popular because of its human like capabilities and for the same reason deployed and shared widely. - Training a deep learning model is a non-trivial task, requires huge amounts of proprietary data, and expends enormous computing, energy, and human expertise. - DNNs should be protected from unauthorised commercialization and monetization the models.
- 4. How is watermarking embedded in DNN - DNN has degree of freedom to encode additional information as they have a substantial number of parameters. - This doesn’t impede the primary task DNN is handling. - Embedding takes place in training phase by properly altering the loss function. - Impact is measurable from the performance achieved by the watermarked model.
- 5. Requirements for DNN watermarking Capacity: Refers to the number of bits encoded by the watermark. Despite the fact that a large payload is good for watermarking algorithms, it conflicts directly with robustness. Fidelity: Models containing watermarks should receive performance levels similar to those of models trained without watermarks. Robustness: Ability to extract the watermark correctly even when it has been modified. The two most common manipulations a DNN watermark must withstand are: ● Fine-tuning: re-training a model to solve a new task, alters weights ● Network pruning: To simplify a complex neural network model for deployment in low power or computationally weak devices. Trade-off triangle
- 6. Other requirements Security: There are two main kinds of intentional attacks: ● Watermark overwriting: This procedure involves adding an additional watermark to the model in order to render the original watermark unnoticeable. ● Surrogate model attack. With a surrogate model attack, an attacker trains a bogus network by feeding it a series of feedback and then uses that output to mimic the original network's functionality. Generality: DNN watermarking algorithms should be adaptable to a range of architectures carrying out a variety of tasks. Efficiency: It is the computational overhead to train the DNN on the task while simultaneously embedding the watermark.
- 7. DNN watermarking models - I Multi-bit vs. zero-bit watermarking techniques - Based on the exact content of the watermark message - When the watermark message is multi-bit, it corresponds to a sequence of N bits - When zero-bit is used, watermark extraction is carried out as a detection task, wherein the detector should determine the presence of a known watermark. Multibit (a) vs zero-bit (b) watermarking
- 8. DNN watermarking models - II Static vs. dynamic DNN watermarking - Based on where the watermark can be read from. Static watermarking: The watermark can be read directly from the network weights, which can be considered similar to conventional multimedia watermarking techniques. Dynamic watermarking: When fed with some crafted inputs, a dynamic watermark can alter the behavior of the network, which makes the watermark message visible in the model output
- 9. Static vs. dynamic DNN watermarking Static (a) vs Dynamic (b) DNN watermarking
- 10. DNN watermarking models - III White-box vs. black-box DNN extraction - Based on the data accessible to the watermark extractor White-Box: When internal parameters/weights of the DNN models are available, watermark recovery is undertaken in a white-box mode. This can be static or dynamic watermarking. Black-box: When using black-box watermarking, only the final output of the DNN is accessible. ● A watermark can be recovered by querying the model and comparing the model output to a set of correctly chosen inputs. ● During the entire decoding or detection process, both the model architecture and internal parameters are completely invisible to the decoder. ● It means this can only be achieved only by dynamic watermarking.
- 11. White-box vs. black-box DNN extraction White-box (a) vs black-box (b) DNN watermark recovery
- 12. Static Watermarking algorithms Algorithm White/ Black box Multi/ Zero bit Methodology Robustness and Security Uchida et al. White Multi In the loss function, a regularization term is added so the watermark is embedded into the model weights. Moderate against fine-tuning and pruning. Li et al White Multi As Uchida’s scheme with ST-DM-like regularization term. Moderate against fine-tuning and pruning. DeepMarks White Multi As Uchida’s scheme with anti-collusion codebooks. Moderate against fine-tuning and pruning, Collusion attack. Tartaglione et al. White Zero Weights with watermarks remain frozen during training. In the loss function, the sensitivity of the network is maximized to changes in watermarked weights. Good robustness against fine- tuning and weights quantization.
- 13. Dynamic watermarking algorithms Algorithm White/ Black box Multi/ Zero bit Methodology Robustness and Security DeepSigns (Activation map) White Multi Adds arbitrary N-bit string to the probability density function of activation maps. Moderate against fine-tuning and pruning. DeepSigns (Output layer) Black Zero Build key image-label pairs from random selected images. Moderate against fine-tuning, pruning and overwriting. Yossi et al Black Zero Inject a backdoor by selecting random images into the target model. Moderate against fine-tuning Szyller et al. Black Zero Installed at the input and output of the target model's API, embeds dynamic watermarks in responses to queries made by the client. Regular surrogate model attack Merrer et al. Black Zero Adversarial attacks can be used to adjust the decision boundary of the target model. Parameter pruning, Overwriting via adversarial fine-tuning. Zhang et al. Black Zero Visible triggering patterns with backdoor-like mechanisms. Model fine-tuning, Parameter pruning, Model inversion attack. Guo et al. Black Zero Invisible triggering patterns with backdoor-like mechanisms. Model fine-tuning
- 14. Attacks on DNN Watermarking ● The attackers can take advantage of the observation that watermark embedding increases the variance of the weights, making it possible to distinguish a watermarked model from a non-watermarked one. ● Also the standard deviation of the weight increases linearly with the watermark dimension, allowing the attacker to estimate the length of the watermark or tell if it is present. ● This information is then used to replace the existing watermark with a new one, making the original watermark unreadable.
- 15. Conclusion - DNN watermarking is immune to vulnerabilities in the same way that any other watermarking solution is. - Challenge in providing robustness against: ● Fine-tuning ● Model-pruning ● Transfer learning. - DNNs are becoming increasingly popular with its human-like capabilities and considering the resources invested in its making, it is important that these advancements be protected, and from our reading we can be quite confident that watermarking is one of the reliable ways to achieve this goal.
- 16. Thank you!