Xin Wang

Biography

Xin Wang is a Project Associate Professor at the National Institute of Informatics and a PRESTO researcher at the Japan Science and Technology Agency. He was an organizing team member of the past ASVspoof Challenges since 2019 and the VoicePrivacy initiatives in 2020 and 2022. His research focuses on speech synthesis, anti-spoofing, and other speech security- and private-related tasks. He is a Member of IEEE, ISCA, and ASJ. He was a guest editor of the Computer Speech & Language Special Issue on Advances in Automatic Speaker Verification Anti-spoofing. He was on the appointed team of the ISCA Special Interest Group on Security and Privacy in Speech Communication from 2022 to 2024.

Career

Awards

• 2017 IEEE Signal Processing Society Japan, 11th IEEE Signal Processing Society Japan Student Best Paper Award
• 2018 SOKENDAI, SOKENDAI Award
• 2019 SSW Best Paper Award (SSW9 2016, test-of-time award)

Academic Activities

• 2022 – 2024 ISCA Special Interest Group: Security and Privacy in Speech Communication, appointed team member
• Member of IEEE, ISCA, and the Acoustical Society of Japan.

Recent Research

• MIDI-VALLE: Improving Expressive Piano Performance Synthesis Through Neural Codec Language Modelling
• LENS-DF: Deepfake Detection and Temporal Localization for Long-Form Noisy Speech
• SecureSpeech: Prompt-based Speaker and Content Protection
• Mitigating Language Mismatch in SSL-Based Speaker Anonymization
• A Comparative Study on Proactive and Passive Detection of Deepfake Speech
• From Sharpness to Better Generalization for Speech Deepfake Detection
• Towards a Transparent and Interpretable Strategy for Spoofed Speech Detection
• ASVspoof 5: Design, collection and validation of resources for spoofing, deepfake, and adversarial attack detection using crowdsourced speech
• Proactive Detection of Speaker Identity Manipulation with Neural Watermarking
• A Benchmark for Multi-speaker Anonymization
• Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesis
• Towards An Integrated Approach for Expressive Piano Performance Synthesis from Music Scores
• SpoofCeleb: Speech Deepfake Detection and SASV in the Wild
• Spoofing-Aware Speaker Verification Robust Against Domain and Channel Mismatches
• Exploring Active Data Selection Strategies for Continuous Training in Deepfake Detection
• A Preliminary Study on Long-Form In-the-Wild Audio Spoofing Detection
• ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations
• DDSP-based Neural Waveform Synthesis of Polyphonic Guitar Performance from String-wise MIDI Input
• Revisiting and Improving Scoring Fusion for Spoofing-aware Speaker Verification Using Compositional Data Analysis
• An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios
• Spoof Diarization: “What Spoofed When” in Partially Spoofed Audio
• To what extent can ASV systems naturally defend against spoofing attacks?
• Speaker Detection by the Individual Listener and the Crowd: Parametric Models Applicable to Bonafide and Deepfake Speech
• Malacopula: Adversarial Automatic Speaker Verification Attacks using a Neural-based Generalised Hammerstein Model
• ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale
• The VoicePrivacy 2022 Challenge: Progress and Perspectives in Voice Anonymisation
• SynVox2: Towards a privacy-friendly VoxCeleb2 dataset
• Can large-scale vocoded spoofed data improve speech spoofing countermeasure with a self-supervised front end?
• Collaborative Watermarking for Adversarial Speech Synthesis
• Spoofing Attack Augmentation: Can Differently-Trained Attack Models Improve Generalisation?
• Joint Speaker Encoder and Neural Back-end Model for Fully End-to-End Automatic Speaker Verification with Multiple Enrollment Utterances
• [Arxiv] Speaker-Text Retrieval via Contrastive Learning
• Speaker Anonymization using Orthogonal Householder Neural Network
• Towards single integrated spoofing-aware speaker verification embeddings
• Improving Generalization Ability of Countermeasures for New Mismatch Scenario by Combining Multiple Advanced Regularization Terms
• Range-Based Equal Error Rate for Spoof Localization
• ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild
• Can Knowledge of End-to-End Text-to-Speech Models Improve Neural MIDI-to-Audio Synthesis Systems?
• Spoofed training data for speech spoofing countermeasure can be efficiently created using neural vocoders
• Hiding speaker’s sex in speech using zero-evidence speaker representation in an analysis/synthesis pipeline
• Investigating Active-learning-based Training Data Selection for Speech Spoofing Countermeasure
• The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance
• Analyzing Language-Independent Speaker Anonymization Framework under Unseen Conditions
• Privacy and utility of x-vector based speaker anonymization
• Language-Independent Speaker Anonymization Approach using Self-Supervised Pre-Trained Models
• Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation
• Investigating Self-Supervised Front Ends for Speech Spoofing Countermeasures
• Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
• Estimating the Confidence of Speech Spoofing Countermeasure
• The VoicePrivacy 2020 Challenge: Results and Findings
• Benchmarking and Challenges in Security and Privacy for Voice Biometrics
• Multi-Task Learning in Utterance-Level and Segmental-Level Spoof Detection
• ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
• A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection
• An Initial Investigation for Detecting Partially Spoofed Audio
• Text-to-Speech Synthesis Techniques for MIDI-to-Audio Synthesis
• A Multi-Level Attention Model for Evidence-Based Fact Checking
• How Similar or Different Is Rakugo Speech Synthesizer to Professional Performers?
• End-to-End Text-to-Speech using Latent Duration based on VQ-VAE
• ASVspoof 2019: Spoofing Countermeasures for the Detection of Synthesized, Converted and Replayed Speech
• Denoising-and-Dereverberation Hierarchical Neural Vocoder for Robust Waveform Generation
• Investigation of learning abilities on linguistic features in sequence-to-sequence text-to-speech synthesis
• [Arxiv] Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis
• Using Cyclic Noise as the Source Signal for Neural Source-Filter-based Speech Waveform Model
• Design Choices for X-vector Based Speaker Anonymization
• Reverberation Modeling for Source-Filter-based Neural Vocoder
• Introducing the VoicePrivacy Initiative
• [Domestic conference] How close can Rakugo speech synthesis come to the performance of a human Rakugo performer?
• Modeling of Rakugo Speech and Its Limitations: Toward Speech Synthesis That Entertains Audiences
• Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentals
• ASVspoof 2019: a large-scale public database of synthesized, converted and replayed speech
• Effect of choice of probability distribution, randomness, and search methods for alignment modeling in sequence-to-sequence text-to-speech synthesis using hard alignment
• Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings
• Transferring Neural Speech Waveform Synthesizers to Musical Instrument Sounds Generation
• Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis
• [Arxiv] Transformation of low-quality device-recorded speech to high-quality speech using improved SEGAN model
• A Vector Quantized Variational Autoencoder (VQ-VAE) Autoregressive Neural F0 Model for Statistical Parametric Speech Synthesis
• Initial investigation of encoder-decoder end-to-end TTS using marginalization of monotonic hard alignments
• Speaker Anonymization Using X-vector and Neural Waveform Models
• Neural Harmonic-plus-Noise Waveform Model with Trainable Maximum Voice Frequency for Text-to-Speech Synthesis
• Rakugo speech synthesis using segment-to-segment neural transduction and style tokens — toward speech synthesis for entertaining audiences
• Joint Training Framework for Text-to-Speech and Voice Conversion Using Multi-Source Tacotron and WaveNet
• Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corpora
• ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
• [Domestic Conference] Robust Training Method for Rakugo Speech Synthesis Model and Dealing with Variations in Speaking Style
• [Domestic Conference] Initial Study on End-to-End Speech Synthesis without Soft Attention
• Audiovisual speaker conversion: jointly and simultaneously transforming facial expression and acoustic characteristics
• Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language
• Neural source-filter-based waveform model for statistical parametric speech synthesis
• STFT spectral loss for training a neural speech waveform model
• [Domestic Conference] Use of Tacotron and Context Features in Rakugo Speech Synthesis and Its Evaluation
• [Domestic Academic Conference] Investigating Neural Source-Filter Waveform Model for Statistical Parametric Speech Synthesis
• Fundamental Frequency Modeling for Neural-Network-Based Statistical Parametric Speech Synthesis
• [Domestic Conference] Rakugo Speech Synthesis Using WaveNet and Context Analysis - Toward Entertaining Speech Synthesis -
• [Domestic Conference] Towards Japanese End-to-End Speech Synthesis - Initial Study of Japanese Tacotron
• Investigating accuracy of pitch-accent annotations in neural network-based speech synthesis and denoising effects
• [Arxiv] Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• Can we steal your vocal identity from the Internet?: Initial investigation of cloning Obama’s voice using GAN, WaveNet and low-quality found data
• Autoregressive Neural F0 Model for Statistical Parametric Speech Synthesis
• Cyborg Speech: Deep Multilingual Speech Synthesis for Generating Segmental Foreign Accent with Natural Prosody
• Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks
• A Comparison of Recent Waveform Generation and Acoustic Modeling Methods for Neural-Network-Based Speech Synthesis
• [Domestic conference] Do prosodic manual annotations matter for Japanese speech synthesis systems with WaveNet vocoder?
• [Domestic Conference] Investigation of WaveNet for Text-to-Speech Synthesis
• [Domestic Academic Conference] Stealing Your Vocal Identity from the Internet: Cloning Obama’s Voice from Found Data Using GAN and WaveNet
• [Domestic Conference] Generating Segment-Level Foreign-Accented Synthetic Speech with Natural Speech Prosody
• Investigating Very Deep Highway Networks for Parametric Speech Synthesis
• [Domestic academic conference] Autoregressive quantized F0 modeling using a recurrent neural network with feedback links
• An RNN-based quantized f0 model with multi-tier feedback links for text-to-speech synthesis
• [NII Technical Report] A simple RNN-plus-highway network for statistical parametric speech synthesis
• An autoregressive recurrent mixture density network for parametric speech synthesis
• [Domestic academic conference] Investigation on an autoregressive recurrent mixture density network for parametric speech synthesis
• [Domestic academic conference] Investigating the impact of a neural network's depth on spectral and F0 modeling for parametric speech synthesis
• Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis
• The NII speech synthesis entry for Blizzard Challenge 2016
• Investigating Very Deep Highway Networks for Parametric Speech Synthesis
• A Comparative Study of the Performance of HMM, DNN, and RNN based Speech Synthesis Systems Trained on Very Large Speaker-Dependent Corpora
• Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech
• Using text and acoustic features in predicting glottal excitation waveforms for parametric speech synthesis with recurrent neural networks
• Enhance the word vector with prosodic information for the recurrent neural network based TTS system
• Speech Enhancement for a Noise-Robust Text-to-Speech Synthesis System using Deep Recurrent Neural Networks
• [Domestic academic conference] Using Continuous Representation of Various Linguistic Units for Recurrent Neural Network based TTS Synthesis