Sound field analysis and synthesis are fundamental techniques in spatial acoustic signal processing, which are aimed at estimating/synthesizing an acoustic field by a discrete set of microphones/loudspeakers. These techniques are essential in visualization/auralization of a sound field, VR/AR audio, creating personal sound zones, canceling noise in a regional space, and so forth. Conventional techniques are largely based on boundary integral representations of the Helmholtz equation, such as Kirchhoff-Helmholtz and Rayleigh integrals. In recent years, machine learning techniques incorporating characteristics of acoustic fields, which are referred to as wavefield-based machine learning (WBML), have evolved in this research field. WBML has the potential to further enhance the performance and applicability of sound field analysis and synthesis. In this overview talk, we will introduce these recent advancements. Specifically, kernel methods for sound field estimation and their application to spatial audio reproduction will be highlighted.

Along with the rapid advancement of its technologies and expanding abilities, new applications of unmanned aerial vehicles (UAV, a.k.a. drones) have been actively explored over the last decade. One of such emerging applications of UAV is its use for recording sound, i.e. equipping UAVs with “auditory” function, which has a huge potential to deliver both commercial and societal benefits through various industries and sectors, such as filming/broadcasting, monitoring/surveillance, and search/rescue. However, a challenge with the UAV audition is the extensive amount of noise generated by the UAV’s propellers, known as ego noise, significantly deteriorating the quality of sound recorded on UAV. Research in signal processing for better UAV audition has been actively conducted to achieve better auditory function by addressing this challenge. This talk will overview recent studies on audio signal processing for UAV audition, with a particular focus on techniques to emphasise sound from the target source while minimising the propeller noise. The talk will also introduce a case study where such technology is applied to commercial products.

Blind source separation (BSS) for audio signals is a technique to extract specific audio sources from an observed mixture signal. In particular, multichannel determined BSS has been studied for many years because of its capability: the separation can be achieved by a linear operation (multiplication of a demixing matrix) and the quality of estimated audio sources is much better than that of other non-linear ASS algorithms. Determined BSS algorithms have its roots in independent component analysis (ICA), which assumes the independence among sources and estimates the demixing matrix. Then, ICA was extended to independent low-rank matrix analysis (ILRMA) by introducing a low-rank time-frequency structure model for each source. With the advent of ILRMA, the combination of “demixing matrix estimation for linear BSS” and “time-frequency structure models for each source” has become a reliable approach for audio BSS problems. In this talk, we focus on a new flexible BSS algorithm called time-frequency-masking-based BSS (TFMBSS). In this method, thanks to a model-independent optimization algorithm, arbitrary time-frequency structure models can easily be utilized to estimate the demixing matrix in a plug-and-play manner. In addition to the theoretical basis of this algorithm, some TFMBSS applications combining group sparsity, harmonicity, or smoothness in the time-frequency domain will be reviewed.

Voice communication and human-machine speech interaction systems are facing more and more challenging application environments where besides the speech signals of interest there is coexistence of noise, reverberation, echo, and interference. How to acquire high-fidelity speech signals in such complicated acoustic environments is a very challenging problem, which involves the use of microphone arrays and many multichannel acoustic signal processing techniques. In this talk, I will present a brief overview of the basic problems and principles of sensing and processing of speech signals. I will then focus on discussing important challenges faced by teleconferencing, audio-bridging, and human-machine interface systems. I will elaborate, using examples, on how to design microphone arrays and beamforming algorithms to achieve noise reduction and interference suppression, thereby extracting speech signals of interest in noisy and reverberant acoustic environments.

In this talk, Dr. Sisman will introduce the fundamentals of emotion in speech synthesis, and the recent advancements in the field through live demonstrations. She will present her technical contributions that cover both emotional voice conversion and deep learning solutions to high quality speech synthesis. She will also provide my perspectives on the technology challenges and future directions moving forward.

We are surrounded by various kinds of sounds such as speech, music, and environmental sounds. To understand these sounds by computers and to realize human-like listening systems, environmental sound analysis is essential technology, and they have been extensively developed. In this talk, we review the fundamentals of environmental sound analysis, including its problem definitions (e.g., acoustic scene analysis, sound event detection, and anomalous sound detection), available public datasets, applications, and challenges. We also introduce recent efforts in the environmental sound analysis, including various deep learning-based methods, performance evaluation methods, and our recent works, and discuss the future research directions in this research area.

Traditionally, data hiding is realized at the expense of slight quality degradation in the host content. This also applies to reversible methods because the modified content (now containing the inserted data) is inevitably distorted. Although the introduced distortion may not be noticeable to the naked eyes, a direct comparison between the original and modified contents will instantly reveal the differences. Recently, our research group put forward a few proposals to hide data while completely preserving the quality of the host content. Essentially, the quality of the content will be exactly the same before and after data insertion. In general, the quality preserving property is achieved by using two representations to render / encode the same entity, where one representation can be associated with `0’, while another associated with `1’. In addition, the proposed techniques are reversible and the hiding capacity is scalable. In this overview, two proposed techniques, specifically one for animated GIF and another for PDF file, will be presented, followed by some discussions on potential future research directions.

A secure robust JPEG steganographic scheme based on an autoencoder with an adaptive BCH encoding is proposed, which can resist JPEG compression channel interference in secret messages. In the proposed scheme, the autoencoder is first pretrained to fit the transformation relationship between the JPEG image before and after compression by the compression channel, which makes the autoencoder model fit the inverse procedure of the JPEG compression channel so that the model can generate an intermediate image that resists the JPEG compression channel. Then, the BCH encoding is adaptively utilized according to the content of cover image to decrease the error rate of secret message extraction. Finally, the DCT coefficient adjustment based on practical JPEG channel characteristics further improves the robustness and statistical security. Experimental results demonstrate that the proposed robust JPEG steganographic algorithm can provide a more robust performance and statistical security than prior state-of-the-art JPEG steganographic schemes.

Watermarking is the art of hiding data in multimedia content in a robust manner, while keeping the imperceptibility of the hidden data. In the early studies of digital watermarking, the security received little attention, yet was later shown to be as important as the robustness and the imperceptibility. In this talk, I will first introduce the concept and requirements of watermarking security. Then, I will review some of the state-of-the-art watermarking security theories and algorithms, and discuss their merits and limitations. By analyzing the limitations of existing methods, I will outline the major challenges in watermarking security, which provide some future research directions on this topic.

Image representation learning is one of the most fundamental problems in computer vision with a broad range of applications, including object recognition and action recognition. Many recent studies on self-supervised learning have shown that the need for labeled images can be significantly reduced if a large number of unlabeled images is available for pre-training. However, a major effort is still required to collect data, and to resolve the dataset bias issue during the data collection process. In this overview talk, I will present a series of our recent work on pre-training without natural images, which aims to learn image representations from automatically generated images. Specifically, this talk covers pre-training algorithms, in which neural networks learn to classify procedural patterns such as fractals, color tiles, and Perlin noise. This talk will also include a review of recent research on self-supervised learning and discussions on future research directions.

Second order optimization methods require the computation of Hessian, Gauss-Newton, or Fisher information matrices, which results in an intractable computational cost when done naively. Such matrices are not only used for optimization, but also for generalization metrics, continual learning, structured pruning, and gradient-based hyperparameter optimization. We show that these matrices can be computed using backpropagation and that various approximation methods exist that can accelerate their computation significantly. Furthermore, when training on distributed systems, the overhead of computing these matrices can be reduced significantly.

With the development of deep learning, the market for an edge AI including embedded systems is expected to expand in the market point of view. Edge AIs need to process a large number of operations on limited　computational and power resources. Data structures and architectures have been researched and developed. Previous studies have proposed networks as various data structures. We introduce various networks that are effective for hardware implementation. Next, we describe a weight reduction method for hardware implementation. It is a quantization in low bits. High speed can be achieved while reducing the amount of dedicated hardware by quantization techniques. However, the recognition accuracy deteriorates, so we consider the optimization method. Next, we show a weight sparseness that approximates the weight to 0. It also exits a trade-off with accuracy, deterioration and acceleration, so we introduce the optimization method. Finally, we describe three types of hardware implementations. The implementation results will be demonstrated targeting an FPGA as a prototype of deep learning applications.