0
With smart:deess, we have taken the de-esser to new levels of capability. Here’s how artificial intelligence played a role in getting there.
smart:deess is our latest AI-powered audio plugin. Like smart:comp, smart:gate, smart:EQ and the rest of our smart line-up before it, smart:deess is a reinvention of a traditional audio processor.
In true sonible style, smart:deess takes advantage of our artificial intelligence expertise to achieve things that no other de-esser has before. We have totally reimagined how the de-esser is triggered with phoneme detection, and the way in which sibilance is removed from audio recordings.
We have also taken advantage of smart:deess’ unique phoneme detection algorithm to take the plugin beyond just de-essing. With smart:deess, producers have the power to remove plosives from audio recordings as well!
With phoneme detection under the hood, smart:deess is also able to bring life to dull recordings by “re-essing” – adding sparkle to your vocal takes.
So how exactly does artificial intelligence enable smart:deess to break new ground in the world of de-essing? Though AI is at the core of smart:deess’ superpowers, it’s not the only factor at play.
smart:deess also introduces spectral processing to the de-esser. This is a technology that you will find in several of our plugins including smart:comp 2 and smart:EQ 4, and it allows for precise audio shaping and editing which is unachievable with traditional audio processors.
In this section we will take a look at how artificial intelligence-based phoneme detection and spectral processing combine to help smart:deess achieve the previously impossible.
Traditional de-essers typically detect sibilance like a compressor. However, rather than having a threshold that can be triggered over the entire frequency spectrum, de-essers are only triggered in one frequency range.
The workflow is typically to set the frequency range of your de-esser (normally somewhere between 4 and 10 kHz), and then to set a threshold value. When the threshold is crossed in the set frequency range, the de-esser has detected sibilance and then compresses the signal.
However, this is not a 100% accurate way of detecting sibilance. It is possible for sibilant frequencies to not be detected by this method if they are not loud enough, or for loud phonemes that are not sibilant to trigger the de-esser.
smart:deess solves these problems by using artificial intelligence to detect sibilance. sonible’s algorithms are able to detect sibilant components of vocal recordings without having to set a threshold value or range.
smart:deess doesn’t just detect “ess” sounds, it can also identify Z, Sh, Ch, K, T, P. By actually detecting sibilance in a vocal recording, smart:deess can also process the sibilance from beginning to end, not just when it crosses the threshold as is the case with traditional de-essers.
Phoneme detection enables smart:deess to remove more than sibilance from recordings. The biggest benefit of phoneme detection is that smart:deess can detect plosives in vocal recordings and then remove them.
What are plosives? Plosives occur when the human mouth pronounces a “pee” or “bee” sound. In pronouncing these phonemes, a rush of air exits the mouth. This can hit the diaphragm of your microphone causing it to vibrate. This vibration is then picked up in audio recordings as a bassy rumble or sudden spike.
It is common practice to record vocals with a pop shield. Anyone who has any experience with audio engineering will be familiar with these handy tools. Often called a pop filter, they disperse these blasts of air before they hit the microphone diaphragm.
However, even with pop filters, plosives can make an unwanted appearance in vocal recordings. Therefore a tool like smart:deess which can detect and remove plosives is very useful.
Traditional de-essers are unable to detect plosives because plosives occur in a different area of the frequency spectrum to sibilance. Whilst sibilance – depending on the vocalist – typically occurs in the upper mid range of the frequency spectrum between 4kHz and 10kHz, plosives are found right at the bottom end of the spectrum between 20 and 500Hz.
Traditional de-essers are designed to only be triggered by loud frequencies in the specific sibilant range, and so plosives in the sub-500Hz range will pass through totally undetected. Because smart:deess uses AI to detect specific phonemes it doesn’t matter where in the frequency spectrum the phonemes occur. Therefore it is able to detect plosives as well.
Thanks to AI, smart:deess is able to accurately detect sibilance and other unwanted sounds such as plosives. But when it comes to removing these sounds from vocal recordings, smart:deess employs another bit of cutting-edge audio technology.
Spectral processing is a technology used by sonible in many smart plugins. You can think of spectral processing as incredibly high-resolution multiband processing. The frequency spectrum is divided up into thousands of bands, and each of these bands can be processed separately.
A traditional de-esser uses a broadband, or single band approach. Hence, when sibilance is detected, the entire frequency spectrum is compressed. This means that frequencies that are not harsh or unpleasant are removed from the recording along with the sibilant frequencies. This can result in unwanted tonal changes to the recording.
Spectral processing allows smart:deess to zero in on the frequencies where the sibilance is actually occurring and only reduce the volume of these frequencies. This is only possible thanks to AI phoneme detection which identifies where the sibilance is taking place.
For more insight into how exactly spectral processing works, read our article on How we do Spectral Processing at sonible.
smart:deess isn’t our first plugin to utilize AI to level up traditional audio plugins. Here’s how we’ve used artificial intelligence in our range of smart plugins to create next-generation audio processors.
Like smart:deess, smart:gate uses artificial intelligence to analyze incoming audio and detect wanted and unwanted components. smart:gate is able to identify instruments and vocals in recordings, retaining the component you want to hear, whilst removing any unwanted noise.
smart:comp 2 is a powerful, full-featured compressor that can be used to achieve transparent and professional compression without ever using the AI functionality. However, with artificial intelligence, smart:comp 2 can analyze an audio signal and – based on an instrument profile the user selects – apply optimum compression settings.
This is great for pros to establish a quick, great sounding starting point for compression processing, or for beginners to achieve professional compression without professional knowledge.
We’ve come a long way from the days of reverb chambers. These days reverb plugins are very advanced, but few can boast artificial intelligence powered source aware reverb like smart:reverb.
smart:reverb adapts to the spectral and temporal characteristics of your source material, enabling it to avoid muddy reverbs and unwanted resonant frequencies right at the start of your reverb process.
Like smart:comp 2, smart:EQ 4 can be used manually to achieve fantastic results. However, using AI you can achieve a balanced, source-dependent EQ curve in seconds if you want to.
smart:EQ 4 uses spectral processing to get an ultra-smooth and clear sound and cross-channel processing can give you exceptional results across up to ten channels.
smart:limit is designed to be set, checked, and left. This content-aware limiter sets limiting parameters based on the source material you feed it, giving loud and transparent limiting every time.
Don’t forget, if you want to try the next generation of AI powered de-essing for yourself, smart:deess is available now with a 30-day free trial.