Backbone Drum Synth Now Lets You Use Artificial Intelligence To Create New Drum Sounds

[embedded content]

Steinberg has announced the immediate availability of Backbone 1.5, an update to their drum re-synthesizer, that lets you create original drum sounds using artificial intelligence and Neural Audio Synthesis.

This person does not exist, but this photo is the output of a generative adversarial network.

Backbone 1.5 features DrumGAN, a recent development by Sony Computer Science Laboratories (Sony CSL), that uses Generative Adversarial Networks to generate original drum sounds. Available samples can also be analyzed to create similar versions.

A generative adversarial network (GAN) is a machine learning approach, in which two neural networks are used to improve the accuracy of artificially-generated output, with the goal of making the artificially-generated output indistinguishable from the source material that the model is trained on.

Well known examples of GANs can generate photorealistic portraits of peoplecute cat pics and more.

“The cooperation between Sony CSL and Steinberg is a great example of how AI can be used in music production,” notes Steinberg’s Florian Haack. “The ability to generate new samples with DrumGAN for kicks, snare or cymbals, or analyze existing samples and further re-synthesize and
reassemble the components, gives producers unprecedented flexibility to design new
sounds.”

Here’s a look at the technology behind DrumGAN:

[embedded content]

Availability and Pricing

Backbone is available through the Steinberg Online Shop for 149 euros or 149.99 US dollars. With the release of version 1.5, Backbone is currently available at a 40% special offer discount until the end of July 2022. The update to version 1.5 is free for users who already purchased Backbone

Designing & Building An AI Musical Instrument

[embedded content]

Charis Cat shared this in-depth look at creating the After Eight Step sequencer – a DIY project that extends traditional step sequencing with artificial intelligence.

The After Eight Step is an Arduino and Max-powered 8-step sequencer. It uses Markov Chains to generate musical accompaniment, based on any sequence of notes. Details on this are available on Cat’s site.

If you’re interested in creating an After Eight Step sequencer, source for the project is available on github.

Topics covered:

00:00 Introduction
01:18 Why am I doing this?
02:11 Let’s get building!
03:22 First row of Potentiometers
03:40 Potentiometers to MIDI
04:22 Sequencing in Max
05:17 Mounting the LCD Screen
05:50 Wiring potentiometers
06:12 Preparing the LEDs
07:30 Second potentiometer row
07:51 Markov Chains and Transition Matrix’s
08:39 Rotary Encoder & Settings
09:17 AI Attempt 1, Troubleshooting, & David Cope
10:51 Wiring the potentiometers
11:12 Multiplexing
13:05 Rotary Encoders & Interrupt Pins
14:06 Clicky Green Button
14:17 Wiring up the LCD Screen
14:44 Literal Explosions
15:24 Tearing the heart from an old project
15:47 Rewiring the Arduino
16:26 Conclusion
18:47 Demonstration
21:34 Pure Pain

iZotope Intros AI-Powered Neoverb ‘Intelligent Reverb’

[embedded content]

iZotope has introduced Neoverb, a new AI-powered ‘intelligent reverb’ effects for Mac & Windows.

Features:

  • Reverb Assistant: With an easy four-step assistant that adjusts iZotope Neoverb’s settings in real time, users can get a great reverb for any track and create complex, customized reverbs,.
  • Blend Pad: Visualize and mix three different types of reverbs at once.
  • EQ Section: This feature helps clean the input signal to keep a reverb tail from overwhelming a mix, to get great reverb sound without using a long plug-in chain.
  • Advanced Panel: Allows users to dive as deep as they need to find the exact vibe they are looking for, while Advanced Controls set by Reverb Assistant that can be saved as unique presets for later use.
  • Input Smoothing: Keep dynamics even and avoid artifacts in the reverb tail with one simple control that reduces the sharp, percussive transients.
  • Modulation (Mod): A modulation panel that gives even more control over the reverb tail. Two different modes offer subtle modulation to avoid buildup or a classic chorus for a more pronounced mixing effect.
  • Resizable: Window resizability to fit any screen.
  • Presets: Includes over 100 presets, including presets for vocals, guitars, experimental spaces and more.

Pricing and Availability:

Neoverb is available now for $199.

Factorsynth 2 Features AI-Powered Tools For Resynthesizing Audio

[embedded content]

Isotonik Studios has released Factorsynth 2, a major update to the AI-powered Max For Live device for analyzing and resynthesizing audio.

Factorsynth is a Max For Live device that uses a data analysis algorithm called matrix factorization to decompose any audio clip into a set of temporal and spectral elements.

By rearranging and modifying these components, you can create powerful transformations in your audio clips, such as removing notes or motifs, creating new ones, randomizing melodies or timbres, changing rhythmic patterns, remixing loops in real time, applying effects selectively only to certain elements of the sound, creating complex sound textures and more.

Factorsynth 2 offers a simplified workflow and numerous new features. It is now possible to individually pan the components, which means you can do things like upmixing a mono clip to stereo. Another powerful new feature is quantized shifting of the components, which lets you change the rhythmic structure of riffs and drum loops.

Pricing and Availability

Factorsynth 2 is available now for $61.66 USD.

djay For iOS Adds AI-Powered Real-Time ‘Unmixing’

[embedded content]

Algoriddim has released version 3.5 of djay Pro, its iOS DJ application.

The update introduces Neural Mix, an artificial intelligence feature that can automatically ‘unmix’ tracks into vocal, drum and instrumental stems in realtime, introducing new options for live remixing, including vocal removal, vocal isolation and more.

Here is what’s new in djay Pro 3.5:

  • Neural Mix: isolate vocals, drums, instrumentals
  • AI multi-track waveforms: see vocals, drums, instruments side by side
  • UI refresh
  • Enhanced Automix AI with Neural Mix transition
  • New dark mode for waveforms in 2 Deck mode
  • Sequence Recorder – now accessible in Classic Mode
  • New full screen waveforms in 2 Deck mode and expanded screen in Video mode (simply toggle the library/mixer icon to show large HD waveforms and video)
  • Enhanced Accessibility/VoiceOver integration for Neural Mix
  • Enhanced support for Magic Keyboard
  • Numerous new features, fixes, and improvements

Note: Neural Mix is processor-intensive, so Algoriddim recommends using an iOS device with an A12 Bionic chip or later.

Pricing and Availability:

djay Pro is available now as free download, but full access is priced at $39.99 USD.

New Site Uses Artificial Intelligence To Create Yamaha DX7 Synth Patches So You Don’t Have To

The Yamaha DX7 is one of the most popular synthesizers ever made, because it offered an unprecedented combination of polyphony, playability and pricing when it was introduced.

It also earned a reputation as one of the least user-friendly synths to create patches on, because the DX7 used an approach to synthesis that was new to keyboardists at the time it was introduced and editing was slowed by a menu-heavy interface.

While many synthesists have taken the time to learn DX style FM programming in the subsequent years, a new website promises to make creating custom patch banks faster than ever before.

The site, This DX7 Cartridge Does Not Exist, is fount of DX patches. It uses an artificial intelligence model, trained on a collection of hand-crafted DX7 patches, to create synth patches on the fly.

Here’s what the developer has to say about it:

“The Yamaha DX7 has a total of 155 parameters, the AI uses a machine learning model known as a VAE that reduces these 155 parameters down to just 8.

The model can then choose these 8 numbers randomly and map them back to the full 155 parameters of the DX7, it has learned to do this by training on a large selection of presets compiled by Bobby Blue

Each cartridge is generated in real time by sampling 32 sets of the 8 VAE parameters and compiling them together in real time. The model generates 99.9% unique voices and so it is highly unlikely that any two cartridges created on this site will be the same.”

The site generates a MIDI .syx that you can download and send to your DX7. It’s also compatible with Dexed for Linux, OS X and Windows and Rockrelay Synth FM for Android.

New Site Uses Artificial Intelligence To Create Yamaha DX7 Synth Patches So You Don’t Have To

The Yamaha DX7 is one of the most popular synthesizers ever made, because it offered an unprecedented combination of polyphony, playability and pricing when it was introduced.

It also earned a reputation as one of the least user-friendly synths to create patches on, because the DX7 used an approach to synthesis that was new to keyboardists at the time it was introduced and editing was slowed by a menu-heavy interface.

While many synthesists have taken the time to learn DX style FM programming in the subsequent years, a new website promises to make creating custom patch banks faster than ever before.

The site, This DX7 Cartridge Does Not Exist, is fount of DX patches. It uses an artificial intelligence model, trained on a collection of hand-crafted DX7 patches, to create synth patches on the fly.

Here’s what the developer has to say about it:

“The Yamaha DX7 has a total of 155 parameters, the AI uses a machine learning model known as a VAE that reduces these 155 parameters down to just 8.

The model can then choose these 8 numbers randomly and map them back to the full 155 parameters of the DX7, it has learned to do this by training on a large selection of presets compiled by Bobby Blue

Each cartridge is generated in real time by sampling 32 sets of the 8 VAE parameters and compiling them together in real time. The model generates 99.9% unique voices and so it is highly unlikely that any two cartridges created on this site will be the same.”

The site generates a MIDI .syx that you can download and send to your DX7. It’s also compatible with Dexed for Linux, OS X and Windows and Rockrelay Synth FM for Android.

New App, Drum Replacer, Uses Artificial Intelligence To Improve Your Drum Tracks

[embedded content]

UVI has introduced Drum Replacer, a realtime, program-adaptive, drum replacement and processing utility that uses artificial intelligence to help you improve your drum tracks.

With Drum Replacer, you can remove bleed from drum recordings, replace drums in a mix with your favorite samples or VSTi, add sub-bass, layer existing drum sounds and more.

Features:

  • Machine learning-based analysis with realtime component separation and triggering
  • Trigger internally-hosted samples and VSTi, with browser, preview, and favorites
  • Positive and negative offset allows pre-triggering, gain and pitch matching

Here’s an in-depth demo of how it works:

[embedded content]

Pricing and Availability

Drum Replacer is available now with an intro price of $59 USD (normally $99).

Speech Synthesis Enters The Uncanny Valley, Or ‘What Will Biggie Rap Next?’

[embedded content]

Speech synthesis, the use of computers to generate realistic human speech, is rapidly entering the ‘uncanny valley’ – creepily almost-realistic.

Recent approaches have used neural networks, trained using only speech examples and text transcripts, to generate human-like text-to-speech synthesis. In the example embedded above, the late Christopher George Latore Wallace (May 21, 1972 – March 9, 1997), aka The Notorious B.I.G., raps The Book of Genesis.

The ‘voice’ voice was computer-generated, using a text-to-speech model trained on the speech patterns of The Notorious B.I.G. In a nutshell, the approach uses an AI to ‘learn’ how an audio file of an individual’s speech compares to a text transcript. Once trained, the model can synthesize speech from text that conforms to the ‘learned’ speech patterns.

The Vocal Synthesis channel on Youtube features a wide range of examples that demonstrate what’s currently possible. The introduction to the channel features the voices of six presidents:

[embedded content]

Bill Clinton reads Baby Got Back by Sir Mix-A-Lot, embedded below, is an example that highlights both the entertaining and troubling potential of the technology:

[embedded content]

“In an evaluation where we asked human listeners to rate the naturalness of the generated speech, we obtained a score that was comparable to that of professional recordings.

While our samples sound great, there are still some difficult problems to be tackled. For example, our system has difficulties pronouncing complex words (such as “decorum” and “merlot”), and in extreme cases it can even randomly generate strange noises. Also, our system cannot yet generate audio in realtime. Furthermore, we cannot yet control the generated speech, such as directing it to sound happy or sad. Each of these is an interesting research problem on its own.”

What Will Biggie Rap Next?

Biggie has been gone for more than two decades, but this technology is being used to make him rap from beyond the grave. The results are realistic enough that artists and agents are taking notice.

For example, in April 2020, Youtube temporarily took down this video of Jay-Z rapping the To Be, Or Not To Be soliloquy from Hamlet:

[embedded content]

Jay-Z’s agency, Roc Nation LLC, claimed that the video “unlawfully uses an AI to impersonate our client’s voice.”

The legality of unauthorized speech synthesis is still to be determined, and could depend on how the audio is used. Some uses may fall into the category of fair use and others may infringe on copyright or an artist’s publicity rights.

The anonymous creator of the Vocal Synthesis channel recognizes that the channel is in ‘legally uncharted waters’, but suggests that it is important to explore the creative possibilities of AI-generated speech:

“I wanted to show that synthetic media doesn’t have to be exclusively made for malicious/evil purposes, and I think there’s currently massive amounts of untapped potential in terms of fun/entertaining uses of the technology.

I think the scariness of deepfakes and synthetic media is being overblown by the media, and I’m not at all convinced that the net impact will be negative, so I hoped that my channel could be a counterexample to that narrative.”

Check out the examples and share your thoughts on AI speech synthesis, and its creative possibilities, in the comments!

Artificial Intelligence Now Synthesizing Complete Songs In The Style Of Elvis, Frank Sinatra, Katy Perry

Faux Katy Perry is counting every moment since she said ‘Goodbye’….

San Francisco research lab OpenAI has introduced Jukebox, a neural net that synthesizes music in a variety of artists’ styles, including rudimentary singing.

They’ve trained the AI with a wide range of artists’ music, including Elvis, Frank Sinatra, Katy Perry and others. The AI directly synthesizes audio, with instruments and faux-vocals.

The results are terrible and awesome, sounding like music from another universe, captured via a vintage tube radio.

Some of the tracks seem like plausible bad songwriting, like faux-Katy Perry singing:

“I count every moment, every hour since I said goodbye.
I count every minute every hour, since your lips were touching mine.
I count every minute, every hour, hoping I’m the one you want.”

Pop, in the style of Katy Perry:

Other tracks seem like something out of a David Lynch dream sequence, like a drunken Frank Sinatra singing “It’s Christmas time, and you know what that means. Its hot tub time!”

“Its hot tub time!”

All this needs to be a warped holiday classic is a deep-fake video of Sinatra crooning “It’s hot tub time!” with a martini in his hand.

In addition to synthesizing tracks with original lyrics, the AI can create disturbing new music based on classic lyrics.

You may know Marni Nixon’s upbeat version of I Whistle a Happy Tune, from the musical The King & I.

The upbeat lyrics sound completely different, though, with AI-generated music and vocals.

Here’s faux drunk Sinatra with his take on I Whistle a Happy Tune:

The researchers have gone further, creating tracks combine models, so ?0.25 Ella Fitzgerald + 0.75 Frank Sinatra can sing an original version of City Of Stars:

Jukebox is a bizarre rabbit hole to fall down – but also a preview of the insane possibilities of a new type of synthesis.

Check out the examples and let us know what you think of them in the comments!