Nabid In Motion Nabid In Motion

Study interactive :: Progress tools open in the Study Hub reader.

Audio and Speech Processing Quick Reference

Quick reference for audio processing, ASR, and TTS.

Audio Processing

Load Audio

import librosa
audio, sr = librosa.load('audio.wav', sr=22050)

Spectrogram

D = librosa.stft(audio)
magnitude = np.abs(D)
spectrogram = librosa.amplitude_to_db(magnitude)

Mel Spectrogram

mel_spec = librosa.feature.melspectrogram(y=audio, sr=sr, n_mels=128)
mel_spec_db = librosa.power_to_db(mel_spec)

MFCC

mfccs = librosa.feature.mfcc(y=audio, sr=sr, n_mfcc=13)

ASR

Whisper

from transformers import WhisperProcessor, WhisperForConditionalGeneration

processor = WhisperProcessor.from_pretrained("openai/whisper-base")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-base")
inputs = processor(audio, return_tensors="pt")
generated_ids = model.generate(inputs["input_features"])
transcription = processor.batch_decode(generated_ids)[0]

TTS

SpeechT5

from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech

processor = SpeechT5Processor.from_pretrained("microsoft/speecht5_tts")
model = SpeechT5ForTextToSpeech.from_pretrained("microsoft/speecht5_tts")
inputs = processor(text=text, return_tensors="pt")
speech = model.generate_speech(inputs["input_ids"], speaker_embeddings)

Libraries