ai-music-generation

AI Music Generation

Overview

Two open-source AI music generation frameworks, each with different strengths:

Framework	Best for	Output	License
AudioCraft (Meta)	Text-to-music, text-to-sound, melody conditioning, audio codec	WAV/MP3, 32/16kHz	MIT
HeartMuLa	Lyrics+tags → full songs, Suno-like, multilingual	MP3, 48kHz stereo	Apache-2.0

Decision:

• AudioCraft when you need text prompts only, sound effects, melody conditioning, or stereo audio infrastructure
• HeartMuLa when you have specific lyrics and want a full song with tags (like Suno)

---

Section A: AudioCraft (MusicGen / AudioGen / EnCodec)

Quick Start

pip install audiocraft
# Or via HuggingFace transformers:
pip install transformers torch torchaudio

Text-to-Music (MusicGen)

import torchaudio
from audiocraft.models import MusicGen

model = MusicGen.get_pretrained('facebook/musicgen-medium')
model.set_generation_params(duration=8, top_k=250, temperature=1.0)

wav = model.generate(["happy upbeat electronic dance music"])
torchaudio.save("output.wav", wav[0].cpu(), sample_rate=32000)

Using HuggingFace Transformers:

from transformers import AutoProcessor, MusicgenForConditionalGeneration
import scipy

processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small").to("cuda")

inputs = processor(text=["80s pop with bassy drums"], padding=True, return_tensors="pt").to("cuda")
audio_values = model.generate(**inputs, do_sample=True, guidance_scale=3, max_new_tokens=256)
scipy.io.wavfile.write("output.wav", rate=model.config.audio_encoder.sampling_rate, data=audio_values[0, 0].cpu().numpy())

Model Variants

Model	Size	Use Case
musicgen-small	300M	Quick generation
musicgen-medium	1.5B	Balanced quality/speed
musicgen-large	3.3B	Best quality
musicgen-melody	1.5B	Melody conditioning
musicgen-stereo-*	Varies	Stereo output
musicgen-style	1.5B	Style transfer
audiogen-medium	1.5B	Sound effects