Spaces:
Running
Running
AndrΓ© Oliveira commited on
Commit Β·
81aeeb4
1
Parent(s): 722b473
performance boost
Browse files- README.md +13 -8
- app.py +59 -54
- draw/vision.py +17 -6
- synths/guitar.py +22 -10
- synths/harp.py +18 -11
- synths/voice.py +5 -1
- synths_/__init__.py +0 -0
- synths_/guitar.py +393 -0
- synths_/harp.py +381 -0
- synths_/musicbox.py +320 -0
- synths_/ocarina.py +197 -0
- synths_/piano.py +234 -0
- synths_/voice.py +170 -0
- synths_/xylophone.py +298 -0
README.md
CHANGED
|
@@ -41,11 +41,13 @@ Built for the [Hugging Face **Build Small Hackathon 2026**](https://huggingface.
|
|
| 41 |
|
| 42 |
## The Problem
|
| 43 |
|
| 44 |
-
|
| 45 |
|
| 46 |
-
|
| 47 |
|
| 48 |
-
|
|
|
|
|
|
|
| 49 |
|
| 50 |
## How it works
|
| 51 |
|
|
@@ -62,7 +64,7 @@ Drawing AND typed loves? β’ Both inform the song.
|
|
| 62 |
| Component | Model / Library | Where it runs |
|
| 63 |
| ---------------- | -------------------------------------------------- | ------------- |
|
| 64 |
| Lyric generation | **Llama 3.2 3B**, fine-tuned, via `llama.cpp` | CPU, locally |
|
| 65 |
-
| Drawing β words | **MiniCPM-V 4.6** (1.3B) via `transformers` |
|
| 66 |
| Stroke fallback | Pure NumPy color/density analysis | CPU, locally |
|
| 67 |
| Singing voice | **Kokoro 82M** | CPU, locally |
|
| 68 |
| Instruments | Custom DSP synths, built from spectral analysis | CPU, locally |
|
|
@@ -72,10 +74,13 @@ Drawing AND typed loves? β’ Both inform the song.
|
|
| 72 |
|
| 73 |
## Hardware
|
| 74 |
|
| 75 |
-
Lolaby runs locally on whatever machine you give it β a laptop, a CPU-only Hugging Face Space
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
## Badges
|
| 81 |
|
|
|
|
| 41 |
|
| 42 |
## The Problem
|
| 43 |
|
| 44 |
+
Getting a small child to fall asleep is a daily battle for parents and anyone who looks after kids.
|
| 45 |
|
| 46 |
+
My partner's sister teaches kindergarten. Every day she runs nap time for fifteen 4-year-olds, and ever since they learned about music and instruments in class, it starts the same way: *"sing me a song with my name in it."* She'd love to give each child their own song, built from whatever they love that week β a stuffed fox, a new puppy, the rainbow. She doesn't have the time, the musical training, or a tool that could do it.
|
| 47 |
|
| 48 |
+
**Lolaby is that tool.** The child shows Lola what they love β doodling on screen, or handing over a paper drawing for the teacher to photograph. The teacher types the child's name. A small, on-device AI looks at the drawing, writes a lullaby about it, and sings it back β in about a minute.
|
| 49 |
+
|
| 50 |
+
Everything runs locally. No cloud LLM, no per-song API cost, no child's drawing or name ever leaving the device.
|
| 51 |
|
| 52 |
## How it works
|
| 53 |
|
|
|
|
| 64 |
| Component | Model / Library | Where it runs |
|
| 65 |
| ---------------- | -------------------------------------------------- | ------------- |
|
| 66 |
| Lyric generation | **Llama 3.2 3B**, fine-tuned, via `llama.cpp` | CPU, locally |
|
| 67 |
+
| Drawing β words | **MiniCPM-V 4.6** (1.3B) via `transformers` | CPU, locally |
|
| 68 |
| Stroke fallback | Pure NumPy color/density analysis | CPU, locally |
|
| 69 |
| Singing voice | **Kokoro 82M** | CPU, locally |
|
| 70 |
| Instruments | Custom DSP synths, built from spectral analysis | CPU, locally |
|
|
|
|
| 74 |
|
| 75 |
## Hardware
|
| 76 |
|
| 77 |
+
Lolaby runs locally on whatever machine you give it β a laptop, or a CPU-only Hugging Face Space. There's no cloud LLM in the loop at runtime: the lyric model, the vision model, and the audio synthesis all run on-device.
|
| 78 |
+
|
| 79 |
+
The whole pipeline is CPU-only by design. The fine-tuned Llama 3.2 3B runs as a Q4_K_M GGUF through llama.cpp; the MiniCPM-V vision model and Kokoro TTS run in-process on CPU. Nothing is offloaded to a GPU or an external API, so the experience is identical wherever it runs β no model is skipped and no feature degrades depending on the host.
|
| 80 |
+
|
| 81 |
+
If the vision model can't load for any reason, the app falls back to a NumPy stroke-and-colour analyzer so songs keep generating rather than breaking.
|
| 82 |
+
|
| 83 |
+
This portability is intentional: the same repo can be forked and run anywhere β a laptop, a free CPU Space, an offline box β without changing a line of code. That's the "build small" idea taken literally: a complete, personalised lullaby pipeline that fits on modest hardware and owes nothing to the cloud at runtime.
|
| 84 |
|
| 85 |
## Badges
|
| 86 |
|
app.py
CHANGED
|
@@ -11,39 +11,6 @@ The user picks both layers with image buttons in the UI.
|
|
| 11 |
Aesthetic: children's drawing β crayon textures, wobbly hand-drawn borders.
|
| 12 |
"""
|
| 13 |
|
| 14 |
-
# ZeroGPU requirement: `spaces` must be imported BEFORE any CUDA/torch-touching
|
| 15 |
-
# library, and at least one @spaces.GPU-decorated function must be registered at
|
| 16 |
-
# import time, or the Space aborts at startup with:
|
| 17 |
-
# "No @spaces.GPU function detected during startup"
|
| 18 |
-
# Our real GPU function (describe_with_vision) lives in draw/vision.py, which is
|
| 19 |
-
# imported below behind a try/except β if that import ever fails, ZeroGPU would
|
| 20 |
-
# see zero decorated functions. So we import spaces first and register a tiny
|
| 21 |
-
# top-level probe here that ALWAYS exists. It's never called; it just guarantees
|
| 22 |
-
# the startup scan succeeds regardless of what happens to the vision import.
|
| 23 |
-
try:
|
| 24 |
-
import spaces
|
| 25 |
-
_HAVE_SPACES = True
|
| 26 |
-
except ImportError:
|
| 27 |
-
# Not on a ZeroGPU Space (e.g. local/laptop or a CPU Space). Provide a no-op
|
| 28 |
-
# decorator so the probe definition below is harmless. Mirrors HF's behavior.
|
| 29 |
-
class _SpacesShim:
|
| 30 |
-
def GPU(self, *args, **kwargs):
|
| 31 |
-
if len(args) == 1 and callable(args[0]) and not kwargs:
|
| 32 |
-
return args[0]
|
| 33 |
-
def deco(fn):
|
| 34 |
-
return fn
|
| 35 |
-
return deco
|
| 36 |
-
spaces = _SpacesShim()
|
| 37 |
-
_HAVE_SPACES = False
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
@spaces.GPU(duration=10)
|
| 41 |
-
def _zerogpu_startup_probe():
|
| 42 |
-
# Never invoked. Exists only so ZeroGPU's startup scan always finds a
|
| 43 |
-
# @spaces.GPU function, even if the vision module import fails.
|
| 44 |
-
return True
|
| 45 |
-
|
| 46 |
-
|
| 47 |
import base64
|
| 48 |
import glob
|
| 49 |
import os
|
|
@@ -80,9 +47,7 @@ except ImportError:
|
|
| 80 |
#
|
| 81 |
# We catch Exception (not just ImportError) so a heavy transitive failure
|
| 82 |
# doesn't crash the whole app, but we PRINT the traceback β a silent swallow
|
| 83 |
-
# here previously hid the real reason the vision module wasn't loading
|
| 84 |
-
# Space. The top-level _zerogpu_startup_probe above still guarantees ZeroGPU's
|
| 85 |
-
# startup scan passes even if this import fails.
|
| 86 |
try:
|
| 87 |
import draw.vision as vision # vision.describe(image) -> {"loves": str, ...}
|
| 88 |
except Exception as _vision_import_err:
|
|
@@ -135,18 +100,35 @@ if SKIP_LLM:
|
|
| 135 |
else:
|
| 136 |
print(f"Loading {MODEL_REPO}...")
|
| 137 |
try:
|
| 138 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
repo_id=MODEL_REPO,
|
| 140 |
filename="*Q4_K_M.gguf",
|
| 141 |
n_ctx=1024,
|
| 142 |
-
n_threads=
|
| 143 |
-
|
|
|
|
|
|
|
| 144 |
chat_format="llama-3",
|
| 145 |
verbose=False,
|
| 146 |
)
|
| 147 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
except Exception as e:
|
| 149 |
-
# If the GGUF is missing or won't load, don't take the whole
|
| 150 |
# down with a cryptic import-time crash. Leave llm=None; the UI still
|
| 151 |
# loads and make_lullaby surfaces a friendly message.
|
| 152 |
print(f"WARNING: could not load model at {MODEL_REPO}: {e}")
|
|
@@ -156,7 +138,7 @@ else:
|
|
| 156 |
# ---------------------------------------------------------------------------
|
| 157 |
# Pre-warm everything else at app startup, so the first user click is fast.
|
| 158 |
#
|
| 159 |
-
# Honest tradeoff: pre-warming makes COLD STARTUP slower (the
|
| 160 |
# longer to come online) in exchange for the FIRST USER GENERATION being
|
| 161 |
# much faster β which is the moment that actually matters for a judge who
|
| 162 |
# clicks the link and waits to see something happen.
|
|
@@ -164,19 +146,36 @@ else:
|
|
| 164 |
# What we pre-warm and why:
|
| 165 |
# - Llama lyric model: already eager-loaded above.
|
| 166 |
# - Kokoro TTS: ~340 MB voice model + downloads. ~5-10s saved on first
|
| 167 |
-
# generation.
|
| 168 |
# - MiniCPM-V vision: ~3 GB download (first run ever) + model load
|
| 169 |
-
# (~10-30s on CPU).
|
| 170 |
-
#
|
| 171 |
-
# decorator at call time, so we never hold a GPU here. Saves ~20-30s
|
| 172 |
-
# on first user click.
|
| 173 |
#
|
| 174 |
# Both pre-warms are wrapped in try/except so a failure during warmup
|
| 175 |
-
# doesn't crash the
|
| 176 |
# still work if warmup fails for any reason.
|
| 177 |
# ---------------------------------------------------------------------------
|
| 178 |
|
| 179 |
if not SKIP_LLM:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
try:
|
| 181 |
print("Pre-warming Kokoro TTS voice...")
|
| 182 |
# warmup() (not just _load_kokoro()) so the af_nicole.pt voice tensor
|
|
@@ -192,8 +191,7 @@ if not SKIP_LLM:
|
|
| 192 |
try:
|
| 193 |
print("Pre-warming MiniCPM-V vision (CPU load only)...")
|
| 194 |
# Triggers _try_load() inside vision.py, which downloads + loads the
|
| 195 |
-
# model into
|
| 196 |
-
# inside the @spaces.GPU-decorated describe_with_vision call.
|
| 197 |
import draw.vision as _vision_warmup
|
| 198 |
_vision_warmup._try_load()
|
| 199 |
print("Vision ready.")
|
|
@@ -375,21 +373,28 @@ def build_prompt(name, age, loves, fears, mood, key, meter):
|
|
| 375 |
return "\n".join(lines)
|
| 376 |
|
| 377 |
|
| 378 |
-
def generate_lullaby(prompt, temperature=0.75):
|
| 379 |
if SKIP_LLM or llm is None:
|
| 380 |
lola_trace.stage("lyric",
|
| 381 |
model="(skipped β using canned default lyric)",
|
| 382 |
system_prompt=SYSTEM_PROMPT, user_prompt=prompt,
|
| 383 |
temperature=temperature, raw_completion=DEFAULT_LULLABY)
|
| 384 |
return DEFAULT_LULLABY
|
|
|
|
|
|
|
|
|
|
|
|
|
| 385 |
resp = llm.create_chat_completion(
|
| 386 |
messages=[
|
| 387 |
{"role": "system", "content": SYSTEM_PROMPT},
|
| 388 |
{"role": "user", "content": prompt},
|
| 389 |
],
|
| 390 |
temperature=temperature,
|
| 391 |
-
max_tokens=
|
| 392 |
top_p=0.9,
|
|
|
|
|
|
|
|
|
|
| 393 |
)
|
| 394 |
completion = resp["choices"][0]["message"]["content"].strip()
|
| 395 |
lola_trace.stage("lyric",
|
|
@@ -398,7 +403,7 @@ def generate_lullaby(prompt, temperature=0.75):
|
|
| 398 |
user_prompt=prompt,
|
| 399 |
temperature=temperature,
|
| 400 |
top_p=0.9,
|
| 401 |
-
max_tokens=
|
| 402 |
raw_completion=completion)
|
| 403 |
return completion
|
| 404 |
|
|
@@ -2712,7 +2717,7 @@ with gr.Blocks(css_paths="style.css", title="Lolaby", theme=gr.themes.Citrus(),
|
|
| 2712 |
"</div>"),
|
| 2713 |
elem_id="success-banner",
|
| 2714 |
)
|
| 2715 |
-
audio_out = gr.Audio(label="", type="filepath")
|
| 2716 |
# "What Lola saw" hint β sits between the song output and the
|
| 2717 |
# lyrics, explaining how the drawing/typed inputs informed
|
| 2718 |
# what's playing. Stays blank until the first generation; hides
|
|
|
|
| 11 |
Aesthetic: children's drawing β crayon textures, wobbly hand-drawn borders.
|
| 12 |
"""
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
import base64
|
| 15 |
import glob
|
| 16 |
import os
|
|
|
|
| 47 |
#
|
| 48 |
# We catch Exception (not just ImportError) so a heavy transitive failure
|
| 49 |
# doesn't crash the whole app, but we PRINT the traceback β a silent swallow
|
| 50 |
+
# here previously hid the real reason the vision module wasn't loading.
|
|
|
|
|
|
|
| 51 |
try:
|
| 52 |
import draw.vision as vision # vision.describe(image) -> {"loves": str, ...}
|
| 53 |
except Exception as _vision_import_err:
|
|
|
|
| 100 |
else:
|
| 101 |
print(f"Loading {MODEL_REPO}...")
|
| 102 |
try:
|
| 103 |
+
# Thread count is the biggest CPU lever for llama.cpp β token gen
|
| 104 |
+
# scales ~linearly with physical cores up to the memory-bandwidth
|
| 105 |
+
# ceiling. The old n_threads=2 left most of the box idle. Default to
|
| 106 |
+
# all cores; override with LULLABY_N_THREADS (on hyperthreaded CPUs,
|
| 107 |
+
# physical-core count sometimes beats logical-core count).
|
| 108 |
+
_n_threads = int(os.environ.get("LULLABY_N_THREADS", os.cpu_count() or 4))
|
| 109 |
+
_llama_kwargs = dict(
|
| 110 |
repo_id=MODEL_REPO,
|
| 111 |
filename="*Q4_K_M.gguf",
|
| 112 |
n_ctx=1024,
|
| 113 |
+
n_threads=_n_threads,
|
| 114 |
+
n_threads_batch=_n_threads, # also parallelize prompt ingestion
|
| 115 |
+
n_batch=512, # whole prompt in one eval batch
|
| 116 |
+
n_gpu_layers=0, # CPU only
|
| 117 |
chat_format="llama-3",
|
| 118 |
verbose=False,
|
| 119 |
)
|
| 120 |
+
try:
|
| 121 |
+
# flash_attn trims prompt-eval + decode where the build supports
|
| 122 |
+
# it; older llama-cpp-python rejects the kwarg with TypeError, so
|
| 123 |
+
# fall back to a load without it rather than crashing.
|
| 124 |
+
llm = Llama.from_pretrained(flash_attn=True, **_llama_kwargs)
|
| 125 |
+
except TypeError:
|
| 126 |
+
print("[llm] flash_attn unsupported on this llama-cpp-python; "
|
| 127 |
+
"loading without it")
|
| 128 |
+
llm = Llama.from_pretrained(**_llama_kwargs)
|
| 129 |
+
print(f"Model loaded (n_threads={_n_threads}).")
|
| 130 |
except Exception as e:
|
| 131 |
+
# If the GGUF is missing or won't load, don't take the whole app
|
| 132 |
# down with a cryptic import-time crash. Leave llm=None; the UI still
|
| 133 |
# loads and make_lullaby surfaces a friendly message.
|
| 134 |
print(f"WARNING: could not load model at {MODEL_REPO}: {e}")
|
|
|
|
| 138 |
# ---------------------------------------------------------------------------
|
| 139 |
# Pre-warm everything else at app startup, so the first user click is fast.
|
| 140 |
#
|
| 141 |
+
# Honest tradeoff: pre-warming makes COLD STARTUP slower (the app takes
|
| 142 |
# longer to come online) in exchange for the FIRST USER GENERATION being
|
| 143 |
# much faster β which is the moment that actually matters for a judge who
|
| 144 |
# clicks the link and waits to see something happen.
|
|
|
|
| 146 |
# What we pre-warm and why:
|
| 147 |
# - Llama lyric model: already eager-loaded above.
|
| 148 |
# - Kokoro TTS: ~340 MB voice model + downloads. ~5-10s saved on first
|
| 149 |
+
# generation.
|
| 150 |
# - MiniCPM-V vision: ~3 GB download (first run ever) + model load
|
| 151 |
+
# (~10-30s on CPU). Loads weights into RAM at startup so the first user
|
| 152 |
+
# click skips the load. Saves ~20-30s on first user click.
|
|
|
|
|
|
|
| 153 |
#
|
| 154 |
# Both pre-warms are wrapped in try/except so a failure during warmup
|
| 155 |
+
# doesn't crash the app β the lazy fallback paths inside each module
|
| 156 |
# still work if warmup fails for any reason.
|
| 157 |
# ---------------------------------------------------------------------------
|
| 158 |
|
| 159 |
if not SKIP_LLM:
|
| 160 |
+
# Warm the lyric model with one tiny throwaway generation. Weights are
|
| 161 |
+
# already resident (loaded at import), but the FIRST create_chat_completion
|
| 162 |
+
# still pays a one-off prompt-eval/graph warmup; doing it here moves that
|
| 163 |
+
# cost off the first real user click.
|
| 164 |
+
if llm is not None:
|
| 165 |
+
try:
|
| 166 |
+
print("Pre-warming lyric model...")
|
| 167 |
+
llm.create_chat_completion(
|
| 168 |
+
messages=[
|
| 169 |
+
{"role": "system", "content": SYSTEM_PROMPT},
|
| 170 |
+
{"role": "user", "content": "Write a lullaby for: test, age 3"},
|
| 171 |
+
],
|
| 172 |
+
max_tokens=1,
|
| 173 |
+
temperature=0.0,
|
| 174 |
+
)
|
| 175 |
+
print("Lyric model ready.")
|
| 176 |
+
except Exception as e:
|
| 177 |
+
print(f"WARNING: lyric model pre-warm failed (non-fatal): {e}")
|
| 178 |
+
|
| 179 |
try:
|
| 180 |
print("Pre-warming Kokoro TTS voice...")
|
| 181 |
# warmup() (not just _load_kokoro()) so the af_nicole.pt voice tensor
|
|
|
|
| 191 |
try:
|
| 192 |
print("Pre-warming MiniCPM-V vision (CPU load only)...")
|
| 193 |
# Triggers _try_load() inside vision.py, which downloads + loads the
|
| 194 |
+
# model into RAM so the first real describe() call doesn't pay for it.
|
|
|
|
| 195 |
import draw.vision as _vision_warmup
|
| 196 |
_vision_warmup._try_load()
|
| 197 |
print("Vision ready.")
|
|
|
|
| 373 |
return "\n".join(lines)
|
| 374 |
|
| 375 |
|
| 376 |
+
def generate_lullaby(prompt, temperature=0.75, max_tokens=256):
|
| 377 |
if SKIP_LLM or llm is None:
|
| 378 |
lola_trace.stage("lyric",
|
| 379 |
model="(skipped β using canned default lyric)",
|
| 380 |
system_prompt=SYSTEM_PROMPT, user_prompt=prompt,
|
| 381 |
temperature=temperature, raw_completion=DEFAULT_LULLABY)
|
| 382 |
return DEFAULT_LULLABY
|
| 383 |
+
# max_tokens=256 (was 512): a lullaby with its header is ~80-150 tokens,
|
| 384 |
+
# and decode time is linear in tokens PRODUCED β 512 just gave room to
|
| 385 |
+
# ramble. Stop sequences end generation the moment the model starts a
|
| 386 |
+
# second song or adds commentary instead of burning tokens to the cap.
|
| 387 |
resp = llm.create_chat_completion(
|
| 388 |
messages=[
|
| 389 |
{"role": "system", "content": SYSTEM_PROMPT},
|
| 390 |
{"role": "user", "content": prompt},
|
| 391 |
],
|
| 392 |
temperature=temperature,
|
| 393 |
+
max_tokens=max_tokens,
|
| 394 |
top_p=0.9,
|
| 395 |
+
top_k=40,
|
| 396 |
+
repeat_penalty=1.1,
|
| 397 |
+
stop=["<|eot_id|>", "\n\n\n", "Note:", "Here is", "I hope"],
|
| 398 |
)
|
| 399 |
completion = resp["choices"][0]["message"]["content"].strip()
|
| 400 |
lola_trace.stage("lyric",
|
|
|
|
| 403 |
user_prompt=prompt,
|
| 404 |
temperature=temperature,
|
| 405 |
top_p=0.9,
|
| 406 |
+
max_tokens=max_tokens,
|
| 407 |
raw_completion=completion)
|
| 408 |
return completion
|
| 409 |
|
|
|
|
| 2717 |
"</div>"),
|
| 2718 |
elem_id="success-banner",
|
| 2719 |
)
|
| 2720 |
+
audio_out = gr.Audio(label="", type="filepath",buttons=['download'])
|
| 2721 |
# "What Lola saw" hint β sits between the song output and the
|
| 2722 |
# lyrics, explaining how the drawing/typed inputs informed
|
| 2723 |
# what's playing. Stays blank until the first generation; hides
|
draw/vision.py
CHANGED
|
@@ -34,6 +34,10 @@ import traceback
|
|
| 34 |
|
| 35 |
VISION_MODEL_ID = "openbmb/MiniCPM-V-4.6"
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
# Tuned for "child's drawing β short loves phrase". We want concrete nouns, no
|
| 38 |
# stylistic description ("crayon", "stick figure"), and safety baked in so the
|
| 39 |
# vision model can't surface something dark we'd then have to filter out.
|
|
@@ -155,7 +159,11 @@ def describe_with_vision(image, seed=None):
|
|
| 155 |
with torch.inference_mode():
|
| 156 |
out_ids = _model.generate(
|
| 157 |
**inputs,
|
| 158 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
do_sample=False,
|
| 160 |
downsample_mode="16x",
|
| 161 |
)
|
|
@@ -251,12 +259,15 @@ def _to_pil(image):
|
|
| 251 |
if (arr > 240).all():
|
| 252 |
return None
|
| 253 |
|
| 254 |
-
# Cap size β MiniCPM-V handles big images but
|
| 255 |
-
#
|
| 256 |
-
# child's drawing
|
|
|
|
|
|
|
|
|
|
| 257 |
w, h = image.size
|
| 258 |
-
if max(w, h) >
|
| 259 |
-
scale =
|
| 260 |
image = image.resize((int(w * scale), int(h * scale)), Image.LANCZOS)
|
| 261 |
return image
|
| 262 |
|
|
|
|
| 34 |
|
| 35 |
VISION_MODEL_ID = "openbmb/MiniCPM-V-4.6"
|
| 36 |
|
| 37 |
+
# Long-side cap for input images. Vision cost scales with area, so smaller =
|
| 38 |
+
# faster; 512 is plenty of detail for naming objects in a child's drawing.
|
| 39 |
+
_MAX_LONG_SIDE = 512
|
| 40 |
+
|
| 41 |
# Tuned for "child's drawing β short loves phrase". We want concrete nouns, no
|
| 42 |
# stylistic description ("crayon", "stick figure"), and safety baked in so the
|
| 43 |
# vision model can't surface something dark we'd then have to filter out.
|
|
|
|
| 159 |
with torch.inference_mode():
|
| 160 |
out_ids = _model.generate(
|
| 161 |
**inputs,
|
| 162 |
+
# 8-word answer is ~12-16 tokens; 24 is comfortable headroom.
|
| 163 |
+
# Decode time is linear in tokens generated, and
|
| 164 |
+
# _clean_description truncates to 12 words anyway, so a lower
|
| 165 |
+
# cap costs nothing and trims any rambling response.
|
| 166 |
+
max_new_tokens=24,
|
| 167 |
do_sample=False,
|
| 168 |
downsample_mode="16x",
|
| 169 |
)
|
|
|
|
| 259 |
if (arr > 240).all():
|
| 260 |
return None
|
| 261 |
|
| 262 |
+
# Cap size β MiniCPM-V handles big images but vision cost scales with
|
| 263 |
+
# image area (more pixels β more visual tokens β slower generate). For
|
| 264 |
+
# "name the objects in a child's drawing in 8 words" 512px on the long
|
| 265 |
+
# side is plenty of detail, and 512 vs 768 cuts the area to ~44%, a real
|
| 266 |
+
# speedup. Bump _MAX_LONG_SIDE back toward 768 if you ever see the model
|
| 267 |
+
# missing small details in busy drawings.
|
| 268 |
w, h = image.size
|
| 269 |
+
if max(w, h) > _MAX_LONG_SIDE:
|
| 270 |
+
scale = _MAX_LONG_SIDE / max(w, h)
|
| 271 |
image = image.resize((int(w * scale), int(h * scale)), Image.LANCZOS)
|
| 272 |
return image
|
| 273 |
|
synths/guitar.py
CHANGED
|
@@ -178,17 +178,29 @@ def karplus_strong(freq, n_samples, sr=SR, damping=0.997, brightness=0.5):
|
|
| 178 |
b, a = butter(2, cutoff / (sr / 2), btype="low")
|
| 179 |
excitation = lfilter(b, a, excitation).astype(np.float32)
|
| 180 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
buf = excitation.copy()
|
| 182 |
-
|
| 183 |
-
prev = 0.0
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
|
|
|
|
|
|
|
|
|
| 192 |
return out
|
| 193 |
|
| 194 |
|
|
|
|
| 178 |
b, a = butter(2, cutoff / (sr / 2), btype="low")
|
| 179 |
excitation = lfilter(b, a, excitation).astype(np.float32)
|
| 180 |
|
| 181 |
+
# Block-vectorized KS. The original was a per-sample Python loop running
|
| 182 |
+
# at 44.1k iterations/sec/string β the dominant cost in the whole synth.
|
| 183 |
+
# Within one lap around the delay line there's no read-after-write (a
|
| 184 |
+
# buffer slot written on lap k isn't re-read until lap k+1), so each lap
|
| 185 |
+
# of length `delay_len` can be updated as a single vector op:
|
| 186 |
+
# new[j] = damping*0.5*(cur[j] + cur[j-1]) with cur[-1] = carry
|
| 187 |
+
# The only cross-lap state is `prev` (the last sample read), carried over.
|
| 188 |
+
# This is bit-exact with the original loop and ~15x faster on CPU.
|
| 189 |
+
out = np.empty(n_samples, dtype=np.float32)
|
| 190 |
buf = excitation.copy()
|
| 191 |
+
g = np.float32(damping * 0.5)
|
| 192 |
+
prev = np.float32(0.0)
|
| 193 |
+
pos = 0
|
| 194 |
+
while pos < n_samples:
|
| 195 |
+
block = min(delay_len, n_samples - pos)
|
| 196 |
+
cur = buf[:block].copy() # snapshot before we overwrite buf
|
| 197 |
+
out[pos:pos + block] = cur
|
| 198 |
+
shifted = np.empty(block, dtype=np.float32)
|
| 199 |
+
shifted[0] = prev
|
| 200 |
+
shifted[1:] = cur[:block - 1]
|
| 201 |
+
buf[:block] = g * (cur + shifted)
|
| 202 |
+
prev = cur[block - 1]
|
| 203 |
+
pos += block
|
| 204 |
return out
|
| 205 |
|
| 206 |
|
synths/harp.py
CHANGED
|
@@ -150,18 +150,25 @@ def _karplus_strong(freq, n_samples, sr, damping=0.9985):
|
|
| 150 |
b, a = butter(2, [200 / (sr / 2), 3500 / (sr / 2)], btype="band")
|
| 151 |
excitation = lfilter(b, a, excitation).astype(np.float32)
|
| 152 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
buf = excitation.copy()
|
| 154 |
-
|
| 155 |
-
prev = 0.0
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
|
|
|
|
|
|
| 165 |
return out
|
| 166 |
|
| 167 |
|
|
|
|
| 150 |
b, a = butter(2, [200 / (sr / 2), 3500 / (sr / 2)], btype="band")
|
| 151 |
excitation = lfilter(b, a, excitation).astype(np.float32)
|
| 152 |
|
| 153 |
+
# Block-vectorized KS β see guitar.py for the derivation. Within one lap
|
| 154 |
+
# around the delay line there's no read-after-write, so each lap updates
|
| 155 |
+
# as a single vector op; only the `prev` carry crosses lap boundaries.
|
| 156 |
+
# Bit-exact with the original per-sample loop, ~15x faster on CPU.
|
| 157 |
+
out = np.empty(n_samples, dtype=np.float32)
|
| 158 |
buf = excitation.copy()
|
| 159 |
+
g = np.float32(damping * 0.5)
|
| 160 |
+
prev = np.float32(0.0)
|
| 161 |
+
pos = 0
|
| 162 |
+
while pos < n_samples:
|
| 163 |
+
block = min(delay_len, n_samples - pos)
|
| 164 |
+
cur = buf[:block].copy()
|
| 165 |
+
out[pos:pos + block] = cur
|
| 166 |
+
shifted = np.empty(block, dtype=np.float32)
|
| 167 |
+
shifted[0] = prev
|
| 168 |
+
shifted[1:] = cur[:block - 1]
|
| 169 |
+
buf[:block] = g * (cur + shifted)
|
| 170 |
+
prev = cur[block - 1]
|
| 171 |
+
pos += block
|
| 172 |
return out
|
| 173 |
|
| 174 |
|
synths/voice.py
CHANGED
|
@@ -6,6 +6,7 @@ No singing, no pitch manipulation β just a warm reading voice over music.
|
|
| 6 |
"""
|
| 7 |
|
| 8 |
import numpy as np
|
|
|
|
| 9 |
|
| 10 |
SR_TARGET = 44100
|
| 11 |
|
|
@@ -89,13 +90,16 @@ def _gentle_vocal_eq(audio):
|
|
| 89 |
return out.astype(np.float32)
|
| 90 |
|
| 91 |
|
| 92 |
-
def speak_lyrics(lyrics, target_seconds=None, speed=
|
| 93 |
"""
|
| 94 |
Render lyrics as gentle spoken voice. Returns mono float32 at SR_TARGET.
|
| 95 |
|
| 96 |
speed=0.85 β slightly slower than normal, bedtime pacing.
|
| 97 |
target_seconds (if given) β pad with intro silence so voice ends near track end.
|
| 98 |
"""
|
|
|
|
|
|
|
|
|
|
| 99 |
if not _load_kokoro():
|
| 100 |
print("WARNING: TTS unavailable, returning silent vocal track")
|
| 101 |
return _silence(target_seconds or 1.0)
|
|
|
|
| 6 |
"""
|
| 7 |
|
| 8 |
import numpy as np
|
| 9 |
+
import random
|
| 10 |
|
| 11 |
SR_TARGET = 44100
|
| 12 |
|
|
|
|
| 90 |
return out.astype(np.float32)
|
| 91 |
|
| 92 |
|
| 93 |
+
def speak_lyrics(lyrics, target_seconds=None, speed=None):
|
| 94 |
"""
|
| 95 |
Render lyrics as gentle spoken voice. Returns mono float32 at SR_TARGET.
|
| 96 |
|
| 97 |
speed=0.85 β slightly slower than normal, bedtime pacing.
|
| 98 |
target_seconds (if given) β pad with intro silence so voice ends near track end.
|
| 99 |
"""
|
| 100 |
+
if speed is None:
|
| 101 |
+
speed = round(random.uniform(0.85, 0.95), 2)
|
| 102 |
+
|
| 103 |
if not _load_kokoro():
|
| 104 |
print("WARNING: TTS unavailable, returning silent vocal track")
|
| 105 |
return _silence(target_seconds or 1.0)
|
synths_/__init__.py
ADDED
|
File without changes
|
synths_/guitar.py
ADDED
|
@@ -0,0 +1,393 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Acoustic guitar synthesizer β rebuilt from spectral analysis of two
|
| 3 |
+
Freesound acoustic guitar reference recordings.
|
| 4 |
+
|
| 5 |
+
Analysis findings (F#4 isolated note from ref_guitar2):
|
| 6 |
+
Spectrum (ΓN is harmonic of fundamental):
|
| 7 |
+
H1 (Γ1.0): amp 202 the fundamental
|
| 8 |
+
H2 (Γ2.0): amp 55 weaker than H1 and H3
|
| 9 |
+
H3 (Γ3.0): amp 235 STRONGER than fundamental β steel string signature
|
| 10 |
+
H4 (Γ4.0): amp 26
|
| 11 |
+
H5 (Γ5.0): amp 23
|
| 12 |
+
H6 (Γ6.0): amp 13
|
| 13 |
+
H7-H13: present out to 5kHz+ β bright, articulate
|
| 14 |
+
|
| 15 |
+
Envelope (post-attack peak):
|
| 16 |
+
Attack to peak: ~50-90ms
|
| 17 |
+
-3dB: 186ms
|
| 18 |
+
-6dB: 348ms
|
| 19 |
+
-12dB: ~700-1100ms
|
| 20 |
+
|
| 21 |
+
Strong pluck transient on attack (pick or fingernail click).
|
| 22 |
+
|
| 23 |
+
The dominant-H3 character is what makes a steel-string acoustic guitar
|
| 24 |
+
sound bright and "country" rather than warm and "classical". A naive
|
| 25 |
+
Karplus-Strong does NOT reproduce this β it generates a monotonically
|
| 26 |
+
falling harmonic series. So we layer KS (for natural string sympathetic
|
| 27 |
+
behavior) with explicit boosted-H3 partials and pluck transient noise.
|
| 28 |
+
|
| 29 |
+
Public interface (used by app.py):
|
| 30 |
+
GuitarSynth().sequence(events, effects=ACOUSTIC_PRESET) β np.ndarray
|
| 31 |
+
ACOUSTIC_PRESET: list of effect callables, applied in order
|
| 32 |
+
|
| 33 |
+
Event format:
|
| 34 |
+
{"type": "chord", "name": "C", "time": 0.0, "duration": 3.0,
|
| 35 |
+
"direction": "down", "spread_ms": 60,
|
| 36 |
+
"volume": 0.6, "decay": 0.997, "brightness": 0.5}
|
| 37 |
+
{"type": "note", "name": "G4", "time": 0.0, "duration": 1.0,
|
| 38 |
+
"volume": 1.0, "brightness": 0.5}
|
| 39 |
+
"""
|
| 40 |
+
|
| 41 |
+
import numpy as np
|
| 42 |
+
from scipy.signal import lfilter, butter
|
| 43 |
+
|
| 44 |
+
SR = 44100
|
| 45 |
+
|
| 46 |
+
NOTE_TO_SEMI = {
|
| 47 |
+
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, "E": 4, "F": 5,
|
| 48 |
+
"F#": 6, "Gb": 6, "G": 7, "G#": 8, "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
+
# Chord voicings (semitone offsets from root). Tuned for the guitar's
|
| 52 |
+
# natural register β fuller than a triad to sound like a strummed shape.
|
| 53 |
+
CHORD_INTERVALS = {
|
| 54 |
+
"": [0, 7, 12, 16, 19, 24], # major: root + 5 + oct + 3rd-up + 5-up + 2oct
|
| 55 |
+
"m": [0, 7, 12, 15, 19, 24], # minor: same but b3
|
| 56 |
+
"7": [0, 7, 10, 16, 19, 22],
|
| 57 |
+
"m7": [0, 7, 10, 15, 19, 22],
|
| 58 |
+
"maj7": [0, 7, 11, 16, 19, 23],
|
| 59 |
+
"sus2": [0, 7, 12, 14, 19, 24],
|
| 60 |
+
"sus4": [0, 7, 12, 17, 19, 24],
|
| 61 |
+
}
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def parse_chord(name):
|
| 65 |
+
name = name.strip()
|
| 66 |
+
if len(name) >= 2 and name[1] in ("#", "b"):
|
| 67 |
+
root, quality = name[:2], name[2:]
|
| 68 |
+
else:
|
| 69 |
+
root, quality = name[:1], name[1:]
|
| 70 |
+
return root, quality
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def note_to_hz(name):
|
| 74 |
+
name = name.strip()
|
| 75 |
+
for i, ch in enumerate(name):
|
| 76 |
+
if ch.isdigit() or ch == "-":
|
| 77 |
+
pitch = name[:i]
|
| 78 |
+
octave = int(name[i:])
|
| 79 |
+
break
|
| 80 |
+
else:
|
| 81 |
+
pitch, octave = name, 4
|
| 82 |
+
midi = 12 * (octave + 1) + NOTE_TO_SEMI[pitch]
|
| 83 |
+
return 440.0 * 2 ** ((midi - 69) / 12)
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def chord_to_freqs(chord_name, base_octave=3):
|
| 87 |
+
"""Return list of frequencies in Hz for each string in the chord voicing."""
|
| 88 |
+
root, quality = parse_chord(chord_name)
|
| 89 |
+
intervals = CHORD_INTERVALS.get(quality, CHORD_INTERVALS[""])
|
| 90 |
+
root_midi = 12 * (base_octave + 1) + NOTE_TO_SEMI[root]
|
| 91 |
+
return [440.0 * 2 ** ((root_midi + i - 69) / 12) for i in intervals]
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
def add_at(track, audio, start_sec, sr=SR):
|
| 95 |
+
start = int(start_sec * sr)
|
| 96 |
+
end = start + len(audio)
|
| 97 |
+
if end > len(track):
|
| 98 |
+
track = np.pad(track, (0, end - len(track)))
|
| 99 |
+
track[start:end] += audio
|
| 100 |
+
return track
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def normalize(x, target=0.95):
|
| 104 |
+
peak = float(np.max(np.abs(x)))
|
| 105 |
+
if peak > 1e-9:
|
| 106 |
+
return x * (target / peak)
|
| 107 |
+
return x
|
| 108 |
+
|
| 109 |
+
|
| 110 |
+
# ---------- effect building blocks ----------
|
| 111 |
+
|
| 112 |
+
def lowpass(audio, cutoff=7500, sr=SR, order=2):
|
| 113 |
+
b, a = butter(order, cutoff / (sr / 2), btype="low")
|
| 114 |
+
return lfilter(b, a, audio).astype(np.float32)
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
def highpass(audio, cutoff=80, sr=SR, order=1):
|
| 118 |
+
b, a = butter(order, cutoff / (sr / 2), btype="high")
|
| 119 |
+
return lfilter(b, a, audio).astype(np.float32)
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
def gentle_compress(audio, threshold=0.7, ratio=2.5):
|
| 123 |
+
"""Soft-knee compression for evening out plucks."""
|
| 124 |
+
out = audio.copy()
|
| 125 |
+
over = np.abs(out) > threshold
|
| 126 |
+
sign = np.sign(out[over])
|
| 127 |
+
excess = np.abs(out[over]) - threshold
|
| 128 |
+
out[over] = sign * (threshold + excess / ratio)
|
| 129 |
+
return out
|
| 130 |
+
|
| 131 |
+
|
| 132 |
+
def body_resonance(audio, sr=SR, mix=0.18):
|
| 133 |
+
"""
|
| 134 |
+
Body resonance simulation β three narrow bandpass peaks at the
|
| 135 |
+
typical acoustic guitar body resonances:
|
| 136 |
+
- Helmholtz (air): ~100 Hz
|
| 137 |
+
- Top plate: ~200 Hz
|
| 138 |
+
- Back plate: ~380 Hz
|
| 139 |
+
Mixed back in at low gain β gives the "boxy warm body" sound.
|
| 140 |
+
"""
|
| 141 |
+
res = np.zeros_like(audio)
|
| 142 |
+
for freq, q_width, gain in [
|
| 143 |
+
(100, 60, 0.40),
|
| 144 |
+
(200, 100, 0.35),
|
| 145 |
+
(380, 150, 0.25),
|
| 146 |
+
]:
|
| 147 |
+
low = max(20, freq - q_width)
|
| 148 |
+
high = min(sr / 2 * 0.95, freq + q_width)
|
| 149 |
+
b, a = butter(2, [low / (sr / 2), high / (sr / 2)], btype="band")
|
| 150 |
+
res += lfilter(b, a, audio).astype(np.float32) * gain
|
| 151 |
+
return audio + res * mix
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
# ACOUSTIC_PRESET: pipeline applied to the final mix. Each callable takes
|
| 155 |
+
# (audio,) and returns audio.
|
| 156 |
+
ACOUSTIC_PRESET = [
|
| 157 |
+
lambda x: body_resonance(x, mix=0.20),
|
| 158 |
+
lambda x: highpass(x, cutoff=70),
|
| 159 |
+
lambda x: lowpass(x, cutoff=8500),
|
| 160 |
+
lambda x: gentle_compress(x, threshold=0.7, ratio=2.5),
|
| 161 |
+
lambda x: normalize(x, target=0.92),
|
| 162 |
+
]
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
# ---------- core string synthesis ----------
|
| 166 |
+
|
| 167 |
+
def karplus_strong(freq, n_samples, sr=SR, damping=0.997, brightness=0.5):
|
| 168 |
+
"""
|
| 169 |
+
Karplus-Strong plucked string. Provides the natural harmonic comb +
|
| 170 |
+
realistic decay-with-pitch behavior (high frequencies decay faster).
|
| 171 |
+
"""
|
| 172 |
+
delay_len = max(2, int(round(sr / freq)))
|
| 173 |
+
|
| 174 |
+
# Pluck excitation β pre-filtered noise. Brightness controls the
|
| 175 |
+
# high-frequency content of the initial energy.
|
| 176 |
+
excitation = (np.random.rand(delay_len).astype(np.float32) * 2 - 1)
|
| 177 |
+
cutoff = 600 + brightness * 5000
|
| 178 |
+
b, a = butter(2, cutoff / (sr / 2), btype="low")
|
| 179 |
+
excitation = lfilter(b, a, excitation).astype(np.float32)
|
| 180 |
+
|
| 181 |
+
buf = excitation.copy()
|
| 182 |
+
out = np.zeros(n_samples, dtype=np.float32)
|
| 183 |
+
prev = 0.0
|
| 184 |
+
i = 0
|
| 185 |
+
for s in range(n_samples):
|
| 186 |
+
cur = buf[i]
|
| 187 |
+
out[s] = cur
|
| 188 |
+
new = damping * 0.5 * (cur + prev)
|
| 189 |
+
buf[i] = new
|
| 190 |
+
prev = cur
|
| 191 |
+
i = (i + 1) % delay_len
|
| 192 |
+
return out
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
def pluck_transient(sr=SR, brightness=0.5, hardness=0.5, length_ms=8):
|
| 196 |
+
"""
|
| 197 |
+
Brief filtered noise burst β the pick/fingernail attack click.
|
| 198 |
+
"""
|
| 199 |
+
n = int(length_ms / 1000 * sr)
|
| 200 |
+
if n <= 0:
|
| 201 |
+
return np.zeros(0, dtype=np.float32)
|
| 202 |
+
noise = np.random.randn(n).astype(np.float32) * (0.10 + hardness * 0.10)
|
| 203 |
+
# Bandpass for "pick" character β 1.5-6 kHz
|
| 204 |
+
low = 1200 + hardness * 600
|
| 205 |
+
high = min(sr / 2 * 0.95, 4000 + brightness * 2500)
|
| 206 |
+
b, a = butter(2, [low / (sr / 2), high / (sr / 2)], btype="band")
|
| 207 |
+
click = lfilter(b, a, noise).astype(np.float32)
|
| 208 |
+
# Fast exponential decay
|
| 209 |
+
env = np.exp(-np.arange(n) / (sr * 0.0015))
|
| 210 |
+
return (click * env).astype(np.float32)
|
| 211 |
+
|
| 212 |
+
|
| 213 |
+
class GuitarSynth:
|
| 214 |
+
"""
|
| 215 |
+
Acoustic steel-string guitar synthesizer.
|
| 216 |
+
|
| 217 |
+
Hybrid: Karplus-Strong (sympathetic behavior + natural harmonic comb)
|
| 218 |
+
layered with explicit boosted-H3 partials (the steel-string signature)
|
| 219 |
+
and a pluck transient.
|
| 220 |
+
"""
|
| 221 |
+
|
| 222 |
+
def __init__(self, sr=SR):
|
| 223 |
+
self.sr = sr
|
| 224 |
+
|
| 225 |
+
def note(self, freq, duration_s, volume=1.0, brightness=0.5):
|
| 226 |
+
"""
|
| 227 |
+
Single plucked string.
|
| 228 |
+
|
| 229 |
+
brightness: 0..1. 0.5 matches the reference reasonably; bump higher
|
| 230 |
+
for more bright country-strum feel.
|
| 231 |
+
"""
|
| 232 |
+
sr = self.sr
|
| 233 |
+
# Render long enough for the full decay tail (~1.5s perceptible)
|
| 234 |
+
n = max(int(duration_s * sr), int(1.8 * sr))
|
| 235 |
+
t = np.arange(n) / sr
|
| 236 |
+
|
| 237 |
+
# Damping scales with pitch: lower notes sustain longer.
|
| 238 |
+
# Tuned so F#4 (370 Hz) hits the ~350ms -6dB measured in the reference.
|
| 239 |
+
damping = max(0.9955, 0.9992 - (freq / 18000.0))
|
| 240 |
+
ks = karplus_strong(freq, n, sr, damping=damping, brightness=brightness)
|
| 241 |
+
|
| 242 |
+
# Envelope: ~70ms attack-to-peak (the reference had peak at 70-93ms,
|
| 243 |
+
# which is the body response building up), then exponential decay
|
| 244 |
+
# with tau β 0.9s (matches -6dB at ~348ms after a 60ms plateau).
|
| 245 |
+
plateau_n = int(0.06 * sr)
|
| 246 |
+
decay_tau = 0.9
|
| 247 |
+
env = np.ones(n, dtype=np.float32)
|
| 248 |
+
if plateau_n > 1:
|
| 249 |
+
env[:plateau_n] = np.linspace(0.6, 1.0, plateau_n) ** 0.8
|
| 250 |
+
decay_n = n - plateau_n
|
| 251 |
+
decay_t = np.arange(decay_n) / sr
|
| 252 |
+
env[plateau_n:] = np.exp(-decay_t / decay_tau)
|
| 253 |
+
ks = ks * env
|
| 254 |
+
|
| 255 |
+
# Explicit partials to inject the steel-string-signature H3 dominance.
|
| 256 |
+
# Reference ratios H1:H2:H3:H4:H5 β 1.00 : 0.27 : 1.16 : 0.13 : 0.11.
|
| 257 |
+
# KS already provides H1 + falling tail. We add JUST enough H3 to push
|
| 258 |
+
# it slightly above the fundamental, plus a touch of H5 for sparkle.
|
| 259 |
+
h3_amp = 0.07 + brightness * 0.04
|
| 260 |
+
h5_amp = 0.02 + brightness * 0.02
|
| 261 |
+
|
| 262 |
+
h3_env_tau = decay_tau * 0.7
|
| 263 |
+
h5_env_tau = decay_tau * 0.5
|
| 264 |
+
h3_env = np.ones(n, dtype=np.float32)
|
| 265 |
+
h5_env = np.ones(n, dtype=np.float32)
|
| 266 |
+
if plateau_n > 1:
|
| 267 |
+
h3_env[:plateau_n] = np.linspace(0.6, 1.0, plateau_n) ** 0.8
|
| 268 |
+
h5_env[:plateau_n] = np.linspace(0.6, 1.0, plateau_n) ** 0.8
|
| 269 |
+
h3_env[plateau_n:] = np.exp(-decay_t / h3_env_tau)
|
| 270 |
+
h5_env[plateau_n:] = np.exp(-decay_t / h5_env_tau)
|
| 271 |
+
|
| 272 |
+
partials = h3_amp * np.sin(2 * np.pi * freq * 3 * t) * h3_env
|
| 273 |
+
if freq * 5 < sr / 2 * 0.9:
|
| 274 |
+
partials += h5_amp * np.sin(2 * np.pi * freq * 5 * t) * h5_env
|
| 275 |
+
|
| 276 |
+
# Pluck transient β adds the attack "click"
|
| 277 |
+
click = pluck_transient(sr=sr, brightness=brightness,
|
| 278 |
+
hardness=0.3 + brightness * 0.2, length_ms=6)
|
| 279 |
+
|
| 280 |
+
signal = ks * 0.75 + partials * 0.5
|
| 281 |
+
# Stamp the click at the front
|
| 282 |
+
click_n = len(click)
|
| 283 |
+
if click_n > 0 and click_n < n:
|
| 284 |
+
signal[:click_n] += click * 0.5
|
| 285 |
+
|
| 286 |
+
# Soft attack ramp (3ms) to prevent any DC pop
|
| 287 |
+
ramp_n = int(0.003 * sr)
|
| 288 |
+
if ramp_n > 1:
|
| 289 |
+
signal[:ramp_n] *= np.linspace(0, 1, ramp_n)
|
| 290 |
+
# Release tail
|
| 291 |
+
rel_n = int(0.05 * sr)
|
| 292 |
+
if rel_n > 1:
|
| 293 |
+
signal[-rel_n:] *= np.linspace(1, 0, rel_n)
|
| 294 |
+
|
| 295 |
+
# Final lowpass to tame any KS aliasing or harshness
|
| 296 |
+
signal = lowpass(signal, cutoff=9000, sr=sr)
|
| 297 |
+
|
| 298 |
+
# Normalize per-note for consistent loudness across pitches
|
| 299 |
+
peak = float(np.max(np.abs(signal)))
|
| 300 |
+
if peak > 1e-9:
|
| 301 |
+
signal = signal / peak * 0.85
|
| 302 |
+
return (signal * volume).astype(np.float32)
|
| 303 |
+
|
| 304 |
+
def chord(self, chord_name, duration_s, base_octave=3,
|
| 305 |
+
direction="down", spread_ms=20,
|
| 306 |
+
volume=0.7, brightness=0.5, decay=None):
|
| 307 |
+
"""
|
| 308 |
+
Render a strummed chord.
|
| 309 |
+
|
| 310 |
+
spread_ms is the TOTAL time the pick takes to cross all strings
|
| 311 |
+
(not per-string). A real strum sweeps in 15-30ms; anything over
|
| 312 |
+
~80ms starts to sound arpeggiated rather than strummed.
|
| 313 |
+
|
| 314 |
+
direction: "down" = bass-to-treble (typical down-strum)
|
| 315 |
+
"up" = treble-to-bass (up-strum)
|
| 316 |
+
"""
|
| 317 |
+
freqs = chord_to_freqs(chord_name, base_octave=base_octave)
|
| 318 |
+
if direction == "up":
|
| 319 |
+
freqs = list(reversed(freqs))
|
| 320 |
+
|
| 321 |
+
# Total strum across all strings β per-string stagger
|
| 322 |
+
# Subtract 1 because we have N-1 gaps between N strings
|
| 323 |
+
n_strings = len(freqs)
|
| 324 |
+
if n_strings <= 1:
|
| 325 |
+
stagger = 0.0
|
| 326 |
+
else:
|
| 327 |
+
stagger = (spread_ms / 1000.0) / (n_strings - 1)
|
| 328 |
+
|
| 329 |
+
note_dur = max(duration_s, 1.5)
|
| 330 |
+
total_len = int((duration_s + stagger * n_strings + 1.5) * self.sr)
|
| 331 |
+
out = np.zeros(total_len, dtype=np.float32)
|
| 332 |
+
|
| 333 |
+
for i, freq in enumerate(freqs):
|
| 334 |
+
# Voicing: bass string slightly stronger
|
| 335 |
+
voice_vol = 1.0 if i == 0 else (0.85 if i < 3 else 0.7)
|
| 336 |
+
note_audio = self.note(freq, note_dur,
|
| 337 |
+
volume=volume * voice_vol,
|
| 338 |
+
brightness=brightness)
|
| 339 |
+
start = int(i * stagger * self.sr)
|
| 340 |
+
end = start + len(note_audio)
|
| 341 |
+
if end > len(out):
|
| 342 |
+
out = np.pad(out, (0, end - len(out)))
|
| 343 |
+
out[start:end] += note_audio
|
| 344 |
+
|
| 345 |
+
# Normalize the strummed chord
|
| 346 |
+
peak = float(np.max(np.abs(out)))
|
| 347 |
+
if peak > 1.0:
|
| 348 |
+
out = out / peak
|
| 349 |
+
return out.astype(np.float32)
|
| 350 |
+
|
| 351 |
+
def sequence(self, events, effects=None):
|
| 352 |
+
"""
|
| 353 |
+
Render a list of chord/note events. Apply `effects` pipeline to the
|
| 354 |
+
final mix (in order). ACOUSTIC_PRESET is the recommended default.
|
| 355 |
+
"""
|
| 356 |
+
if not events:
|
| 357 |
+
return np.zeros(int(self.sr), dtype=np.float32)
|
| 358 |
+
|
| 359 |
+
events = sorted(events, key=lambda e: e["time"])
|
| 360 |
+
end_time = max(e["time"] + max(e["duration"], 0.6) for e in events) + 2.0
|
| 361 |
+
track = np.zeros(int(end_time * self.sr) + 1, dtype=np.float32)
|
| 362 |
+
|
| 363 |
+
for ev in events:
|
| 364 |
+
typ = ev.get("type")
|
| 365 |
+
if typ == "chord":
|
| 366 |
+
# Accept the same params app.py passes today: name, time,
|
| 367 |
+
# duration, direction, spread_ms, volume, decay, brightness.
|
| 368 |
+
# `decay` is informational β actual decay is per-pitch.
|
| 369 |
+
audio = self.chord(
|
| 370 |
+
ev["name"], ev["duration"],
|
| 371 |
+
base_octave=ev.get("octave", ev.get("base_octave", 3)),
|
| 372 |
+
direction=ev.get("direction", "down"),
|
| 373 |
+
spread_ms=ev.get("spread_ms", 60),
|
| 374 |
+
volume=ev.get("volume", 0.7),
|
| 375 |
+
brightness=ev.get("brightness", 0.5),
|
| 376 |
+
)
|
| 377 |
+
elif typ == "note":
|
| 378 |
+
freq = note_to_hz(ev["name"])
|
| 379 |
+
audio = self.note(
|
| 380 |
+
freq, max(ev["duration"], 1.5),
|
| 381 |
+
volume=ev.get("volume", 1.0),
|
| 382 |
+
brightness=ev.get("brightness", 0.5),
|
| 383 |
+
)
|
| 384 |
+
else:
|
| 385 |
+
continue
|
| 386 |
+
track = add_at(track, audio, ev["time"], sr=self.sr)
|
| 387 |
+
|
| 388 |
+
# Apply post-effects pipeline if given
|
| 389 |
+
if effects:
|
| 390 |
+
for fx in effects:
|
| 391 |
+
track = fx(track)
|
| 392 |
+
|
| 393 |
+
return track
|
synths_/harp.py
ADDED
|
@@ -0,0 +1,381 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Harp synthesizer β based on spectral analysis of two Freesound reference clips
|
| 3 |
+
(freesound_community-harp-motif2 and remembrance-harp).
|
| 4 |
+
|
| 5 |
+
Analysis findings:
|
| 6 |
+
- Strong fundamental, with H2 at ~0.67Γ the fundamental's amplitude
|
| 7 |
+
(warm "woody" character β much more H2 than the xylophone's 1:0.03)
|
| 8 |
+
- H3 weak (~0.06Γ), faint H6 visible
|
| 9 |
+
- Envelope: ~70ms plateau at peak, then exponential decay
|
| 10 |
+
-6dB at ~210ms post-peak
|
| 11 |
+
-20dB at ~770ms
|
| 12 |
+
-40dB at ~1.7s
|
| 13 |
+
Fully inaudible by ~2-3s
|
| 14 |
+
- Chord arpeggio interval (when arpeggiated): ~110-200ms between strikes
|
| 15 |
+
|
| 16 |
+
Hybrid synthesis: a long-decay Karplus-Strong string for the natural harmonic
|
| 17 |
+
comb and sympathetic-ringing character, layered with explicit sine partials
|
| 18 |
+
for the fundamental + H2 to match the reference's warmth.
|
| 19 |
+
"""
|
| 20 |
+
|
| 21 |
+
import numpy as np
|
| 22 |
+
from scipy.signal import lfilter, butter
|
| 23 |
+
|
| 24 |
+
SR = 44100
|
| 25 |
+
|
| 26 |
+
NOTE = {
|
| 27 |
+
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, "E": 4, "F": 5,
|
| 28 |
+
"F#": 6, "Gb": 6, "G": 7, "G#": 8, "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
CHORD_INTERVALS = {
|
| 32 |
+
"": [0, 4, 7, 12],
|
| 33 |
+
"m": [0, 3, 7, 12],
|
| 34 |
+
"7": [0, 4, 7, 10],
|
| 35 |
+
"m7": [0, 3, 7, 10],
|
| 36 |
+
"maj7": [0, 4, 7, 11],
|
| 37 |
+
"sus2": [0, 2, 7, 12],
|
| 38 |
+
"sus4": [0, 5, 7, 12],
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def parse_chord(name):
|
| 43 |
+
name = name.strip()
|
| 44 |
+
if len(name) >= 2 and name[1] in ("#", "b"):
|
| 45 |
+
root, quality = name[:2], name[2:]
|
| 46 |
+
else:
|
| 47 |
+
root, quality = name[:1], name[1:]
|
| 48 |
+
return root, quality
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def note_to_hz(name):
|
| 52 |
+
name = name.strip()
|
| 53 |
+
for i, ch in enumerate(name):
|
| 54 |
+
if ch.isdigit() or ch == "-":
|
| 55 |
+
pitch = name[:i]
|
| 56 |
+
octave = int(name[i:])
|
| 57 |
+
break
|
| 58 |
+
else:
|
| 59 |
+
pitch, octave = name, 4
|
| 60 |
+
midi = 12 * (octave + 1) + NOTE[pitch]
|
| 61 |
+
return 440.0 * 2 ** ((midi - 69) / 12)
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def chord_to_freqs(chord_name, octave=3):
|
| 65 |
+
root, quality = parse_chord(chord_name)
|
| 66 |
+
intervals = CHORD_INTERVALS.get(quality, CHORD_INTERVALS[""])
|
| 67 |
+
root_midi = 12 * (octave + 1) + NOTE[root]
|
| 68 |
+
return [440.0 * 2 ** ((root_midi + i - 69) / 12) for i in intervals]
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
def add_at(track, audio, start_sec):
|
| 72 |
+
start = int(start_sec * SR)
|
| 73 |
+
end = start + len(audio)
|
| 74 |
+
if end > len(track):
|
| 75 |
+
track = np.pad(track, (0, end - len(track)))
|
| 76 |
+
track[start:end] += audio
|
| 77 |
+
return track
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def simple_reverb(audio, sr, room_size=0.6, damping=0.55, wet=0.22):
|
| 81 |
+
"""
|
| 82 |
+
Multi-tap delay reverb. Produces ~1.5-2s tail with diffusion.
|
| 83 |
+
|
| 84 |
+
The tail is built from 6 delayed, lowpassed, feedback-recirculated copies
|
| 85 |
+
of the input. Each tap has a slightly different delay (prime-ish ratios
|
| 86 |
+
to avoid metallic resonance) and decay factor.
|
| 87 |
+
|
| 88 |
+
Parameters:
|
| 89 |
+
room_size: 0..1 β scales tap delays (bigger = longer tail)
|
| 90 |
+
damping: 0..1 β lowpass cutoff scaling (more = darker tail)
|
| 91 |
+
wet: 0..1 β wet/dry mix (0 = dry, 1 = all reverb)
|
| 92 |
+
"""
|
| 93 |
+
if wet <= 0.0:
|
| 94 |
+
return audio
|
| 95 |
+
|
| 96 |
+
n = len(audio)
|
| 97 |
+
# Tap delays in ms (prime-ratio spread, scaled by room_size)
|
| 98 |
+
base_delays_ms = [29.0, 37.0, 53.0, 67.0, 89.0, 113.0]
|
| 99 |
+
# Per-tap gain β earlier taps stronger
|
| 100 |
+
gains = [0.55, 0.48, 0.42, 0.36, 0.30, 0.24]
|
| 101 |
+
|
| 102 |
+
# Build the wet signal
|
| 103 |
+
wet_buf = np.zeros(n + int(sr * 2.0), dtype=np.float32)
|
| 104 |
+
|
| 105 |
+
# Lowpass for the diffuse tail (darker as `damping` increases)
|
| 106 |
+
cutoff = max(800.0, 5000.0 * (1.0 - damping))
|
| 107 |
+
b_lp, a_lp = butter(2, cutoff / (sr / 2), btype="low")
|
| 108 |
+
pre_lp = lfilter(b_lp, a_lp, audio).astype(np.float32)
|
| 109 |
+
|
| 110 |
+
for ms, g in zip(base_delays_ms, gains):
|
| 111 |
+
delay_samples = int(ms * sr / 1000.0 * (0.6 + room_size * 0.8))
|
| 112 |
+
end = delay_samples + n
|
| 113 |
+
if end > len(wet_buf):
|
| 114 |
+
wet_buf = np.pad(wet_buf, (0, end - len(wet_buf)))
|
| 115 |
+
wet_buf[delay_samples:end] += pre_lp * g
|
| 116 |
+
|
| 117 |
+
# Feedback recirculation β pass the wet signal through a single combed
|
| 118 |
+
# delay to build a smooth tail. Decay set by `room_size`.
|
| 119 |
+
fb_delay = int(0.071 * sr * (0.6 + room_size * 0.8))
|
| 120 |
+
fb_gain = 0.45 + room_size * 0.20 # 0.45..0.65
|
| 121 |
+
for k in range(1, 5):
|
| 122 |
+
offset = fb_delay * k
|
| 123 |
+
if offset >= len(wet_buf):
|
| 124 |
+
break
|
| 125 |
+
wet_buf[offset:] += wet_buf[:-offset] * (fb_gain ** k) * 0.5
|
| 126 |
+
|
| 127 |
+
# Trim wet to length, lowpass again to smooth
|
| 128 |
+
wet_buf = wet_buf[:n]
|
| 129 |
+
wet_buf = lfilter(b_lp, a_lp, wet_buf).astype(np.float32)
|
| 130 |
+
|
| 131 |
+
return (audio * (1.0 - wet) + wet_buf * wet).astype(np.float32)
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
def _karplus_strong(freq, n_samples, sr, damping=0.9985):
|
| 135 |
+
"""
|
| 136 |
+
Karplus-Strong plucked string. Integer-sample delay line.
|
| 137 |
+
|
| 138 |
+
Note on tuning: at very high pitches (above A6 or so), the integer-sample
|
| 139 |
+
delay rounding introduces a small pitch error (a few cents). We accept
|
| 140 |
+
this because (a) musical content rarely sits above A6 in lullabies, and
|
| 141 |
+
(b) the perceived "out of tune" issue in chord contexts isn't this β it's
|
| 142 |
+
overlapping ring-out from previous chords. We mitigate that by fading the
|
| 143 |
+
track between chord changes in sequence().
|
| 144 |
+
|
| 145 |
+
Damping near 1.0 = long sustain. For the harp, ~0.9985 (very slow decay).
|
| 146 |
+
"""
|
| 147 |
+
delay_len = max(2, int(round(sr / freq)))
|
| 148 |
+
# Soft pre-filtered noise excitation β finger pad not pick
|
| 149 |
+
excitation = (np.random.rand(delay_len).astype(np.float32) * 2 - 1)
|
| 150 |
+
b, a = butter(2, [200 / (sr / 2), 3500 / (sr / 2)], btype="band")
|
| 151 |
+
excitation = lfilter(b, a, excitation).astype(np.float32)
|
| 152 |
+
|
| 153 |
+
buf = excitation.copy()
|
| 154 |
+
out = np.zeros(n_samples, dtype=np.float32)
|
| 155 |
+
prev = 0.0
|
| 156 |
+
i = 0
|
| 157 |
+
for s in range(n_samples):
|
| 158 |
+
cur = buf[i]
|
| 159 |
+
out[s] = cur
|
| 160 |
+
# One-pole lowpass averaging = string damping
|
| 161 |
+
new = damping * 0.5 * (cur + prev)
|
| 162 |
+
buf[i] = new
|
| 163 |
+
prev = cur
|
| 164 |
+
i = (i + 1) % delay_len
|
| 165 |
+
return out
|
| 166 |
+
|
| 167 |
+
|
| 168 |
+
class HarpSynth:
|
| 169 |
+
"""Concert / Celtic harp β long-sustain plucked string with warm H2."""
|
| 170 |
+
|
| 171 |
+
def __init__(self, sr=SR):
|
| 172 |
+
self.sr = sr
|
| 173 |
+
|
| 174 |
+
def note(self, freq, duration_s, volume=1.0, brightness=0.4):
|
| 175 |
+
"""
|
| 176 |
+
One plucked harp string.
|
| 177 |
+
|
| 178 |
+
brightness: 0..1 β controls additive H2/H3 level and pluck attack
|
| 179 |
+
noise. 0.4 matches the reference well.
|
| 180 |
+
"""
|
| 181 |
+
sr = self.sr
|
| 182 |
+
# Render long enough for the full decay tail β at least 2.5s.
|
| 183 |
+
# Reference decays to -40dB at ~1.7s and is inaudible by ~2.5s.
|
| 184 |
+
n = max(int(duration_s * sr), int(2.5 * sr))
|
| 185 |
+
t = np.arange(n) / sr
|
| 186 |
+
|
| 187 |
+
# Damping: scaled with pitch. Low strings sustain longer (concert harp
|
| 188 |
+
# bass strings ring ~3s; high strings ~1-1.5s).
|
| 189 |
+
damping = max(0.994, 0.9991 - (freq / 8000.0))
|
| 190 |
+
ks = _karplus_strong(freq, n, sr, damping=damping)
|
| 191 |
+
|
| 192 |
+
# Reference envelope: ~70ms plateau then exponential decay with tau
|
| 193 |
+
# such that -6dB @ 210ms, -20dB @ 770ms. tau β 0.33s.
|
| 194 |
+
plateau_end = 0.07
|
| 195 |
+
decay_tau = 0.33
|
| 196 |
+
env = np.ones(n, dtype=np.float32)
|
| 197 |
+
plateau_n = int(plateau_end * sr)
|
| 198 |
+
if plateau_n > 1:
|
| 199 |
+
env[:plateau_n] = np.linspace(0.85, 1.0, plateau_n)
|
| 200 |
+
decay_n = n - plateau_n
|
| 201 |
+
decay_t = np.arange(decay_n) / sr
|
| 202 |
+
env[plateau_n:] = np.exp(-decay_t / decay_tau)
|
| 203 |
+
|
| 204 |
+
ks = ks * env
|
| 205 |
+
|
| 206 |
+
# Additive partials to match reference spectrum:
|
| 207 |
+
# fundamental:H2 β 1:0.67 H3 β 1:0.06
|
| 208 |
+
# KS naturally produces some H2 but typically under-weights it for
|
| 209 |
+
# a warm harp tone. Layer explicit sines with their own envelopes.
|
| 210 |
+
h1_amp = 0.45
|
| 211 |
+
h2_amp = 0.30 + brightness * 0.10 # the warm "woody" component
|
| 212 |
+
h3_amp = 0.04 + brightness * 0.025
|
| 213 |
+
|
| 214 |
+
# Partials use the same envelope curve but with slightly faster decay
|
| 215 |
+
# for higher harmonics (real strings lose high frequencies first)
|
| 216 |
+
h2_env_tau = decay_tau * 0.75
|
| 217 |
+
h3_env_tau = decay_tau * 0.55
|
| 218 |
+
h2_env = np.ones(n, dtype=np.float32)
|
| 219 |
+
h3_env = np.ones(n, dtype=np.float32)
|
| 220 |
+
if plateau_n > 1:
|
| 221 |
+
h2_env[:plateau_n] = np.linspace(0.85, 1.0, plateau_n)
|
| 222 |
+
h3_env[:plateau_n] = np.linspace(0.85, 1.0, plateau_n)
|
| 223 |
+
h2_env[plateau_n:] = np.exp(-decay_t / h2_env_tau)
|
| 224 |
+
h3_env[plateau_n:] = np.exp(-decay_t / h3_env_tau)
|
| 225 |
+
|
| 226 |
+
partials = (h1_amp * np.sin(2 * np.pi * freq * t) * env
|
| 227 |
+
+ h2_amp * np.sin(2 * np.pi * freq * 2 * t) * h2_env
|
| 228 |
+
+ (h3_amp * np.sin(2 * np.pi * freq * 3 * t) * h3_env
|
| 229 |
+
if freq * 3 < sr / 2 * 0.9 else 0))
|
| 230 |
+
|
| 231 |
+
signal = ks * 0.55 + partials
|
| 232 |
+
|
| 233 |
+
# Soft attack ramp β no clicks
|
| 234 |
+
atk_n = int(0.004 * sr)
|
| 235 |
+
if atk_n > 1:
|
| 236 |
+
signal[:atk_n] *= np.linspace(0, 1, atk_n)
|
| 237 |
+
|
| 238 |
+
# Gentle release tail
|
| 239 |
+
rel_n = int(0.05 * sr)
|
| 240 |
+
if rel_n > 1:
|
| 241 |
+
signal[-rel_n:] *= np.linspace(1, 0, rel_n)
|
| 242 |
+
|
| 243 |
+
# Final lowpass for warm tone β kills any KS high-freq harshness
|
| 244 |
+
b_warm, a_warm = butter(2, 4000 / (sr / 2), btype="low")
|
| 245 |
+
signal = lfilter(b_warm, a_warm, signal).astype(np.float32)
|
| 246 |
+
|
| 247 |
+
# Normalize per-note for consistent loudness across pitches
|
| 248 |
+
peak = float(np.max(np.abs(signal)))
|
| 249 |
+
if peak > 1e-9:
|
| 250 |
+
signal = signal / peak * 0.85
|
| 251 |
+
return (signal * volume).astype(np.float32)
|
| 252 |
+
|
| 253 |
+
def chord(self, chord_name, duration_s, octave=3,
|
| 254 |
+
volume=0.7, brightness=0.4,
|
| 255 |
+
arpeggio_ms=130, direction="up",
|
| 256 |
+
sympathetic=True):
|
| 257 |
+
"""
|
| 258 |
+
Rolled chord arpeggio β the harp signature.
|
| 259 |
+
|
| 260 |
+
arpeggio_ms: 80-180ms between successive note onsets. 130ms is dreamy.
|
| 261 |
+
sympathetic: add faint lowpassed delayed copy for ringing-strings feel.
|
| 262 |
+
"""
|
| 263 |
+
freqs = chord_to_freqs(chord_name, octave=octave)
|
| 264 |
+
if direction == "down":
|
| 265 |
+
freqs = list(reversed(freqs))
|
| 266 |
+
|
| 267 |
+
stagger = arpeggio_ms / 1000.0
|
| 268 |
+
note_dur = max(duration_s, 2.5)
|
| 269 |
+
total_len = int((duration_s + stagger * len(freqs) + 3.0) * self.sr)
|
| 270 |
+
out = np.zeros(total_len, dtype=np.float32)
|
| 271 |
+
|
| 272 |
+
for i, freq in enumerate(freqs):
|
| 273 |
+
# Voicing: bass slightly stronger, top slightly softer
|
| 274 |
+
if i == 0:
|
| 275 |
+
voice_vol = 1.0
|
| 276 |
+
elif i == len(freqs) - 1:
|
| 277 |
+
voice_vol = 0.7
|
| 278 |
+
else:
|
| 279 |
+
voice_vol = 0.82
|
| 280 |
+
note_audio = self.note(freq, note_dur,
|
| 281 |
+
volume=volume * voice_vol,
|
| 282 |
+
brightness=brightness)
|
| 283 |
+
start = int(i * stagger * self.sr)
|
| 284 |
+
end = start + len(note_audio)
|
| 285 |
+
if end > len(out):
|
| 286 |
+
out = np.pad(out, (0, end - len(out)))
|
| 287 |
+
out[start:end] += note_audio
|
| 288 |
+
|
| 289 |
+
# Sympathetic resonance: delayed lowpassed copy mixed back at low gain
|
| 290 |
+
if sympathetic:
|
| 291 |
+
delay_samples = int(0.025 * self.sr)
|
| 292 |
+
b_sym, a_sym = butter(2, 1200 / (self.sr / 2), btype="low")
|
| 293 |
+
tail = lfilter(b_sym, a_sym, out).astype(np.float32)
|
| 294 |
+
shifted = np.zeros_like(out)
|
| 295 |
+
shifted[delay_samples:] = tail[:-delay_samples] * 0.18
|
| 296 |
+
out = out + shifted
|
| 297 |
+
|
| 298 |
+
# NOTE: hall-style reverb is applied in sequence() at the mix level
|
| 299 |
+
# (one pass over the whole performance) rather than per-chord, so we
|
| 300 |
+
# don't double-process.
|
| 301 |
+
|
| 302 |
+
# Normalize the rolled chord
|
| 303 |
+
peak = float(np.max(np.abs(out)))
|
| 304 |
+
if peak > 1.0:
|
| 305 |
+
out = out / peak
|
| 306 |
+
return out.astype(np.float32)
|
| 307 |
+
|
| 308 |
+
def sequence(self, events):
|
| 309 |
+
"""Render chord and/or note events."""
|
| 310 |
+
if not events:
|
| 311 |
+
return np.zeros(int(self.sr), dtype=np.float32)
|
| 312 |
+
|
| 313 |
+
events = sorted(events, key=lambda e: e["time"])
|
| 314 |
+
end_time = max(e["time"] + max(e["duration"], 0.6) for e in events) + 3.0
|
| 315 |
+
track = np.zeros(int(end_time * self.sr) + 1, dtype=np.float32)
|
| 316 |
+
|
| 317 |
+
# Render each event into its own buffer, then mix with chord-aware
|
| 318 |
+
# tail damping so previous chords' rings don't beat against the new
|
| 319 |
+
# chord's harmonics (the main perceived "out of tune" sound).
|
| 320 |
+
chord_events = [e for e in events if e.get("type") == "chord"]
|
| 321 |
+
chord_starts = [e["time"] for e in chord_events]
|
| 322 |
+
|
| 323 |
+
for ev in events:
|
| 324 |
+
typ = ev.get("type")
|
| 325 |
+
if typ == "chord":
|
| 326 |
+
audio = self.chord(
|
| 327 |
+
ev["name"], ev["duration"],
|
| 328 |
+
octave=ev.get("octave", 3),
|
| 329 |
+
volume=ev.get("volume", 0.7),
|
| 330 |
+
brightness=ev.get("brightness", 0.4),
|
| 331 |
+
arpeggio_ms=ev.get("arpeggio_ms", 130),
|
| 332 |
+
direction=ev.get("direction", "up"),
|
| 333 |
+
sympathetic=ev.get("sympathetic", True),
|
| 334 |
+
)
|
| 335 |
+
|
| 336 |
+
# Find the next chord boundary AFTER this one. Apply a
|
| 337 |
+
# quick exponential fade-down at that boundary on this
|
| 338 |
+
# chord's audio, so its harmonics stop ringing into the
|
| 339 |
+
# next chord. 250ms fade window β short enough to feel like
|
| 340 |
+
# natural string-damping (a harpist's palm muting), long
|
| 341 |
+
# enough not to click.
|
| 342 |
+
this_t = ev["time"]
|
| 343 |
+
next_chord_starts = [t for t in chord_starts if t > this_t + 0.1]
|
| 344 |
+
if next_chord_starts:
|
| 345 |
+
next_t = min(next_chord_starts)
|
| 346 |
+
# Where in `audio` does the next chord fall?
|
| 347 |
+
relative_next = next_t - this_t # seconds
|
| 348 |
+
fade_start_sample = int(relative_next * self.sr)
|
| 349 |
+
if 0 < fade_start_sample < len(audio):
|
| 350 |
+
# Exponential fade from 1.0 β 0.18 over 250ms,
|
| 351 |
+
# then hold at 0.18 (don't kill the tail entirely β
|
| 352 |
+
# we want a hint of overhang for naturalness)
|
| 353 |
+
fade_n = int(0.25 * self.sr)
|
| 354 |
+
end_sample = min(fade_start_sample + fade_n, len(audio))
|
| 355 |
+
n = end_sample - fade_start_sample
|
| 356 |
+
if n > 0:
|
| 357 |
+
curve = np.linspace(0, 1, n) ** 1.6
|
| 358 |
+
envelope = 1.0 - curve * (1.0 - 0.18)
|
| 359 |
+
audio[fade_start_sample:end_sample] = (
|
| 360 |
+
audio[fade_start_sample:end_sample] * envelope
|
| 361 |
+
)
|
| 362 |
+
# Beyond the fade window, hold at 0.18
|
| 363 |
+
if end_sample < len(audio):
|
| 364 |
+
audio[end_sample:] *= 0.18
|
| 365 |
+
|
| 366 |
+
elif typ == "note":
|
| 367 |
+
freq = note_to_hz(ev["name"])
|
| 368 |
+
audio = self.note(
|
| 369 |
+
freq, max(ev["duration"], 2.0),
|
| 370 |
+
volume=ev.get("volume", 1.0),
|
| 371 |
+
brightness=ev.get("brightness", 0.4),
|
| 372 |
+
)
|
| 373 |
+
else:
|
| 374 |
+
continue
|
| 375 |
+
track = add_at(track, audio, ev["time"])
|
| 376 |
+
|
| 377 |
+
# Apply hall-style reverb to the whole performance. A single pass
|
| 378 |
+
# over the mix is cheaper and more coherent than per-event reverb.
|
| 379 |
+
track = simple_reverb(track, self.sr,
|
| 380 |
+
room_size=0.55, damping=0.50, wet=0.22)
|
| 381 |
+
return track
|
synths_/musicbox.py
ADDED
|
@@ -0,0 +1,320 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Music box synthesizer β the quintessential lullaby instrument.
|
| 3 |
+
|
| 4 |
+
A music box uses steel tines plucked by pins on a rotating cylinder.
|
| 5 |
+
Acoustically, that produces a sound with very specific properties:
|
| 6 |
+
|
| 7 |
+
1) Near-pure sine fundamental (tines vibrate dominantly in their
|
| 8 |
+
fundamental mode, much more so than a struck-bar like xylophone)
|
| 9 |
+
2) A whisper of 2nd harmonic β slight metallic flavor
|
| 10 |
+
3) Brief inharmonic 'ping' at ~5-6Γ the fundamental β the stiffness
|
| 11 |
+
of the steel tine producing a non-integer-multiple partial that
|
| 12 |
+
fades fast (the metallic shimmer at attack)
|
| 13 |
+
4) Sharp, brief metallic attack transient β the pin-on-tine pluck.
|
| 14 |
+
Much sharper than a felt mallet, much more metallic than a piano
|
| 15 |
+
hammer. Brief HF burst (4-8 kHz) then immediate decay
|
| 16 |
+
5) Long slow decay (~2-3s for low tines, ~0.8-1.2s for high)
|
| 17 |
+
6) Distinctive 'bell' quality from the wooden box body that the tines
|
| 18 |
+
are mounted on β gives a slight reverb tail with low-mid emphasis
|
| 19 |
+
7) Sweet spot is the C5-C7 register; lower notes lose definition because
|
| 20 |
+
real music-box tines get physically large and unwieldy
|
| 21 |
+
|
| 22 |
+
Reverb is baked in (small wooden-box character, not a hall). This matters:
|
| 23 |
+
without the box body, the tines would sound thin and toy-like.
|
| 24 |
+
|
| 25 |
+
Public interface (mirrors other synths):
|
| 26 |
+
|
| 27 |
+
from synths.musicbox import MusicBoxSynth, SR
|
| 28 |
+
synth = MusicBoxSynth()
|
| 29 |
+
synth.sequence(events) β np.ndarray @ SR
|
| 30 |
+
|
| 31 |
+
Event format:
|
| 32 |
+
{"type": "chord", "name": "C", "time": 0.0, "duration": 3.0,
|
| 33 |
+
"octave": 5, "volume": 0.7, "spread_ms": 20, "direction": "up"}
|
| 34 |
+
{"type": "note", "name": "G5", "time": 0.0, "duration": 1.0,
|
| 35 |
+
"volume": 0.9}
|
| 36 |
+
"""
|
| 37 |
+
|
| 38 |
+
import numpy as np
|
| 39 |
+
from scipy.signal import butter, lfilter
|
| 40 |
+
|
| 41 |
+
SR = 44100
|
| 42 |
+
|
| 43 |
+
NOTE = {
|
| 44 |
+
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, "E": 4, "F": 5,
|
| 45 |
+
"F#": 6, "Gb": 6, "G": 7, "G#": 8, "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
|
| 46 |
+
}
|
| 47 |
+
|
| 48 |
+
CHORD_INTERVALS = {
|
| 49 |
+
"": [0, 4, 7, 12],
|
| 50 |
+
"m": [0, 3, 7, 12],
|
| 51 |
+
"7": [0, 4, 7, 10],
|
| 52 |
+
"m7": [0, 3, 7, 10],
|
| 53 |
+
"maj7": [0, 4, 7, 11],
|
| 54 |
+
"sus2": [0, 2, 7, 12],
|
| 55 |
+
"sus4": [0, 5, 7, 12],
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def parse_chord(name):
|
| 60 |
+
name = name.strip()
|
| 61 |
+
if len(name) >= 2 and name[1] in ("#", "b"):
|
| 62 |
+
root, quality = name[:2], name[2:]
|
| 63 |
+
else:
|
| 64 |
+
root, quality = name[:1], name[1:]
|
| 65 |
+
return root, quality
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
def note_to_hz(name):
|
| 69 |
+
"""e.g. 'A5' β 880.0"""
|
| 70 |
+
name = name.strip()
|
| 71 |
+
for i, ch in enumerate(name):
|
| 72 |
+
if ch.isdigit() or ch == "-":
|
| 73 |
+
pitch = name[:i]
|
| 74 |
+
octave = int(name[i:])
|
| 75 |
+
break
|
| 76 |
+
else:
|
| 77 |
+
pitch, octave = name, 5
|
| 78 |
+
midi = 12 * (octave + 1) + NOTE[pitch]
|
| 79 |
+
return 440.0 * 2 ** ((midi - 69) / 12)
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def chord_to_freqs(chord_name, octave=5):
|
| 83 |
+
root, quality = parse_chord(chord_name)
|
| 84 |
+
intervals = CHORD_INTERVALS.get(quality, CHORD_INTERVALS[""])
|
| 85 |
+
root_midi = 12 * (octave + 1) + NOTE[root]
|
| 86 |
+
return [440.0 * 2 ** ((root_midi + i - 69) / 12) for i in intervals]
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
def add_at(track, audio, start_sec, sr=SR):
|
| 90 |
+
start = int(start_sec * sr)
|
| 91 |
+
end = start + len(audio)
|
| 92 |
+
if end > len(track):
|
| 93 |
+
track = np.pad(track, (0, end - len(track)))
|
| 94 |
+
track[start:end] += audio
|
| 95 |
+
return track
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def box_reverb(audio, sr=SR, wet=0.20):
|
| 99 |
+
"""
|
| 100 |
+
Small wooden-box reverb β the resonance of the music box's housing.
|
| 101 |
+
Short (~400ms tail), dark, intimate β NOT a concert hall.
|
| 102 |
+
|
| 103 |
+
Implementation: 4 lowpassed delay taps + brief feedback. Cheap.
|
| 104 |
+
"""
|
| 105 |
+
if wet <= 0.0:
|
| 106 |
+
return audio
|
| 107 |
+
n = len(audio)
|
| 108 |
+
# Tight tap spread β small box, not a room
|
| 109 |
+
delay_ms = [11.0, 19.0, 29.0, 41.0]
|
| 110 |
+
gains = [0.55, 0.42, 0.32, 0.24]
|
| 111 |
+
|
| 112 |
+
# Pre-lowpass (the wooden box doesn't reflect high frequencies well)
|
| 113 |
+
b_lp, a_lp = butter(2, 2200 / (sr / 2), btype="low")
|
| 114 |
+
pre = lfilter(b_lp, a_lp, audio).astype(np.float32)
|
| 115 |
+
|
| 116 |
+
wet_buf = np.zeros(n + int(sr * 0.5), dtype=np.float32)
|
| 117 |
+
for ms, g in zip(delay_ms, gains):
|
| 118 |
+
d = int(ms * sr / 1000.0)
|
| 119 |
+
end = d + n
|
| 120 |
+
if end > len(wet_buf):
|
| 121 |
+
wet_buf = np.pad(wet_buf, (0, end - len(wet_buf)))
|
| 122 |
+
wet_buf[d:end] += pre * g
|
| 123 |
+
|
| 124 |
+
# Feedback for smooth decay
|
| 125 |
+
fb_delay = int(0.053 * sr)
|
| 126 |
+
for k in range(1, 3):
|
| 127 |
+
offset = fb_delay * k
|
| 128 |
+
if offset >= len(wet_buf):
|
| 129 |
+
break
|
| 130 |
+
wet_buf[offset:] += wet_buf[:-offset] * (0.35 ** k) * 0.5
|
| 131 |
+
|
| 132 |
+
wet_buf = wet_buf[:n]
|
| 133 |
+
wet_buf = lfilter(b_lp, a_lp, wet_buf).astype(np.float32)
|
| 134 |
+
|
| 135 |
+
return (audio * (1.0 - wet) + wet_buf * wet).astype(np.float32)
|
| 136 |
+
|
| 137 |
+
|
| 138 |
+
def _pluck_transient(freq, sr=SR, length_ms=22):
|
| 139 |
+
"""
|
| 140 |
+
The sharp metallic pin-on-tine pluck.
|
| 141 |
+
|
| 142 |
+
Two components:
|
| 143 |
+
(a) Brief broadband click in 2-8kHz (the pin striking the tine edge β
|
| 144 |
+
this is the 'tink' sound)
|
| 145 |
+
(b) A short transient sine burst at ~5Γ the fundamental (the inharmonic
|
| 146 |
+
stiffness partial of the tine β gives the metallic shimmer)
|
| 147 |
+
|
| 148 |
+
Decays in ~10ms. Much sharper than a felt mallet or piano hammer.
|
| 149 |
+
"""
|
| 150 |
+
n = int(length_ms / 1000 * sr)
|
| 151 |
+
if n <= 0:
|
| 152 |
+
return np.zeros(0, dtype=np.float32)
|
| 153 |
+
t = np.arange(n) / sr
|
| 154 |
+
|
| 155 |
+
# Component A: HF click
|
| 156 |
+
noise = np.random.randn(n).astype(np.float32)
|
| 157 |
+
low = 2000.0
|
| 158 |
+
high = min(sr / 2 * 0.95, 8000.0)
|
| 159 |
+
b, a = butter(2, [low / (sr / 2), high / (sr / 2)], btype="band")
|
| 160 |
+
click = lfilter(b, a, noise).astype(np.float32)
|
| 161 |
+
# Very fast decay β 2ms time constant
|
| 162 |
+
click_env = np.exp(-np.arange(n) / (sr * 0.002))
|
| 163 |
+
click = click * click_env * 0.18
|
| 164 |
+
|
| 165 |
+
# Component B: brief inharmonic stiffness partial (~5Γ fundamental)
|
| 166 |
+
# This is what gives music box its distinctive sweet metallic ping.
|
| 167 |
+
# Higher tines have a stronger one (smaller tines are more stiff-coupled).
|
| 168 |
+
if freq < 2000: # above this it'd alias or be inaudible
|
| 169 |
+
stiff_freq = freq * 5.4 # slightly inharmonic (not exact 5Γ)
|
| 170 |
+
if stiff_freq < sr / 2 * 0.9:
|
| 171 |
+
stiff_env = np.exp(-np.arange(n) / (sr * 0.008)) # 8ms decay
|
| 172 |
+
stiff_amp = 0.12 + min(0.10, freq / 8000)
|
| 173 |
+
stiff = stiff_amp * np.sin(2 * np.pi * stiff_freq * t) * stiff_env
|
| 174 |
+
click += stiff.astype(np.float32)
|
| 175 |
+
|
| 176 |
+
return click.astype(np.float32)
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
class MusicBoxSynth:
|
| 180 |
+
"""
|
| 181 |
+
Music box: steel tines plucked by cylinder pins.
|
| 182 |
+
Near-pure sine + faint H2 + brief inharmonic ping + long slow decay,
|
| 183 |
+
finished with a small wooden-box reverb.
|
| 184 |
+
"""
|
| 185 |
+
|
| 186 |
+
def __init__(self, sr=SR):
|
| 187 |
+
self.sr = sr
|
| 188 |
+
|
| 189 |
+
def note(self, freq, duration_s, volume=1.0):
|
| 190 |
+
"""
|
| 191 |
+
One plucked tine.
|
| 192 |
+
|
| 193 |
+
duration_s is mostly ignored beyond a minimum β the tine has its own
|
| 194 |
+
natural decay (~1-2.5s depending on pitch). You can't damp a music
|
| 195 |
+
box tine mid-note.
|
| 196 |
+
"""
|
| 197 |
+
sr = self.sr
|
| 198 |
+
# Bass tines sustain ~2.5s; treble ~0.8s.
|
| 199 |
+
# Use pitch-scaled tau: tau = 0.6 + (110/freq) * 0.5
|
| 200 |
+
# β at 110Hz: 1.1s tau (long); at 880Hz: 0.66s tau
|
| 201 |
+
tau = max(0.30, min(1.20, 0.6 + (110.0 / freq) * 0.5))
|
| 202 |
+
|
| 203 |
+
# Render to fully capture the decay tail (5Ο β -40dB)
|
| 204 |
+
n = max(int(duration_s * sr), int(min(3.0, tau * 5) * sr))
|
| 205 |
+
t = np.arange(n) / sr
|
| 206 |
+
|
| 207 |
+
# Fundamental β exponential decay, no plateau (tines don't plateau
|
| 208 |
+
# like soft-mallet bars do, they start decaying immediately).
|
| 209 |
+
fund_env = np.exp(-t / tau)
|
| 210 |
+
signal = (np.sin(2 * np.pi * freq * t) * fund_env).astype(np.float32)
|
| 211 |
+
|
| 212 |
+
# Whisper of 2nd harmonic β characteristic metallic flavor
|
| 213 |
+
# H2 amp ~0.06Γ fundamental, decays faster (tau Γ 0.5)
|
| 214 |
+
if freq * 2 < sr / 2 * 0.95:
|
| 215 |
+
h2_env = np.exp(-t / (tau * 0.5))
|
| 216 |
+
signal += (0.06 * np.sin(2 * np.pi * freq * 2 * t)
|
| 217 |
+
* h2_env).astype(np.float32)
|
| 218 |
+
|
| 219 |
+
# Tiny H3 for very low notes only β adds warmth in the bass register
|
| 220 |
+
if freq < 200 and freq * 3 < sr / 2 * 0.9:
|
| 221 |
+
h3_env = np.exp(-t / (tau * 0.4))
|
| 222 |
+
signal += (0.025 * np.sin(2 * np.pi * freq * 3 * t)
|
| 223 |
+
* h3_env).astype(np.float32)
|
| 224 |
+
|
| 225 |
+
# Stamp the pluck transient at the front
|
| 226 |
+
pluck = _pluck_transient(freq, sr=sr)
|
| 227 |
+
if len(pluck) > 0 and len(pluck) < n:
|
| 228 |
+
signal[:len(pluck)] += pluck
|
| 229 |
+
|
| 230 |
+
# Tiny attack ramp on the sine body (no DC click)
|
| 231 |
+
ramp_n = int(0.0015 * sr)
|
| 232 |
+
if ramp_n > 1:
|
| 233 |
+
signal[:ramp_n] *= np.linspace(0, 1, ramp_n)
|
| 234 |
+
|
| 235 |
+
# Normalize per-note then scale by volume
|
| 236 |
+
peak = float(np.max(np.abs(signal)))
|
| 237 |
+
if peak > 1e-9:
|
| 238 |
+
signal = signal / peak * 0.85
|
| 239 |
+
return (signal * volume).astype(np.float32)
|
| 240 |
+
|
| 241 |
+
def chord(self, chord_name, duration_s, octave=5,
|
| 242 |
+
volume=0.7, spread_ms=20, direction="up"):
|
| 243 |
+
"""
|
| 244 |
+
Render a chord. Real music boxes either pluck all tines simultaneously
|
| 245 |
+
(the cylinder pins for a chord line up) or sequentially (the cylinder
|
| 246 |
+
rotates through the notes). Both are valid.
|
| 247 |
+
|
| 248 |
+
spread_ms 0-40 = simultaneous chord; 100+ = sequential cylinder roll.
|
| 249 |
+
"""
|
| 250 |
+
freqs = chord_to_freqs(chord_name, octave=octave)
|
| 251 |
+
if direction == "down":
|
| 252 |
+
freqs = list(reversed(freqs))
|
| 253 |
+
|
| 254 |
+
n_keys = len(freqs)
|
| 255 |
+
stagger = (spread_ms / 1000.0) / max(1, n_keys - 1) if n_keys > 1 else 0
|
| 256 |
+
|
| 257 |
+
note_dur = max(duration_s, 1.5)
|
| 258 |
+
total_len = int((duration_s + stagger * n_keys + 3.0) * self.sr)
|
| 259 |
+
out = np.zeros(total_len, dtype=np.float32)
|
| 260 |
+
|
| 261 |
+
for i, freq in enumerate(freqs):
|
| 262 |
+
# Voicing: top voice slightly stronger (the melody-bearing tine
|
| 263 |
+
# in a real music box is often the prominent one). Bass slightly
|
| 264 |
+
# softer because low tines can rumble.
|
| 265 |
+
if i == n_keys - 1:
|
| 266 |
+
voice_vol = 1.0
|
| 267 |
+
elif i == 0:
|
| 268 |
+
voice_vol = 0.78
|
| 269 |
+
else:
|
| 270 |
+
voice_vol = 0.85
|
| 271 |
+
note_audio = self.note(freq, note_dur, volume=volume * voice_vol)
|
| 272 |
+
start = int(i * stagger * self.sr)
|
| 273 |
+
end = start + len(note_audio)
|
| 274 |
+
if end > len(out):
|
| 275 |
+
out = np.pad(out, (0, end - len(out)))
|
| 276 |
+
out[start:end] += note_audio
|
| 277 |
+
|
| 278 |
+
peak = float(np.max(np.abs(out)))
|
| 279 |
+
if peak > 1.0:
|
| 280 |
+
out = out / peak
|
| 281 |
+
return out.astype(np.float32)
|
| 282 |
+
|
| 283 |
+
def sequence(self, events):
|
| 284 |
+
"""Render mixed chord and note events. Applies the wooden-box reverb
|
| 285 |
+
once at the mix level."""
|
| 286 |
+
if not events:
|
| 287 |
+
return np.zeros(int(self.sr), dtype=np.float32)
|
| 288 |
+
|
| 289 |
+
events = sorted(events, key=lambda e: e["time"])
|
| 290 |
+
end_time = max(e["time"] + max(e["duration"], 0.6) for e in events) + 2.0
|
| 291 |
+
track = np.zeros(int(end_time * self.sr) + 1, dtype=np.float32)
|
| 292 |
+
|
| 293 |
+
for ev in events:
|
| 294 |
+
typ = ev.get("type")
|
| 295 |
+
if typ == "chord":
|
| 296 |
+
audio = self.chord(
|
| 297 |
+
ev["name"], ev["duration"],
|
| 298 |
+
octave=ev.get("octave", 5),
|
| 299 |
+
volume=ev.get("volume", 0.7),
|
| 300 |
+
spread_ms=ev.get("spread_ms", 20),
|
| 301 |
+
direction=ev.get("direction", "up"),
|
| 302 |
+
)
|
| 303 |
+
elif typ == "note":
|
| 304 |
+
freq = note_to_hz(ev["name"])
|
| 305 |
+
audio = self.note(
|
| 306 |
+
freq, ev["duration"],
|
| 307 |
+
volume=ev.get("volume", 1.0),
|
| 308 |
+
)
|
| 309 |
+
else:
|
| 310 |
+
continue
|
| 311 |
+
track = add_at(track, audio, ev["time"], sr=self.sr)
|
| 312 |
+
|
| 313 |
+
# Apply the wooden-box reverb at the mix level.
|
| 314 |
+
track = box_reverb(track, self.sr, wet=0.22)
|
| 315 |
+
|
| 316 |
+
# Safety normalize so overlapping rings don't clip.
|
| 317 |
+
peak = float(np.max(np.abs(track)))
|
| 318 |
+
if peak > 0.95:
|
| 319 |
+
track = track * (0.95 / peak)
|
| 320 |
+
return track
|
synths_/ocarina.py
ADDED
|
@@ -0,0 +1,197 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Whistle synthesizer β human-like whistling.
|
| 3 |
+
|
| 4 |
+
A whistle is acoustically simple: near-sine fundamental + small 2nd harmonic,
|
| 5 |
+
vibrato that fades in, soft envelope, light breath noise, smooth portamento
|
| 6 |
+
between notes.
|
| 7 |
+
|
| 8 |
+
Use as the MELODY layer over a rhythm instrument (guitar/piano/flute).
|
| 9 |
+
|
| 10 |
+
Example:
|
| 11 |
+
from whistle import WhistleSynth
|
| 12 |
+
synth = WhistleSynth()
|
| 13 |
+
events = [
|
| 14 |
+
{"type": "note", "name": "G5", "time": 0.0, "duration": 1.0},
|
| 15 |
+
{"type": "note", "name": "A5", "time": 1.0, "duration": 0.5},
|
| 16 |
+
{"type": "note", "name": "G5", "time": 1.5, "duration": 1.5},
|
| 17 |
+
]
|
| 18 |
+
audio = synth.sequence(events)
|
| 19 |
+
"""
|
| 20 |
+
|
| 21 |
+
import numpy as np
|
| 22 |
+
from scipy.signal import butter, lfilter
|
| 23 |
+
|
| 24 |
+
SR = 44100
|
| 25 |
+
|
| 26 |
+
NOTE = {
|
| 27 |
+
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, "E": 4, "F": 5,
|
| 28 |
+
"F#": 6, "Gb": 6, "G": 7, "G#": 8, "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def note_to_hz(name):
|
| 33 |
+
"""e.g. 'A4' β 440.0"""
|
| 34 |
+
name = name.strip()
|
| 35 |
+
# Find where the digit starts
|
| 36 |
+
for i, ch in enumerate(name):
|
| 37 |
+
if ch.isdigit() or ch == "-":
|
| 38 |
+
pitch = name[:i]
|
| 39 |
+
octave = int(name[i:])
|
| 40 |
+
break
|
| 41 |
+
else:
|
| 42 |
+
pitch, octave = name, 4
|
| 43 |
+
midi = 12 * (octave + 1) + NOTE[pitch]
|
| 44 |
+
return 440.0 * 2 ** ((midi - 69) / 12)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def add_at(track, audio, start_sec):
|
| 48 |
+
start = int(start_sec * SR)
|
| 49 |
+
end = start + len(audio)
|
| 50 |
+
if end > len(track):
|
| 51 |
+
track = np.pad(track, (0, end - len(track)))
|
| 52 |
+
track[start:end] += audio
|
| 53 |
+
return track
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
class WhistleSynth:
|
| 57 |
+
"""Synthesizes a melodic whistle from note events."""
|
| 58 |
+
|
| 59 |
+
def __init__(self, sr=SR):
|
| 60 |
+
self.sr = sr
|
| 61 |
+
|
| 62 |
+
def note(self, freq, duration_s,
|
| 63 |
+
vibrato_hz=4.8, vibrato_cents=32,
|
| 64 |
+
attack_ms=120, release_ms=400,
|
| 65 |
+
breath_level=0.018,
|
| 66 |
+
prev_freq=None, glide_ms=80,
|
| 67 |
+
volume=1.0):
|
| 68 |
+
"""One whistled note."""
|
| 69 |
+
sr = self.sr
|
| 70 |
+
n = int(duration_s * sr)
|
| 71 |
+
if n <= 0:
|
| 72 |
+
return np.zeros(0, dtype=np.float32)
|
| 73 |
+
t = np.arange(n) / sr
|
| 74 |
+
|
| 75 |
+
# Pitch curve with optional glide from previous note
|
| 76 |
+
if prev_freq is not None and glide_ms > 0:
|
| 77 |
+
glide_n = min(int(glide_ms / 1000 * sr), n // 3)
|
| 78 |
+
freq_curve = np.full(n, freq, dtype=np.float32)
|
| 79 |
+
if glide_n > 1:
|
| 80 |
+
freq_curve[:glide_n] = np.linspace(prev_freq, freq, glide_n)
|
| 81 |
+
else:
|
| 82 |
+
freq_curve = np.full(n, freq, dtype=np.float32)
|
| 83 |
+
|
| 84 |
+
# Vibrato fades in (whistlers don't start with wobble)
|
| 85 |
+
vibrato_env = np.minimum((t - 0.2) / 0.3, 1.0).clip(0, 1)
|
| 86 |
+
depth_hz = freq_curve * (2 ** (vibrato_cents / 1200) - 1)
|
| 87 |
+
vibrato = depth_hz * vibrato_env * np.sin(2 * np.pi * vibrato_hz * t)
|
| 88 |
+
inst_freq = freq_curve + vibrato
|
| 89 |
+
|
| 90 |
+
# Phase from instantaneous frequency
|
| 91 |
+
phase = 2 * np.pi * np.cumsum(inst_freq) / sr
|
| 92 |
+
|
| 93 |
+
# Tone: fundamental + soft 2nd harmonic
|
| 94 |
+
tone = np.sin(phase) + 0.06 * np.sin(2 * phase)
|
| 95 |
+
|
| 96 |
+
# Breath: bandpassed noise around the fundamental
|
| 97 |
+
noise = np.random.randn(n).astype(np.float32)
|
| 98 |
+
f_center = float(np.mean(freq_curve))
|
| 99 |
+
low = max(80.0, f_center * 0.8)
|
| 100 |
+
high = min(sr / 2 * 0.95, f_center * 1.6)
|
| 101 |
+
b, a = butter(2, [low / (sr / 2), high / (sr / 2)], btype="band")
|
| 102 |
+
breath = lfilter(b, a, noise).astype(np.float32) * breath_level
|
| 103 |
+
|
| 104 |
+
out = (tone + breath).astype(np.float32)
|
| 105 |
+
|
| 106 |
+
# Envelope
|
| 107 |
+
env = np.ones(n, dtype=np.float32)
|
| 108 |
+
atk_n = min(int(attack_ms / 1000 * sr), n // 3)
|
| 109 |
+
rel_n = min(int(release_ms / 1000 * sr), n // 2)
|
| 110 |
+
if atk_n > 1:
|
| 111 |
+
env[:atk_n] = np.linspace(0, 1, atk_n) ** 1.5
|
| 112 |
+
if rel_n > 1:
|
| 113 |
+
env[-rel_n:] = np.linspace(1, 0, rel_n) ** 1.5
|
| 114 |
+
|
| 115 |
+
return (out * env * volume).astype(np.float32)
|
| 116 |
+
|
| 117 |
+
def _explode_chord(self, ev):
|
| 118 |
+
"""
|
| 119 |
+
Break a chord event into a slow broken-chord pattern of note events.
|
| 120 |
+
|
| 121 |
+
Ocarina is monophonic, so a rhythm-role chord is rendered as a slow
|
| 122 |
+
ascending arpeggio that takes up the chord duration.
|
| 123 |
+
"""
|
| 124 |
+
name = ev["name"]
|
| 125 |
+
# Parse chord
|
| 126 |
+
if len(name) >= 2 and name[1] in ("#", "b"):
|
| 127 |
+
root, quality = name[:2], name[2:]
|
| 128 |
+
else:
|
| 129 |
+
root, quality = name[:1], name[1:]
|
| 130 |
+
is_minor = "m" in quality and "maj" not in quality
|
| 131 |
+
intervals = [0, (3 if is_minor else 4), 7] # root, third, fifth
|
| 132 |
+
|
| 133 |
+
octave = ev.get("octave", 5)
|
| 134 |
+
root_midi = 12 * (octave + 1) + NOTE[root]
|
| 135 |
+
names = ["C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"]
|
| 136 |
+
|
| 137 |
+
duration = ev["duration"]
|
| 138 |
+
t0 = ev["time"]
|
| 139 |
+
vol = ev.get("volume", 0.7)
|
| 140 |
+
# Three notes per chord, spread across most of the chord duration
|
| 141 |
+
note_dur = duration / 2.5
|
| 142 |
+
sub_events = []
|
| 143 |
+
for i, semi in enumerate(intervals):
|
| 144 |
+
midi = root_midi + semi
|
| 145 |
+
o = midi // 12 - 1
|
| 146 |
+
pitch_name = names[midi % 12]
|
| 147 |
+
sub_events.append({
|
| 148 |
+
"type": "note",
|
| 149 |
+
"name": f"{pitch_name}{o}",
|
| 150 |
+
"time": t0 + i * (duration / 3.5),
|
| 151 |
+
"duration": note_dur,
|
| 152 |
+
"volume": vol * (0.85 if i > 0 else 1.0),
|
| 153 |
+
})
|
| 154 |
+
return sub_events
|
| 155 |
+
|
| 156 |
+
def sequence(self, events):
|
| 157 |
+
"""
|
| 158 |
+
Render note and chord events. Chords become slow broken-chord
|
| 159 |
+
arpeggios (ocarina is monophonic).
|
| 160 |
+
"""
|
| 161 |
+
if not events:
|
| 162 |
+
return np.zeros(int(self.sr), dtype=np.float32)
|
| 163 |
+
|
| 164 |
+
# Expand chord events into note events
|
| 165 |
+
expanded = []
|
| 166 |
+
for ev in events:
|
| 167 |
+
if ev.get("type") == "chord":
|
| 168 |
+
expanded.extend(self._explode_chord(ev))
|
| 169 |
+
elif ev.get("type", "note") == "note":
|
| 170 |
+
expanded.append(ev)
|
| 171 |
+
|
| 172 |
+
events = sorted(expanded, key=lambda e: e["time"])
|
| 173 |
+
end_time = max(e["time"] + e["duration"] for e in events) + 0.5
|
| 174 |
+
track = np.zeros(int(end_time * self.sr) + 1, dtype=np.float32)
|
| 175 |
+
|
| 176 |
+
prev_freq = None
|
| 177 |
+
prev_end = -10.0
|
| 178 |
+
|
| 179 |
+
for ev in events:
|
| 180 |
+
freq = note_to_hz(ev["name"])
|
| 181 |
+
t = float(ev["time"])
|
| 182 |
+
dur = float(ev["duration"])
|
| 183 |
+
vol = float(ev.get("volume", 1.0))
|
| 184 |
+
|
| 185 |
+
# Use glide only if previous note ended very recently
|
| 186 |
+
glide_freq = prev_freq if (t - prev_end) < 0.25 else None
|
| 187 |
+
|
| 188 |
+
note_audio = self.note(
|
| 189 |
+
freq, dur,
|
| 190 |
+
prev_freq=glide_freq,
|
| 191 |
+
volume=vol,
|
| 192 |
+
)
|
| 193 |
+
track = add_at(track, note_audio, t)
|
| 194 |
+
prev_freq = freq
|
| 195 |
+
prev_end = t + dur
|
| 196 |
+
|
| 197 |
+
return track
|
synths_/piano.py
ADDED
|
@@ -0,0 +1,234 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Piano synthesizer β soft felt-piano lullaby tone.
|
| 3 |
+
|
| 4 |
+
Uses additive synthesis with a small handful of inharmonic partials, each
|
| 5 |
+
with its own decay envelope (higher partials decay faster β that's what
|
| 6 |
+
makes a piano sound like a piano vs an organ). A short stiff-attack
|
| 7 |
+
transient gives the hammer strike; a long sustain gives the body.
|
| 8 |
+
|
| 9 |
+
The "felt-piano" sound (a piano with felt strips between hammer and string)
|
| 10 |
+
is the soft, intimate, lullaby-friendly variant β less attack, more body,
|
| 11 |
+
fewer harmonics.
|
| 12 |
+
"""
|
| 13 |
+
|
| 14 |
+
import numpy as np
|
| 15 |
+
from scipy.signal import lfilter, butter
|
| 16 |
+
|
| 17 |
+
SR = 44100
|
| 18 |
+
|
| 19 |
+
NOTE = {
|
| 20 |
+
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, "E": 4, "F": 5,
|
| 21 |
+
"F#": 6, "Gb": 6, "G": 7, "G#": 8, "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
|
| 22 |
+
}
|
| 23 |
+
|
| 24 |
+
# Chord voicings: (note offsets in semitones from root) at the given octave
|
| 25 |
+
CHORD_INTERVALS = {
|
| 26 |
+
"": [0, 4, 7], # major
|
| 27 |
+
"m": [0, 3, 7], # minor
|
| 28 |
+
"7": [0, 4, 7, 10], # dom 7
|
| 29 |
+
"m7": [0, 3, 7, 10],
|
| 30 |
+
"maj7": [0, 4, 7, 11],
|
| 31 |
+
"sus2": [0, 2, 7],
|
| 32 |
+
"sus4": [0, 5, 7],
|
| 33 |
+
}
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def parse_chord(name):
|
| 37 |
+
"""'Am' β ('A', 'm'). 'C' β ('C', ''). 'Bbm7' β ('Bb', 'm7')."""
|
| 38 |
+
name = name.strip()
|
| 39 |
+
# Match 1-2 char root, rest is quality
|
| 40 |
+
if len(name) >= 2 and name[1] in ("#", "b"):
|
| 41 |
+
root, quality = name[:2], name[2:]
|
| 42 |
+
else:
|
| 43 |
+
root, quality = name[:1], name[1:]
|
| 44 |
+
return root, quality
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def note_to_hz(name):
|
| 48 |
+
"""e.g. 'A4' β 440.0"""
|
| 49 |
+
name = name.strip()
|
| 50 |
+
for i, ch in enumerate(name):
|
| 51 |
+
if ch.isdigit() or ch == "-":
|
| 52 |
+
pitch = name[:i]
|
| 53 |
+
octave = int(name[i:])
|
| 54 |
+
break
|
| 55 |
+
else:
|
| 56 |
+
pitch, octave = name, 4
|
| 57 |
+
midi = 12 * (octave + 1) + NOTE[pitch]
|
| 58 |
+
return 440.0 * 2 ** ((midi - 69) / 12)
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
def chord_to_freqs(chord_name, octave=4):
|
| 62 |
+
"""Return list of (frequency, relative_volume) for each note in the chord."""
|
| 63 |
+
root, quality = parse_chord(chord_name)
|
| 64 |
+
intervals = CHORD_INTERVALS.get(quality, CHORD_INTERVALS[""])
|
| 65 |
+
root_midi = 12 * (octave + 1) + NOTE[root]
|
| 66 |
+
|
| 67 |
+
notes = []
|
| 68 |
+
for i, interval in enumerate(intervals):
|
| 69 |
+
midi = root_midi + interval
|
| 70 |
+
hz = 440.0 * 2 ** ((midi - 69) / 12)
|
| 71 |
+
# Soften higher voices a touch β top of the chord shouldn't dominate
|
| 72 |
+
vol = 1.0 if i == 0 else (0.7 if i < 3 else 0.55)
|
| 73 |
+
notes.append((hz, vol))
|
| 74 |
+
return notes
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def add_at(track, audio, start_sec):
|
| 78 |
+
start = int(start_sec * SR)
|
| 79 |
+
end = start + len(audio)
|
| 80 |
+
if end > len(track):
|
| 81 |
+
track = np.pad(track, (0, end - len(track)))
|
| 82 |
+
track[start:end] += audio
|
| 83 |
+
return track
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
class PianoSynth:
|
| 87 |
+
"""Soft felt-piano synthesizer using additive synthesis."""
|
| 88 |
+
|
| 89 |
+
def __init__(self, sr=SR):
|
| 90 |
+
self.sr = sr
|
| 91 |
+
|
| 92 |
+
def note(self, freq, duration_s, volume=1.0, brightness=0.5):
|
| 93 |
+
"""
|
| 94 |
+
One piano note via additive synthesis.
|
| 95 |
+
|
| 96 |
+
brightness: 0..1 β controls how much energy is in the upper partials.
|
| 97 |
+
0.3 = warm/felt, 0.7 = bright/grand. Lullabies want ~0.4.
|
| 98 |
+
"""
|
| 99 |
+
sr = self.sr
|
| 100 |
+
n = int(duration_s * sr)
|
| 101 |
+
if n <= 0:
|
| 102 |
+
return np.zeros(0, dtype=np.float32)
|
| 103 |
+
t = np.arange(n) / sr
|
| 104 |
+
|
| 105 |
+
# Partial amplitudes β first 6 harmonics with falloff
|
| 106 |
+
# Brightness shifts the energy curve
|
| 107 |
+
base_amps = np.array([1.0, 0.55, 0.35, 0.22, 0.14, 0.08], dtype=np.float32)
|
| 108 |
+
# Apply brightness as a tilt β more brightness = less rolloff on highs
|
| 109 |
+
amps = base_amps * (1.0 + brightness * np.arange(len(base_amps)) * 0.1)
|
| 110 |
+
amps[0] = 1.0 # keep fundamental at unity
|
| 111 |
+
|
| 112 |
+
# Piano partials are slightly inharmonic (stiffness of real strings).
|
| 113 |
+
# The inharmonicity coefficient grows roughly as nΒ² but is small for
|
| 114 |
+
# the low-mid range β ~0.0004 for a real piano.
|
| 115 |
+
B = 0.0004
|
| 116 |
+
|
| 117 |
+
signal = np.zeros(n, dtype=np.float32)
|
| 118 |
+
for k in range(len(amps)):
|
| 119 |
+
partial = k + 1
|
| 120 |
+
# Inharmonic frequency
|
| 121 |
+
partial_freq = freq * partial * np.sqrt(1 + B * partial ** 2)
|
| 122 |
+
if partial_freq > sr / 2 * 0.95:
|
| 123 |
+
break
|
| 124 |
+
|
| 125 |
+
# Each partial has its own decay β higher partials die faster.
|
| 126 |
+
# Felt piano: fundamental rings ~4-6s, partial 6 dies in <1s.
|
| 127 |
+
decay_time = duration_s * (1.0 - 0.12 * k)
|
| 128 |
+
decay = np.exp(-t / max(decay_time, 0.2))
|
| 129 |
+
|
| 130 |
+
# Slight pitch detune per partial (real pianos have multiple strings
|
| 131 |
+
# per note with tiny detune that creates the chorus shimmer)
|
| 132 |
+
detune = 1.0 + (np.random.randn() * 0.0005 if k == 0 else 0)
|
| 133 |
+
phase = 2 * np.pi * partial_freq * detune * t
|
| 134 |
+
|
| 135 |
+
signal += amps[k] * decay * np.sin(phase)
|
| 136 |
+
|
| 137 |
+
# Hammer strike: very short noise burst at the start, lowpassed.
|
| 138 |
+
# This is what makes it sound percussive instead of bowed.
|
| 139 |
+
attack_n = min(int(0.012 * sr), n) # 12ms
|
| 140 |
+
strike = np.random.randn(attack_n).astype(np.float32) * 0.15
|
| 141 |
+
# Lowpass the strike so it's a thud, not a click
|
| 142 |
+
b_lp, a_lp = butter(2, 2500 / (sr / 2), btype="low")
|
| 143 |
+
strike = lfilter(b_lp, a_lp, strike).astype(np.float32)
|
| 144 |
+
strike_env = np.exp(-np.arange(attack_n) / (sr * 0.005))
|
| 145 |
+
signal[:attack_n] += (strike * strike_env).astype(np.float32)
|
| 146 |
+
|
| 147 |
+
# Soft attack envelope on the body (no clicks)
|
| 148 |
+
atk_n = min(int(0.008 * sr), n)
|
| 149 |
+
if atk_n > 1:
|
| 150 |
+
signal[:atk_n] *= np.linspace(0, 1, atk_n)
|
| 151 |
+
|
| 152 |
+
# Soft release to avoid clipping silence at end
|
| 153 |
+
rel_n = min(int(0.08 * sr), n // 2)
|
| 154 |
+
if rel_n > 1:
|
| 155 |
+
signal[-rel_n:] *= np.linspace(1, 0, rel_n)
|
| 156 |
+
|
| 157 |
+
# Gentle lowpass for "felt" character
|
| 158 |
+
b_warm, a_warm = butter(1, 6000 / (sr / 2), btype="low")
|
| 159 |
+
signal = lfilter(b_warm, a_warm, signal).astype(np.float32)
|
| 160 |
+
|
| 161 |
+
# Normalize per-note then scale by volume
|
| 162 |
+
peak = float(np.max(np.abs(signal)))
|
| 163 |
+
if peak > 1e-9:
|
| 164 |
+
signal = signal / peak * 0.85
|
| 165 |
+
return signal * volume
|
| 166 |
+
|
| 167 |
+
def chord(self, chord_name, duration_s, time, octave=4,
|
| 168 |
+
direction="down", spread_ms=20, volume=0.7, brightness=0.4):
|
| 169 |
+
"""
|
| 170 |
+
Render a chord as multiple piano notes with a tiny stagger.
|
| 171 |
+
|
| 172 |
+
For lullaby use, spread_ms=20-40 gives a gentle "rolled" feel like
|
| 173 |
+
a pianist landing slightly arpeggiated.
|
| 174 |
+
"""
|
| 175 |
+
notes = chord_to_freqs(chord_name, octave=octave)
|
| 176 |
+
if direction == "up":
|
| 177 |
+
notes = list(reversed(notes))
|
| 178 |
+
|
| 179 |
+
# Time-stagger via the spread
|
| 180 |
+
stagger = spread_ms / 1000.0
|
| 181 |
+
chord_audio = np.zeros(int((duration_s + stagger * len(notes) + 0.5) * self.sr),
|
| 182 |
+
dtype=np.float32)
|
| 183 |
+
|
| 184 |
+
for i, (freq, rel_vol) in enumerate(notes):
|
| 185 |
+
note_audio = self.note(freq, duration_s, volume=volume * rel_vol,
|
| 186 |
+
brightness=brightness)
|
| 187 |
+
start = int(i * stagger * self.sr)
|
| 188 |
+
end = start + len(note_audio)
|
| 189 |
+
if end > len(chord_audio):
|
| 190 |
+
chord_audio = np.pad(chord_audio, (0, end - len(chord_audio)))
|
| 191 |
+
chord_audio[start:end] += note_audio
|
| 192 |
+
return chord_audio
|
| 193 |
+
|
| 194 |
+
def sequence(self, events):
|
| 195 |
+
"""
|
| 196 |
+
Render a sequence of chord and/or note events.
|
| 197 |
+
|
| 198 |
+
chord event: {"type": "chord", "name": "Am", "time": 0.0, "duration": 3.0, ...}
|
| 199 |
+
note event: {"type": "note", "name": "A4", "time": 0.0, "duration": 1.0, ...}
|
| 200 |
+
"""
|
| 201 |
+
if not events:
|
| 202 |
+
return np.zeros(int(self.sr), dtype=np.float32)
|
| 203 |
+
|
| 204 |
+
events = sorted(events, key=lambda e: e["time"])
|
| 205 |
+
end_time = max(e["time"] + e["duration"] for e in events) + 2.0
|
| 206 |
+
track = np.zeros(int(end_time * self.sr) + 1, dtype=np.float32)
|
| 207 |
+
|
| 208 |
+
for ev in events:
|
| 209 |
+
typ = ev.get("type")
|
| 210 |
+
if typ == "chord":
|
| 211 |
+
audio = self.chord(
|
| 212 |
+
ev["name"],
|
| 213 |
+
ev["duration"],
|
| 214 |
+
ev["time"],
|
| 215 |
+
octave=ev.get("octave", 4),
|
| 216 |
+
direction=ev.get("direction", "down"),
|
| 217 |
+
spread_ms=ev.get("spread_ms", 25),
|
| 218 |
+
volume=ev.get("volume", 0.7),
|
| 219 |
+
brightness=ev.get("brightness", 0.4),
|
| 220 |
+
)
|
| 221 |
+
elif typ == "note":
|
| 222 |
+
freq = note_to_hz(ev["name"])
|
| 223 |
+
# Piano notes ring out β extend duration for natural decay tail
|
| 224 |
+
audio = self.note(
|
| 225 |
+
freq,
|
| 226 |
+
ev["duration"] + 1.0,
|
| 227 |
+
volume=ev.get("volume", 0.85),
|
| 228 |
+
brightness=ev.get("brightness", 0.4),
|
| 229 |
+
)
|
| 230 |
+
else:
|
| 231 |
+
continue
|
| 232 |
+
track = add_at(track, audio, ev["time"])
|
| 233 |
+
|
| 234 |
+
return track
|
synths_/voice.py
ADDED
|
@@ -0,0 +1,170 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Voice synthesis for the lullaby app β Kokoro TTS, soft preset.
|
| 3 |
+
|
| 4 |
+
Reads the lyrics gently, slowly, with natural breaths between lines.
|
| 5 |
+
No singing, no pitch manipulation β just a warm reading voice over music.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
import numpy as np
|
| 9 |
+
|
| 10 |
+
SR_TARGET = 44100
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
# We always use the soft voice β chosen for lullaby use.
|
| 14 |
+
VOICE_DEFAULT = "af_nicole"
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
_pipeline = None
|
| 18 |
+
_kokoro_sr = 24000
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def _load_kokoro():
|
| 22 |
+
global _pipeline
|
| 23 |
+
if _pipeline is not None:
|
| 24 |
+
return True
|
| 25 |
+
try:
|
| 26 |
+
import torch
|
| 27 |
+
from kokoro import KPipeline
|
| 28 |
+
print("Loading Kokoro (first run downloads ~80MB)...")
|
| 29 |
+
# Force CPU explicitly. On ZeroGPU the main process has no real GPU,
|
| 30 |
+
# but KPipeline auto-detects via torch.cuda.is_available() which can
|
| 31 |
+
# be poisoned by spaces wrapper state. Forcing CPU avoids the silent
|
| 32 |
+
# "model loaded on CUDA but no GPU actually present" failure mode
|
| 33 |
+
# that produces empty audio (~0 samples) downstream.
|
| 34 |
+
_pipeline = KPipeline(
|
| 35 |
+
lang_code='a',
|
| 36 |
+
repo_id='hexgrad/Kokoro-82M',
|
| 37 |
+
device='cpu',
|
| 38 |
+
)
|
| 39 |
+
# Belt-and-braces: if the kokoro version we got ignores device=,
|
| 40 |
+
# walk the pipeline's tensors and move them. Wrapped in try/except
|
| 41 |
+
# because the internal attribute name varies across kokoro releases.
|
| 42 |
+
try:
|
| 43 |
+
if hasattr(_pipeline, 'model') and _pipeline.model is not None:
|
| 44 |
+
_pipeline.model = _pipeline.model.to('cpu')
|
| 45 |
+
except Exception as move_err:
|
| 46 |
+
print(f"[kokoro] could not force model to CPU ({move_err}); "
|
| 47 |
+
f"continuing β device= kwarg should have handled it")
|
| 48 |
+
print("Kokoro loaded (CPU).")
|
| 49 |
+
return True
|
| 50 |
+
except TypeError:
|
| 51 |
+
# Older kokoro versions don't accept device=. Fall back to the
|
| 52 |
+
# constructor without it; KPipeline will auto-detect.
|
| 53 |
+
try:
|
| 54 |
+
from kokoro import KPipeline
|
| 55 |
+
print("Loading Kokoro (older API, no device kwarg)...")
|
| 56 |
+
_pipeline = KPipeline(lang_code='a', repo_id='hexgrad/Kokoro-82M')
|
| 57 |
+
print("Kokoro loaded.")
|
| 58 |
+
return True
|
| 59 |
+
except Exception as e:
|
| 60 |
+
print(f"Kokoro unavailable: {e}")
|
| 61 |
+
return False
|
| 62 |
+
except Exception as e:
|
| 63 |
+
print(f"Kokoro unavailable: {e}")
|
| 64 |
+
return False
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def _resample(x, sr_from, sr_to):
|
| 68 |
+
if sr_from == sr_to:
|
| 69 |
+
return x
|
| 70 |
+
n_out = int(len(x) * sr_to / sr_from)
|
| 71 |
+
return np.interp(
|
| 72 |
+
np.linspace(0, len(x) - 1, n_out),
|
| 73 |
+
np.arange(len(x)),
|
| 74 |
+
x,
|
| 75 |
+
).astype(np.float32)
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
def _silence(seconds):
|
| 79 |
+
return np.zeros(int(seconds * SR_TARGET), dtype=np.float32)
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def _gentle_vocal_eq(audio):
|
| 83 |
+
"""Soften vocal tone β gentle low-pass + mild high-pass."""
|
| 84 |
+
from scipy.signal import butter, lfilter
|
| 85 |
+
b_lp, a_lp = butter(2, 5500 / (SR_TARGET / 2), btype="low")
|
| 86 |
+
b_hp, a_hp = butter(1, 110 / (SR_TARGET / 2), btype="high")
|
| 87 |
+
out = lfilter(b_lp, a_lp, audio)
|
| 88 |
+
out = lfilter(b_hp, a_hp, out)
|
| 89 |
+
return out.astype(np.float32)
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
def speak_lyrics(lyrics, target_seconds=None, speed=0.85):
|
| 93 |
+
"""
|
| 94 |
+
Render lyrics as gentle spoken voice. Returns mono float32 at SR_TARGET.
|
| 95 |
+
|
| 96 |
+
speed=0.85 β slightly slower than normal, bedtime pacing.
|
| 97 |
+
target_seconds (if given) β pad with intro silence so voice ends near track end.
|
| 98 |
+
"""
|
| 99 |
+
if not _load_kokoro():
|
| 100 |
+
print("WARNING: TTS unavailable, returning silent vocal track")
|
| 101 |
+
return _silence(target_seconds or 1.0)
|
| 102 |
+
|
| 103 |
+
try:
|
| 104 |
+
chunks = []
|
| 105 |
+
generator = _pipeline(
|
| 106 |
+
lyrics,
|
| 107 |
+
voice=VOICE_DEFAULT,
|
| 108 |
+
speed=speed,
|
| 109 |
+
split_pattern=r'\n+',
|
| 110 |
+
)
|
| 111 |
+
for _, _, audio in generator:
|
| 112 |
+
audio_np = np.asarray(audio, dtype=np.float32)
|
| 113 |
+
if len(audio_np) == 0:
|
| 114 |
+
continue
|
| 115 |
+
chunks.append(audio_np)
|
| 116 |
+
# 350ms breath between phrases
|
| 117 |
+
chunks.append(np.zeros(int(0.35 * _kokoro_sr), dtype=np.float32))
|
| 118 |
+
if not chunks:
|
| 119 |
+
return _silence(target_seconds or 1.0)
|
| 120 |
+
audio = np.concatenate(chunks)
|
| 121 |
+
audio = _resample(audio, _kokoro_sr, SR_TARGET)
|
| 122 |
+
audio = _gentle_vocal_eq(audio)
|
| 123 |
+
|
| 124 |
+
# Pad with intro silence if requested
|
| 125 |
+
if target_seconds is not None:
|
| 126 |
+
target_n = int(target_seconds * SR_TARGET)
|
| 127 |
+
if len(audio) < target_n:
|
| 128 |
+
intro = min(target_n - len(audio), 4 * SR_TARGET)
|
| 129 |
+
audio = np.concatenate([_silence(intro / SR_TARGET), audio])
|
| 130 |
+
return audio
|
| 131 |
+
except Exception as e:
|
| 132 |
+
print(f"Kokoro generation error: {e}")
|
| 133 |
+
return _silence(target_seconds or 1.0)
|
| 134 |
+
|
| 135 |
+
|
| 136 |
+
# Back-compat alias for code that imports the old name.
|
| 137 |
+
def synthesize_singing(lyrics, target_seconds=20.0, voice_style="soft", **kwargs):
|
| 138 |
+
return speak_lyrics(lyrics, target_seconds=target_seconds)
|
| 139 |
+
|
| 140 |
+
|
| 141 |
+
def warmup():
|
| 142 |
+
"""
|
| 143 |
+
Fully warm the TTS path at startup so the first real user click is fast.
|
| 144 |
+
|
| 145 |
+
_load_kokoro() alone only builds the KPipeline β it does NOT download the
|
| 146 |
+
voice tensor (voices/af_nicole.pt) or compile the inference path. Those
|
| 147 |
+
happen lazily on the FIRST _pipeline(...) call, i.e. during the first real
|
| 148 |
+
generation, which is exactly the stall you see in the logs between
|
| 149 |
+
"combining drawing + typed" and the af_nicole.pt download.
|
| 150 |
+
|
| 151 |
+
Running one tiny throwaway synthesis here forces:
|
| 152 |
+
- the af_nicole.pt voice download, and
|
| 153 |
+
- the first-inference graph/setup cost
|
| 154 |
+
to happen at boot instead of on the user's request. Returns True if the
|
| 155 |
+
voice path is warm.
|
| 156 |
+
"""
|
| 157 |
+
if not _load_kokoro():
|
| 158 |
+
return False
|
| 159 |
+
try:
|
| 160 |
+
# Tiny utterance β we throw the audio away. Just enough to trigger the
|
| 161 |
+
# voice download and exercise one full generate pass.
|
| 162 |
+
gen = _pipeline("ok", voice=VOICE_DEFAULT, speed=1.0,
|
| 163 |
+
split_pattern=r"\n+")
|
| 164 |
+
for _ in gen:
|
| 165 |
+
pass
|
| 166 |
+
print("Kokoro voice warm (af_nicole loaded).")
|
| 167 |
+
return True
|
| 168 |
+
except Exception as e:
|
| 169 |
+
print(f"WARNING: Kokoro voice warmup failed (will warm lazily): {e}")
|
| 170 |
+
return False
|
synths_/xylophone.py
ADDED
|
@@ -0,0 +1,298 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Xylophone synthesizer β based on spectral analysis of a real soft-mallet
|
| 3 |
+
metallophone/xylophone loop reference.
|
| 4 |
+
|
| 5 |
+
Analysis of the reference (van_wiese xylophone loop, 125 BPM):
|
| 6 |
+
- Fundamental at 463 Hz (A#4) with 2nd harmonic at 931 Hz
|
| 7 |
+
- Amplitude ratio fundamental:2nd β 32:1 (essentially a pure sine + whisper of H2)
|
| 8 |
+
- Decay to -6dB in 86ms
|
| 9 |
+
- Decay to -20dB in ~300ms
|
| 10 |
+
- No prominent inharmonic partials (cleaner than a true wooden xylophone)
|
| 11 |
+
- Bright, near-pure tone β closer to a tuned soft-mallet metallophone
|
| 12 |
+
|
| 13 |
+
This synth models that: an almost-sine fundamental, a faint 2nd harmonic,
|
| 14 |
+
a tiny inharmonic shimmer for "real wood/metal" character, a brief filtered
|
| 15 |
+
strike transient, and a fast exponential decay.
|
| 16 |
+
|
| 17 |
+
Discrete hits, no tremolo, no roll β single mallet strikes like the reference.
|
| 18 |
+
For chord roles, we play a fast arpeggio of single hits instead of a tremolo.
|
| 19 |
+
"""
|
| 20 |
+
|
| 21 |
+
import numpy as np
|
| 22 |
+
from scipy.signal import lfilter, butter
|
| 23 |
+
|
| 24 |
+
SR = 44100
|
| 25 |
+
|
| 26 |
+
NOTE = {
|
| 27 |
+
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, "E": 4, "F": 5,
|
| 28 |
+
"F#": 6, "Gb": 6, "G": 7, "G#": 8, "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
CHORD_INTERVALS = {
|
| 32 |
+
"": [0, 4, 7],
|
| 33 |
+
"m": [0, 3, 7],
|
| 34 |
+
"7": [0, 4, 7, 10],
|
| 35 |
+
"m7": [0, 3, 7, 10],
|
| 36 |
+
"maj7": [0, 4, 7, 11],
|
| 37 |
+
"sus2": [0, 2, 7],
|
| 38 |
+
"sus4": [0, 5, 7],
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def parse_chord(name):
|
| 43 |
+
name = name.strip()
|
| 44 |
+
if len(name) >= 2 and name[1] in ("#", "b"):
|
| 45 |
+
root, quality = name[:2], name[2:]
|
| 46 |
+
else:
|
| 47 |
+
root, quality = name[:1], name[1:]
|
| 48 |
+
return root, quality
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def note_to_hz(name):
|
| 52 |
+
"""'A5' β 880.0"""
|
| 53 |
+
name = name.strip()
|
| 54 |
+
for i, ch in enumerate(name):
|
| 55 |
+
if ch.isdigit() or ch == "-":
|
| 56 |
+
pitch = name[:i]
|
| 57 |
+
octave = int(name[i:])
|
| 58 |
+
break
|
| 59 |
+
else:
|
| 60 |
+
pitch, octave = name, 5
|
| 61 |
+
midi = 12 * (octave + 1) + NOTE[pitch]
|
| 62 |
+
return 440.0 * 2 ** ((midi - 69) / 12)
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
def chord_to_freqs(chord_name, octave=4):
|
| 66 |
+
root, quality = parse_chord(chord_name)
|
| 67 |
+
intervals = CHORD_INTERVALS.get(quality, CHORD_INTERVALS[""])
|
| 68 |
+
root_midi = 12 * (octave + 1) + NOTE[root]
|
| 69 |
+
return [440.0 * 2 ** ((root_midi + i - 69) / 12) for i in intervals]
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def add_at(track, audio, start_sec):
|
| 73 |
+
start = int(start_sec * SR)
|
| 74 |
+
end = start + len(audio)
|
| 75 |
+
if end > len(track):
|
| 76 |
+
track = np.pad(track, (0, end - len(track)))
|
| 77 |
+
track[start:end] += audio
|
| 78 |
+
return track
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def simple_reverb(audio, sr, room_size=0.5, damping=0.55, wet=0.30):
|
| 82 |
+
"""
|
| 83 |
+
Multi-tap delay reverb. Adds ~1.5s tail with diffusion.
|
| 84 |
+
|
| 85 |
+
For xylophone we want more wet than the harp (xylo strikes are short
|
| 86 |
+
and dry sounds toy-like). 6 lowpassed delayed taps + a recirculating
|
| 87 |
+
feedback delay build a smooth tail without metallic resonance.
|
| 88 |
+
|
| 89 |
+
Parameters:
|
| 90 |
+
room_size: 0..1 β scales tap delays (bigger = longer tail)
|
| 91 |
+
damping: 0..1 β lowpass cutoff scaling (more = darker tail)
|
| 92 |
+
wet: 0..1 β wet/dry mix
|
| 93 |
+
"""
|
| 94 |
+
if wet <= 0.0:
|
| 95 |
+
return audio
|
| 96 |
+
|
| 97 |
+
n = len(audio)
|
| 98 |
+
base_delays_ms = [29.0, 37.0, 53.0, 67.0, 89.0, 113.0]
|
| 99 |
+
gains = [0.55, 0.48, 0.42, 0.36, 0.30, 0.24]
|
| 100 |
+
|
| 101 |
+
wet_buf = np.zeros(n + int(sr * 2.0), dtype=np.float32)
|
| 102 |
+
|
| 103 |
+
cutoff = max(800.0, 5000.0 * (1.0 - damping))
|
| 104 |
+
b_lp, a_lp = butter(2, cutoff / (sr / 2), btype="low")
|
| 105 |
+
pre_lp = lfilter(b_lp, a_lp, audio).astype(np.float32)
|
| 106 |
+
|
| 107 |
+
for ms, g in zip(base_delays_ms, gains):
|
| 108 |
+
delay_samples = int(ms * sr / 1000.0 * (0.6 + room_size * 0.8))
|
| 109 |
+
end = delay_samples + n
|
| 110 |
+
if end > len(wet_buf):
|
| 111 |
+
wet_buf = np.pad(wet_buf, (0, end - len(wet_buf)))
|
| 112 |
+
wet_buf[delay_samples:end] += pre_lp * g
|
| 113 |
+
|
| 114 |
+
fb_delay = int(0.071 * sr * (0.6 + room_size * 0.8))
|
| 115 |
+
fb_gain = 0.45 + room_size * 0.20
|
| 116 |
+
for k in range(1, 5):
|
| 117 |
+
offset = fb_delay * k
|
| 118 |
+
if offset >= len(wet_buf):
|
| 119 |
+
break
|
| 120 |
+
wet_buf[offset:] += wet_buf[:-offset] * (fb_gain ** k) * 0.5
|
| 121 |
+
|
| 122 |
+
wet_buf = wet_buf[:n]
|
| 123 |
+
wet_buf = lfilter(b_lp, a_lp, wet_buf).astype(np.float32)
|
| 124 |
+
|
| 125 |
+
return (audio * (1.0 - wet) + wet_buf * wet).astype(np.float32)
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
class XylophoneSynth:
|
| 129 |
+
"""
|
| 130 |
+
Soft-mallet metallophone/xylophone modeled on a real reference loop.
|
| 131 |
+
|
| 132 |
+
Characteristics:
|
| 133 |
+
- Near-pure sine fundamental
|
| 134 |
+
- Whisper of 2nd harmonic (~1/30 the amplitude of fundamental)
|
| 135 |
+
- Tiny inharmonic flavor for body
|
| 136 |
+
- Fast exponential decay (~85ms to -6dB, ~300ms to -20dB)
|
| 137 |
+
- Brief click-like strike transient
|
| 138 |
+
"""
|
| 139 |
+
|
| 140 |
+
def __init__(self, sr=SR):
|
| 141 |
+
self.sr = sr
|
| 142 |
+
|
| 143 |
+
def strike(self, freq, volume=1.0, brightness=0.5):
|
| 144 |
+
"""
|
| 145 |
+
One mallet strike.
|
| 146 |
+
|
| 147 |
+
Models the reference envelope: brief attack peak, ~100ms near-peak
|
| 148 |
+
plateau (bar body sustain), then slow exponential decay over ~600ms.
|
| 149 |
+
Full ring-out by ~800ms.
|
| 150 |
+
|
| 151 |
+
brightness: 0..1 β affects strike click intensity and H2 level.
|
| 152 |
+
0.5 matches the reference well.
|
| 153 |
+
"""
|
| 154 |
+
sr = self.sr
|
| 155 |
+
# 900ms render β long enough for the full decay tail
|
| 156 |
+
n = int(0.9 * sr)
|
| 157 |
+
t = np.arange(n) / sr
|
| 158 |
+
|
| 159 |
+
# Reference fundamental envelope: stays near-peak until ~140ms, then
|
| 160 |
+
# decays to ~0.25 by 300ms, ~0.10 by 500ms. That's a slow exponential
|
| 161 |
+
# with tau β 220ms, but with a brief plateau at the start.
|
| 162 |
+
#
|
| 163 |
+
# Model: attack ramp β short plateau β exponential decay
|
| 164 |
+
plateau_end = 0.12 # 120ms plateau
|
| 165 |
+
decay_tau = 0.22 # slow decay constant
|
| 166 |
+
|
| 167 |
+
fund_env = np.ones(n, dtype=np.float32)
|
| 168 |
+
# Plateau phase: gentle taper from 1.0 to 0.95
|
| 169 |
+
plateau_n = int(plateau_end * sr)
|
| 170 |
+
fund_env[:plateau_n] = np.linspace(1.0, 0.95, plateau_n)
|
| 171 |
+
# Decay phase: exponential from 0.95 down
|
| 172 |
+
decay_n = n - plateau_n
|
| 173 |
+
decay_t = np.arange(decay_n) / sr
|
| 174 |
+
fund_env[plateau_n:] = 0.95 * np.exp(-decay_t / decay_tau)
|
| 175 |
+
|
| 176 |
+
# H2 β barely audible, dies faster than fundamental (no plateau)
|
| 177 |
+
h2_amp = 0.04 + brightness * 0.025
|
| 178 |
+
tau_h2 = 0.090
|
| 179 |
+
h2_env = np.exp(-t / tau_h2)
|
| 180 |
+
|
| 181 |
+
signal = (np.sin(2 * np.pi * freq * t) * fund_env
|
| 182 |
+
+ h2_amp * np.sin(2 * np.pi * freq * 2 * t) * h2_env)
|
| 183 |
+
|
| 184 |
+
# Faint inharmonic shimmer at attack only β gives "real bar" texture
|
| 185 |
+
partial_3_freq = freq * 4.2
|
| 186 |
+
if partial_3_freq < sr / 2 * 0.9:
|
| 187 |
+
partial_3_env = np.exp(-t / 0.025)
|
| 188 |
+
partial_3_amp = 0.04 * brightness
|
| 189 |
+
signal += partial_3_amp * np.sin(2 * np.pi * partial_3_freq * t) * partial_3_env
|
| 190 |
+
|
| 191 |
+
# Strike transient: brief filtered noise burst.
|
| 192 |
+
attack_n = int(0.006 * sr) # 6ms
|
| 193 |
+
if attack_n > 0:
|
| 194 |
+
noise = np.random.randn(attack_n).astype(np.float32) * 0.10
|
| 195 |
+
b, a = butter(2,
|
| 196 |
+
[1500 / (sr / 2), min(0.95, 4500 / (sr / 2))],
|
| 197 |
+
btype="band")
|
| 198 |
+
click = lfilter(b, a, noise).astype(np.float32)
|
| 199 |
+
click_env = np.exp(-np.arange(attack_n) / (sr * 0.002))
|
| 200 |
+
click = (click * click_env * (0.5 + brightness * 0.4)).astype(np.float32)
|
| 201 |
+
signal[:attack_n] += click
|
| 202 |
+
|
| 203 |
+
# Tiny attack ramp on the body (no DC pop)
|
| 204 |
+
ramp_n = int(0.001 * sr)
|
| 205 |
+
if ramp_n > 1:
|
| 206 |
+
signal[:ramp_n] *= np.linspace(0, 1, ramp_n)
|
| 207 |
+
|
| 208 |
+
# Normalize per-strike to a known peak, then scale by volume
|
| 209 |
+
peak = float(np.max(np.abs(signal)))
|
| 210 |
+
if peak > 1e-9:
|
| 211 |
+
signal = signal / peak * 0.80
|
| 212 |
+
|
| 213 |
+
return (signal * volume).astype(np.float32)
|
| 214 |
+
|
| 215 |
+
def chord(self, chord_name, duration_s, octave=4,
|
| 216 |
+
volume=0.7, brightness=0.5,
|
| 217 |
+
arpeggio_ms=60, direction="up"):
|
| 218 |
+
"""
|
| 219 |
+
Render a chord as a quick arpeggio of individual strikes (not a roll).
|
| 220 |
+
|
| 221 |
+
Real mallet players play chords as fast arpeggios from low to high
|
| 222 |
+
(or high to low). arpeggio_ms is the delay between strikes. 40-80ms
|
| 223 |
+
feels right for lullaby pacing.
|
| 224 |
+
|
| 225 |
+
The 'duration_s' is the chord's total time-slot, but each strike
|
| 226 |
+
decays on its own ~500ms timeline regardless.
|
| 227 |
+
"""
|
| 228 |
+
freqs = chord_to_freqs(chord_name, octave=octave)
|
| 229 |
+
if direction == "down":
|
| 230 |
+
freqs = list(reversed(freqs))
|
| 231 |
+
|
| 232 |
+
stagger = arpeggio_ms / 1000.0
|
| 233 |
+
total_len = int((duration_s + 0.6) * self.sr)
|
| 234 |
+
out = np.zeros(total_len, dtype=np.float32)
|
| 235 |
+
|
| 236 |
+
for i, freq in enumerate(freqs):
|
| 237 |
+
# Voicing: root slightly louder, top voice slightly quieter
|
| 238 |
+
voice_vol = 1.0 if i == 0 else (0.78 if i < 3 else 0.6)
|
| 239 |
+
strike = self.strike(freq,
|
| 240 |
+
volume=volume * voice_vol,
|
| 241 |
+
brightness=brightness)
|
| 242 |
+
start = int(i * stagger * self.sr)
|
| 243 |
+
end = start + len(strike)
|
| 244 |
+
if end > len(out):
|
| 245 |
+
out = np.pad(out, (0, end - len(out)))
|
| 246 |
+
out[start:end] += strike
|
| 247 |
+
return out
|
| 248 |
+
|
| 249 |
+
def note(self, freq, duration_s, volume=1.0, brightness=0.5):
|
| 250 |
+
"""
|
| 251 |
+
Single mallet strike for melody use.
|
| 252 |
+
|
| 253 |
+
duration_s is ignored beyond a minimum β the strike has its own
|
| 254 |
+
natural decay envelope (~400ms). Don't try to sustain it.
|
| 255 |
+
"""
|
| 256 |
+
return self.strike(freq, volume=volume, brightness=brightness)
|
| 257 |
+
|
| 258 |
+
def sequence(self, events):
|
| 259 |
+
"""
|
| 260 |
+
Render mixed events: 'chord' for rhythm role (arpeggiated),
|
| 261 |
+
'note' for melody role (single strikes).
|
| 262 |
+
"""
|
| 263 |
+
if not events:
|
| 264 |
+
return np.zeros(int(self.sr), dtype=np.float32)
|
| 265 |
+
|
| 266 |
+
events = sorted(events, key=lambda e: e["time"])
|
| 267 |
+
# Extend tail to capture the reverb decay (~2s after the last event)
|
| 268 |
+
end_time = max(e["time"] + max(e["duration"], 0.6) for e in events) + 2.0
|
| 269 |
+
track = np.zeros(int(end_time * self.sr) + 1, dtype=np.float32)
|
| 270 |
+
|
| 271 |
+
for ev in events:
|
| 272 |
+
typ = ev.get("type")
|
| 273 |
+
if typ == "chord":
|
| 274 |
+
audio = self.chord(
|
| 275 |
+
ev["name"], ev["duration"],
|
| 276 |
+
octave=ev.get("octave", 4),
|
| 277 |
+
volume=ev.get("volume", 0.7),
|
| 278 |
+
brightness=ev.get("brightness", 0.5),
|
| 279 |
+
arpeggio_ms=ev.get("arpeggio_ms", 60),
|
| 280 |
+
direction=ev.get("direction", "up"),
|
| 281 |
+
)
|
| 282 |
+
elif typ == "note":
|
| 283 |
+
freq = note_to_hz(ev["name"])
|
| 284 |
+
audio = self.note(
|
| 285 |
+
freq, ev["duration"],
|
| 286 |
+
volume=ev.get("volume", 1.0),
|
| 287 |
+
brightness=ev.get("brightness", 0.5),
|
| 288 |
+
)
|
| 289 |
+
else:
|
| 290 |
+
continue
|
| 291 |
+
track = add_at(track, audio, ev["time"])
|
| 292 |
+
|
| 293 |
+
# Apply reverb at the mix level β xylo strikes are short and dry,
|
| 294 |
+
# so a more generous wet ratio than the harp makes them sit
|
| 295 |
+
# naturally in a "room" rather than feeling like a toy.
|
| 296 |
+
track = simple_reverb(track, self.sr,
|
| 297 |
+
room_size=0.45, damping=0.55, wet=0.32)
|
| 298 |
+
return track
|