SAMPA: A Whisper-Based System for Detecting Prosodic Boundaries in Portuguese Speech

Researchers have developed SAMPA, a system built on the Whisper model that automatically detects prosodic boundaries (the points that separate speech units) in Brazilian Portuguese speech. On the standard test the system scored F1=0.731, and in cross-dataset testing on MuPe-Diversidades it reached F1=0.796.

How SAMPA works

The system is based on Whisper large-v3, OpenAI's multilingual speech-to-text model. The authors fine-tuned it on Portuguese recordings from the NURC-SP dataset (an archive of spoken Brazilian Portuguese) and taught the model to insert special markers at prosodic boundaries. The system was trained on recordings with different speakers, speech styles and recording conditions, which helped it handle new examples as well.

Key details of the approach:

• Base model: Whisper large-v3 from OpenAI
• Fine-tuning: the NURC-SP dataset with manually annotated recordings
• Accuracy on the main test: F1=0.731
• Accuracy on the independent MuPe-Diversidades dataset: F1=0.796
• The model uses morphosyntactic, semantic and prosodic cues

Why this matters for Portuguese

For English, prosody processing is well developed thanks to an abundance of annotated data and years of research. Portuguese, by contrast, has historically relied mainly on rules and traditional machine learning methods, which are less accurate and less flexible than neural approaches.

SAMPA shows that a large pretrained model can be adapted effectively to another language using a relatively small set of annotated recordings. That opens the way to similar solutions for languages beyond English: Russian, Spanish, Chinese and others could get comparable systems by adapting Whisper.

What this means

Correct detection of prosodic boundaries matters for practical Portuguese speech processing: synthesized speech sounds more convincing when pauses and intonation are placed correctly; automatic text annotation becomes possible without manual editing; and analysis of spoken documents and speech archives gets faster.

The study shows that transfer learning works well. Instead of training a model from scratch, the authors adapted an existing Whisper model, which is considerably faster and requires less data. This could become a template for developing speech processing in other languages and regions.