text to speech online natural voice

Turn any text into audio instantly

Listening was Born Before Reading

Listening predates reading in human communication history and remains a natural and intuitive way to absorb information.

Natural-Sounding Voices

The AI Text-to-Speech (TTS) technology powers our free reader with high-quality voices so you can enjoy the timeless advantages of listening.

Do More with Your Time

With our app, you can get through documents, articles, PDFs, and emails effortlessly, freeing your hands and eyes.

Listen to Anything, Anywhere

You can listen to any text on desktop or mobile devices. Use our app now and unlock the potential of listening as the ultimate reading companion.

Select your Speechise Plan

Start free, upgrade when you need

Guaranteed safe & secure checkout

Frequently Asked Questions

If you don't find your answer here, please contact us .

How does Speechise work?

You just open speechise.com in a browser, paste your text and click Play. The system converts the text to audio and the sound starts almost immediately. The chunk of text that is currently playing is highlighted in your browser. You can pause or continue listening.

Is Speechise Free?

Yes, you can use Speechise for free with the limit of 2,000 characters per single request.

All our subscription options are listed on the pricing page for your convenience. You can upgrade to a paid version if you like Speechise and want to use it fully. Your feedback is appreciated in any case.

What Languages are Supported?

You can use 50+ languages and variants in 380+ voices.

Some of the supported languages are English, Spanish, Portuguese, French, German, Turkish, Italian, Dutch, Norwegian, Polish, Swedish, Bulgarian, Czech, Hungarian, Finnish, Greek, Ukrainian, Russian, Arabic, Korean, Hindi, Japanese, Chinese, Thai.

What is text-to-speech (TTS)?

Artificial intelligence (AI) software reads text or a document aloud for you. The text can be a fragment or a PDF, eBook, email or a webpage. The language can be English, Spanish, Portuguese or other. The voice sounds human and you can select accent/character.

Do I need to install anything?

No installation required. Speechise simply works in your browser on a desktop computer or a mobile device.

Realistic Text-to-Speech AI converter

Create realistic Voiceovers online! Insert any text to generate speech and download audio mp3 or wav for any purpose. Speak a text with AI-powered voices.You can convert text to voice for free for reference only. For all features, purchase the paid plans

How to convert text into speech?

Just type some text or import your written content
Press "generate" button
Download MP3 / WAV

Full list of benefits of neural voices

Downloadable tts.

You can download converted audio files in MP3, WAV, OGG for free.

If your Limit balance is sufficient, you can use a single query to convert a text of up to 2,000,000 characters into speech.

Commercial Use

You can use the generated audio for commercial purposes. Examples: YouTube, Tik Tok, Instagram, Facebook, Twitch, Twitter, Podcasts, Video Ads, Advertising, E-book, Presentation and other.

Multi-voice editor

Dialogue with AI Voices. You can use several voices at once in one text.

Custom voice settings

Change Speed, Pitch, Stress, Pronunciation, Intonation , Emphasis , Pauses and more. SSML support .

You spend little on re-dubbing the text. Limits are spent only for changed sentences in the text.

Over 1000 Natural Sounding Voices

Crystal-clear voice over like a Human. Males, females, children's, elderly voices.

Powerful support

We will help you with any questions about text-to-speech. Ask any questions, even the simplest ones. We are happy to help.

Compatible with editing programs

Works with any video creation software: Adobe Premier, After effects, Audition, DaVinci Resolve, Apple Motion, Camtasia, iMovie, Audacity, etc.

You can share the link to the audio. Send audio links to your friends and colleagues.

Cloud save your history

All your files and texts are automatically saved in your profile on our cloud server. Add tracks to your favorites in one click.

Use our text to voice converter to make videos with natural sounding speech!

Say goodbye to expensive traditional audio creation

Cheap price. Create a professional voiceover in real time for pennies. it is 100 times cheaper than a live speaker.

Traditional audio creation

Expensive live speakers, high prices
A long search for freelancers and studios
Editing requires complex tools and knowledge
The announcer in the studio voices a long time. It takes time to give him a task and accept it..

Affordable tts generation starting at $0.08 per 1000 characters
Website accessible in your browser right now
Intuitive interface, suitable for beginners
SpeechGen generates text from speech very quickly. A few clicks and the audio is ready.

Create AI-generated realistic voice-overs.

Ways to use. Cases.

See how other people are already using our realistic speech synthesis. There are hundreds of variations in applications. Here are some of them.

Voice over for videos. Commercial, YouTube, Tik Tok, Instagram, Facebook, and other social media. Add voice to any videos!
E-learning material. Ex: learning foreign languages, listening to lectures, instructional videos.
Advertising. Increase installations and sales! Create AI-generated realistic voice-overs for video ads, promo, and creatives.
Public places. Synthesizing speech from text is needed for airports, bus stations, parks, supermarkets, stadiums, and other public areas.
Podcasts. Turn text into podcasts to increase content reach. Publish your audio files on iTunes, Spotify, and other podcast services.
Mobile apps and desktop software. The synthesized ai voices make the app friendly.
Essay reader. Read your essay out loud to write a better paper.
Presentations. Use text-to-speech for impressive PowerPoint presentations and slideshow.
Reading documents. Save your time reading documents aloud with a speech synthesizer.
Book reader. Use our text-to-speech web app for ebook reading aloud with natural voices.
Welcome audio messages for websites. It is a perfect way to re-engage with your audience.
Online article reader. Internet users translate texts of interesting articles into audio and listen to them to save time.
Voicemail greeting generator. Record voice-over for telephone systems phone greetings.
Online narrator to read fairy tales aloud to children.
For fun. Use the robot voiceover to create memes, creativity, and gags.

Maximize your content’s potential with an audio-version. Increase audience engagement and drive business growth.

Who uses Text to Speech?

SpeechGen.io is a service with artificial intelligence used by about 1,000 people daily for different purposes. Here are examples.

Video makers create voiceovers for videos. They generate audio content without expensive studio production.

Newsmakers convert text to speech with computerized voices for news reporting and sports announcing.

Students and busy professionals to quickly explore content

Foreigners. Second-language students who want to improve their pronunciation or listen to the text comprehension

Software developers add synthesized speech to programs to improve the user experience.

Marketers. Easy-to-produce audio content for any startups

IVR voice recordings. Generate prompts for interactive voice response systems.

Educators. Foreign language teachers generate voice from the text for audio examples.

Booklovers use Speechgen as an out loud book reader. The TTS voiceover is downloadable. Listen on any device.

HR departments and e-learning professionals can make learning modules and employee training with ai text to speech online software.

Webmasters convert articles to audio with lifelike robotic voices. TTS audio increases the time on the webpage and the depth of views.

Animators use ai voices for dialogue and character speech.

Text to Speech enables brands, companies, and organizations to deliver enhanced end-user experience, while minimizing costs.

Frequently Asked Questions

Convert any text to super realistic human voices. See all tariff plans .

Enhance Your Content Accessibility

Boost your experience with our additional features. Easily convert PDFs, DOCx files, and video subtitles into natural-sounding audio.

📄🔊 PDF to Audio

Transform your PDF documents into audible content for easier consumption and enhanced accessibility.

📝🎧 DOCx to mp3

Easily convert Word documents into speech for listening on the go or for those who prefer audio format

📺💬 Subtitles to Speech

Make your video content more accessible by converting subtitles into natural-sounding audio.

Supported languages

Amharic (Ethiopia)
Arabic (Algeria)
Arabic (Egypt)
Arabic (Saudi Arabia)
Bengali (India)
Catalan (Spain)
English (Australia)
English (Canada)
English (GB)
English (Hong Kong)
English (India)
English (Philippines)
German (Austria)
Hindi India
Spanish (Argentina)
Spanish (Mexico)
Spanish (United States)
Tamil (India)
All languages: +76

We use cookies to ensure you get the best experience on our website. Learn more: Privacy Policy

Online Reader

Turn the web into Speech

Instant Text-to-Speech (TTS) using realistic voices

3 Steps to Getting Started

Send your article or text.

Share the URL of the article or upload the text content to Woord. Also you can use our Text-to-Speech API

Select the type of voice you like

There is a wide selection of custom voices available for you to pick from. The voices differ by language, gender, and accent (for some languages)

Download or Play your Audio

Click on 'Submit' and our platform will create the audio that sounds like a person talking

A few of Woord's Best Features

+100 voices from 34 different languages. Regional variations are also available for select languages, such as Canadian French, Brazilian Portuguese, and several other languages.

Unlimited Audios

Have the freedom to convert any text content you want. Blog posts, news, books, research papers or any other text content.

Create and redistribute

MP3 Download and Audio hosting with HTML embed audio player. This means that you can use audio files in YouTube videos, e-Learning modules, or any other commercial purposes.

Smart Voice Technology

Using AI technology, our synthesized voices are of the highest quality, emulating human-like natural sounding speech.

The voices that will bring your projects to life

We support different Varieties of the English Language (US, UK, Australia, India, and Welsh), Spanish, Spanish Mexican, Portuguese, Brazilian Portuguese, French, Canadian French, German, Russian, Catalan, Bengali, Danish, Welsh, Turkish, Hindi, Italian, Japanese, Chinese, Cantonese, Vietnamese, Arabic, Dutch, Norwegian, Korean, Polish, Swedish, Bulgarian, Czech, Filipino, Hungarian, Finnish, Greek, Gujarati, Icelandic, Indonesian, Latvian, Malay, Mandarin Chinese, Romanian, Serbian, Slovak, South African, Thai, Ukrainian, Gujarati, Punjabi, Tamil, Telugu.

Listen to our Voices

Testimonials

Over 100,000 people ♥ woord.

Anthony Larson

Content editor - bbc.

Huge thanks to Woord! Makes my life easier

Jena Kimbol

Entrepeneur.

Everyone doing a podcast should be using Woord.

Mark Fisher

Ceo & founder - nusca.

Thanks Woord for being so easy to use. Its awesome!

Gabriela Rodríguez

Content manager - bbc.

Thanks, Woord, for being user-friendly and brilliant! Converting text to audio has never been this easy. Truly awesome!

Alex Turner

Software developer.

I love how Woord effortlessly converts my documents into audio. It's user-friendly and gets the job done seamlessly.

Claire Harper

Sound engineer.

Its exceptional user-friendliness and brilliance! Transforming text into audio has never been as effortless. Truly impressive!

Richard Santos

Chief technology officer.

Enormous appreciation. Simplifies my daily routine, making life much more convenient.

Maria Fernandez

User experience specialist (ux).

Big thanks for its user-friendly design. It's truly fantastic!

Javier Gonzalez

Software architect.

Woord has simplified podcasting for me. It's incredibly user-friendly and packed with awesome features.

Caroline Rodriguez

Systems analyst.

It is a great TTS tool for converting my documents into audio. It helped me a lot!

Martin Vargas

Product manager.

I was amazed with this text to speech option, one of the best I have ever used.

Valerie Mendez

Development coordinator.

Easy and great! A ready to go tool with a lot of voices. Loved it from the first time.

For All Plans

$9.99/month.

10 audios per month
Audio credits never expires
10,000 characters per audio
For Single User Only
Male, Female voices
Premium voices
+100 voices
34 languages and variations
OCR to read from images & scanned PDFs
Supports pdf, txt, doc(x), pages, odt, ppt(x), ods, non-DRM epub, jpeg, png.
SSML editor
Chrome extension
MP3 Download
High quality audio
Audio Joiner
For Commercial use: Youtube, broadcasts, TV, IVR voiceover and other businesses
You 100% own intellectual property for all files
Private Audio Library
Cancel Anytime

No long term commitments. One click upgrade/downgrade or cancellation. No questions asked.

Free 7-Day Trial

50 audios per month

Get Started

125 audios per month
300 audios per month

Also, we offer our custom Enterprise Pricing for unlimited API calls, dedicated technical support, and more - Request Quote 7-Day-Free Trial: You can only access this benefit with Credit Card. No Paypal allowed.

Why convert Text to Audio?

Audio offers a richer experience, subconsciously engaging the listener with a continuous stream of audio.

Accumulated Audios

In woord, accumulated audios refer to the feature that allows users with a subscription to accumulate unused audios from one month to the next, as long as their subscription remains active. for example, if a user has a starter subscription which offers 10 audios per month, but only uses 5 in the first month, the remaining 5 audios will be carried over to the next month, in addition to the 10 new audios offered in that month. this means the user will have a total of 15 audios to use in the second month. this feature is designed to provide greater flexibility and convenience to users, allowing them to make the most out of their subscription by accumulating unused audios for future use., any questions we're happy to help.

Find your answers here. if you don’t find it here, please contact us.

What are the most common use cases for this service?

With Woord, you can bring your applications to life, by adding life-like speech capabilities. For example, in E-learning and education, you can build applications leveraging Woord’s Text-to-Speech (TTS) capability to help people with reading disabilities. Woord can be used to help the blind and visually impaired consume digital content (eBooks, news etc). Woord can be used in announcement systems in public transportation and industrial control systems for notifications and emergency announcements. There are a wide range of devices such as set-top boxes, smart watches, tablets, smartphones and IoT devices, which can leverage Woord for providing audio output. Woord can be used in telephony solutions to voice Interactive Voice Response systems. Applications such as quiz games, animations, avatars or narration generation are common use-cases for cloud-based TTS solutions like Woord.

Which languages are supported?

Are there any limitations to the amount i can convert.

No, paid subscriptions don't have limit of the number of characters to convert.

Can I choose a different gender to a specific post?

Yes you can. We have male, female voices.

Can I read web pages, documents or scans aloud?

Yes, you can listen to text in your documents, messages, presentations, scans, web pages or notes using Woord.

Does Woord have characters limits per audio?

Yes, you have up to 10000 characters per audio for any plan. If you need more, please contact us.

Can I really cancel anytime?

Yes, absolutely. If you want to cancel your plan, simply go to your account and cancel on the Billing page. Remember that you to cancel your current subscription you can't create more than 2 audios in the month where you are canceling. Also, you will lose the features that you had when you purchased the plan.

What currencies and payment options are available?

Prices are listed in USD. We accept all major debit and credit cards. Our payment system uses the latest security technology and is powered by Stripe, one of the world’s most reliable payment companies. If you have any trouble with paying by card, you can pay using PayPal.

What is your refund policy?

You may request a refund for your current month if you request it within 2 hours of the transaction and only applies to the first payment we receive. We reserve the right to decline that request should you use our software within this time.

Are there discounts for any products?

We don’t have any discounts currently.

Do you offer personalized plans?

Yes! But it has to be for a bigger bundle than what’s available.

What if I’m having issues getting my email verified?

You can message us through our chat popup, or email us using our contact info

When does the billing cycle start?

Your billing cycle starts the day you purchase one of our Plans and ends the same day of the next month or next year (if you are paying annually). Instead, the limit of audios that you can make is renewed on the first day of each month. In other words, if you buy one of our plans on April 10th, your Audios credit will be activated that same day. The next payment will be made automatically on May 10th, however, on May 1st the Audio counter will be reset and it will start again.

How can I upgrade or downgrade my plan?

You can manage all of this on your own from your dashboard!

What happens if I forget to downgrade my plan on time?

Unfortunately, we don’t give refunds on renewals, you can check our terms and conditions Here

How can I change my billing frequency from monthly to yearly or from yearly to monthly?

You will be required to downgrade your account back to the Free Plan. Step 1: Navigate to the Subscription page, click "Downgrade" in the Free Plan section and confirm your downgrade. Downgrades are not effective immediately, your premium subscription will remain active until the end of the current billing period. Step 2: Once your billing period ends and your account downgrade has become effective, navigate back to the Subscription Page and click "Upgrade" in your preferred subscription plan's section. You will now be asked to choose a new billing frequency.

Is my payment info deleted after I downgrade?

Yes! It’s deleted automatically. The information is handled by Stripe or Paypal, we don’t store your credit or debit card data.

Where can I see my invoices?

If you’re paying with a Credit /Debit card, you can find them by going onto link/billing → billing portal-> invoices. If you’re using paypal you have to download the invoice from http://paypal.com/

How can I use the SSML editor?

Here are a few examples: *We have the Break button, we'll use this one by first clicking where we want the break to be, and then clicking the break button. A dropdown menu will open, where you can choose the length of the pause. It’ll look like this: We are speaking, and now we'll have a break here. *Next to that one, we have the emphasis button, to use this one, simply write your text, highlight the text that we want to emphasize, and click the emphasis button. It’ll look like this: We are going to emphasize here . If you’re still unsure, here’s a blog post explaining how to use our SSML Editor.

I am interested in subscribing to a basic or pro plan but prefer to pay annually, is this possible?

Yes, you can pay for a pro plan annually. The basic plan doesn't have an option to pay annually, it’s monthly.

How can I delete my account?

First, you have to downgrade to a free plan to make sure we won’t charge you again. After that, you can delete your account from your dashboard.

When I did my initial test sample, the output was spoken a bit too fast. Do you have the capability to slow down the audio output speed ?

Yes, you have 2 options 1) Modify the speed of the audio before creating (Advanced options -> Choose Voice Speed, 1 is the default). Speaking rate/speed, in the range [0.25, 4.0]. 1.0 is the normal native speed supported by the specific voice. 2.0 is twice as fast, and 0.5 is half as fast. 2) you can use our SSML editor https://www.getwoord.com/ssml-editor to add pauses or modify the speed using SSML tags. SSML API support is only available for enterprise customers (we could enable for you if necessary).

Text to Speech Voice Over with Realistic AI Voices

Murf offers a selection of 100% natural sounding AI voices in 20 languages to make professional voice over for your videos and presentations. Start your free trial.

Quality Guaranteed, No Robotic Voices

Our voices are all human sounding and quality checked across dozens of parameters. Gone are the days of robotic text to speech, most people can’t even tell between our advanced AI voices and recorded human voices.

Text to Speech Voices in 20+ Languages

Murf offers a selection of voices across 20+ languages. Most languages have voices available for testing quality in the free plan. Some languages also support multiple accents like English, Spanish and Portuguese.

A Simple Text to Voice Converter

High-Quality Voices for Every Use Case

Not Just a Text to Speech Tool

Emphasize specific words

Want to make your voiceover sound interesting? Use Murf’s ‘Emphasis’ feature to put that extra force on syllables, words, or phrases that add life to your voiceover.

Take control of your narration with pitch

Use Murf’s ‘Pitch’ functionality to draw the listeners' attention to words or phrases expressing emotions. Customize the voice as you like to make it work for yourself.

Elevate your story with pauses

Add pauses of varying lengths to your narration using Murf’s ‘Pause’ feature to give the listener's attention powers a rest and prepare them to receive your message.

Perfect Word Pronunciation

Articulate words accurately and enhance clarity in speech by customizing pronunciation. Use alternative spellings or IPAs to achieve the right pronunciation.

Fine Tune Narration Speed

Effortlessly increase or decrease the pace of the voiceover to ensure it aligns with the rhythm and flow of the message.

Expressive Voice Style Palette

Infuse your narration with the exact emotion your content needs using Murf’s dynamic voice styles. Choose from versatile options like excited, sad, angry, calm, terrified, friendly, and more.

Text to Voice Made Easy

Reliable and secure. your data, our promise..

Why Use Murf Text to Speech?

Murf's text to audio software changes the way you create and edit voiceovers with lifelike, flawless AI voices. What used to take hours, weeks, or even months now only takes minutes. You can also include images, videos, and presentations to your voiceover and sync them together without the need for a third-party tool. Here are a few reasons why you should use Murf's text to speech.

Save time and hundreds of dollars in recording expensive voice overs.

Editing voice over is as simple as editing text. Just cut, copy paste and render.

Create a consistent brand voice across all your customer touchpoints.

Connect with global customers effectively with our multiple language AI voices.

Build scalable voice applications with Murf’s API.

Voice over in 20+ languages.

@MURFAISTUDIO

Hear from Our Customers

Murf allows me to create TTS voiceovers in a matter of minutes. Previously, I had a tedious process of sending scripts out to agencies and waited days to get voiceovers back. With Murf, I can make changes whenever I like, diversify my speaker portfolio by picking new voices instantly, and even ramp up my course localization.

Murf it's an amazing text-to-speech AI voice generator, easy to work with, flexible and reliable. Its voices, non-pro and pro (either English, Spanish, and French), are both so real that many clients of mine have been surprised to know that they were not from professional voice-over actors.

I recently tried murf.ai and I have to say I am thoroughly impressed. The quality of the generated voice is exceptional and very realistic, which is important for my business needs. The platform is user-friendly and easy to navigate, and the range of voices available is impressive.

This website is so easy and clear that you will find yourself mastering all the tools in no time. The fact that regenerating the voice with different voices, punctuations, and tones does not deduct from your allowed minutes is so fair and reasonable. And the price is affordable too. Highly recommended

This is the most human-like voice I was able to find. It's very lively,and I found it suitable for many types of videos including marketing and e-learning, it kept my audience engaged!

I just started to create a video channel about historical figures, and Murf.ai really brings them to life. I found my top voice for my scripts, and the easy integration of video elements makes it a breeze to create informative videos. I also like the easy changes one can make to the tone of voice from within the editor.

Frequently Asked Questions

Text to speech: what is it and how does it works.

In essence, text to speech is the generation of synthesized speech from text. It was primarily designed as an assistive technology to help individuals with hearing impairments, visual and learning disabilities, and aged citizens to understand and consume content in a better manner. Today, the applications of TTS systems have grown manifold, and range from content creation to voiceover generation to customer service, and more. With a touch of a button, TTS can take words on a computer or other digital device and convert them into audio files. Today, the technology is used to create narratives for explainer videos or product demos , turn a book into an audio book, generate voiceovers for elearning materials, training videos, ads and commercials, YouTube videos, or podcasts, among other things.

How does TTS work?

Text to speech software leverages AI and deep learning algorithms to process the written input and sythesize a spoken output. The written text is first broken down into individual words and phrases by the TTS software’s text analysis component and then various rules and algorithms are applied to determine the appropriate pronunciation, inflection, and emphasis for each word. The speech synthesis component of the software then takes this information along with pre-recorded sound samples of individual phonemes and uses it to generate the spoken words and sentences, which is then spoken out loud using a synthesized voice generated by a computer or other device.

Top Five Use Cases of Text to Speech Software

From increasing brand visibility and customer traction to improving customer service and boosting customer engagement to helping people with visual impairments, reading difficulties, and learning disabilities, text to speech is proving to be a game-changing technology across industries.

Considering the myriad of benefits offered by TTS technology and how simple they make information retention, businesses are integrating text to speech into their workflow in one form or another. Here is a glimpse of all the ways text to speech is currently being utilized:

TTS in Assistive Technology

For quite some time now, text to speech software has been used as an accessibility tool for individuals with a variety of special needs linked to Dyslexia, visual impairments, or other disabilities that make it difficult to read traditional text. Using TTS platforms, people facing such problems can convert text to speech and learn by listening on the go. Text to speech solutions also improves literacy and comprehension skills. When used in language education, they can make learning more engaging. For example, it's much easier and faster to apprehend a foreign language when listening to the live translation of written words with correct intonation and pronunciation than when reading.

TTS in Translations

Given the fact that modern text to speech solutions come with multilingual support, brands can reach local customers by converting their content from text to audio in the local language. This will help target and connect with native-speaking customers or audiences in remote areas.

Furthermore, text to speech solutions can also be used to translate content from one language to another. This is especially beneficial for users who come across a piece of content in a language they don't understand and can have it read aloud in their native language or a language they are adept at for better understanding.

TTS in Customer Service

With advancements in speech synthesis, it has become easier to create text and convert it to pre-recorded voices for interactive voice response calls. Today's TTS technology comes with human-like AI voices that can make natural human conversations on IVR calls. This helps contact centers provide personalized customer interactions without requiring assistance from live agents.

TTS serves as both an inbound and outbound customer service tool. For example, when used in tandem with an IVR system, TTS solutions can provide personalized information to callers, such as greeting a customer by name, providing account information, confirming details about the order, payment, or appointment, and more. Furthermore, by tapping into the extensive range of languages, accents, and a wide variety female and male voices offered by TTS software, companies can provide an experience that matches their customer's profiles or help promote an image for their brand.

TTS in Automotive Industry

Text to speech solutions help make connected and autonomous cars safer and sound truly unique, begetting an on-road revolution. They can be used in in-car conversational systems for navigational prompts and map data, infotainment systems to read aloud information about the car, such as fuel level or tire pressure, and swap music and voice assistants to place phone calls, read messages, and more.

TTS in Healthcare

In the healthcare industry, text to speech solutions can be used to read aloud patient information, instructions for taking medication, and provide information to doctors and other medical professionals about upcoming appointments, scheduling calls, and more.

Why text to speech matters for businesses?

It's an exciting time to stake your claim in the realm of speech synthesis. There are a number of key industries where the text to speech technology has already succeeded in making a dent. Here are a few different ways in which businesses can harness the power of text to speech and save money and time:

Enhances customer experience

Any business can leverage TTS to alleviate human agent workload and offer customized conversational customer support. By integrating these solutions with IVR systems, companies can automate customer interactions, facilitate smart and personalized self-service by providing voice responses in the customer's language and remove communication barriers. Furthermore, organizations can also use TTS to make AI-enabled routine calls to inform customers about promotional offers, payment reminders, and much more. That said, by using text to speech in voice-activated chatbots, businesses can provide customers, especially the visually impaired, with a more immersive experience, thereby enriching the customer experience.

Global market penetration

Text to speech solutions offer synthetic voices in multiple languages enabling businesses to create content in several different languages and reach customers across different countries worldwide. Organizations can build trust with customers by creating voiceovers for ads, commercials, product demos, explainer videos, and PowerPoint presentations, among other content pieces in regional dialects and native languages.

Increases Web Presence

That said, with the help of TTS solutions, businesses can provide an audio version of their content in addition to a written version, enabling more accessibility to a broader audience, who can choose whether to read or listen to it based on their preferences. This increases the brand's web presence. Moreover, using text to speech, brands can create a familiar, recognizable and unique voice across all their voice channels, making it easy for customers to identify the brand the second they hear it. This way, the brand shows up everywhere and improves its web presence.

Who else can benefit from text to speech?

Today’s online text to speech systems can generate speech that is almost indistinguishable from a human voice, making them a valuable tool for a wide range of applications, from improving accessibility for people with disabilities to providing convenient and efficient ways to communicate information.

Here is a list of everybody that can benefit immensely from using best text to speech softwares for their content and voiceover needs:

Many educators struggle to enhance the value of their curriculum while simplifying their workloads. This is where realistic text to speech technology plays a key role. Firstly, it improves accessibility for students with disabilities. Screen readers and other tools which are speech enabled can make learning an equal opportunity and enjoyable experience for those with learning and physical disabilities. Secondly, it helps teach comprehension in an effective manner. Text to speech software offers an easy way for students to listen to how words are spoken in their natural structure and following the same is easier through audio playback.

TTS software also enhances engagement and makes learning interesting for students. For example, using natural sounding text to speech voices, teachers can create engaging presentations and elearning modules that capture student’s attention.

In marketing specifically, text to speech technology can help improve data collection, facilitate comprehensive customer profiling, and better data analysis. Online text to speech tools offer an easy way for businesses to reach a broader audience and create customized user experiences.

For instance, marketing teams can create and deliver videos to prospective clients to establish a connection and brief them on queries and complicated products or services in the language and accent the customer is comfortable with. Furthermore, AI voices enable marketing teams to create crisp, high quality professional-sounding voiceovers in a few simple steps without hiring voice actors or requiring any professional recording studios.

Text to speech generators offer authors numerous advantages. One, it serves as an editing aid and helps storytellers proof read their novels and manuscripts to identify grammatical errors and other mistakes in their drafts before publishing. Listening to their stories being read aloud also allows authors to gauge the response to their work on other people. Authors can also use realistic voice generators to convert their books into audiobooks and podcasts and broaden the reach of their work.

From interviews about true crime to politics and science, there are all sorts of popular podcast formats today. And, regardless of how good your podcast topic is, it won’t matter if the host doesn’t have a good voice. That said, not everyone can have that best podcast voice like an old-school radio anchor or a news presenter. This is where text to speech platforms come in. You don’t have to record scripted intros, prologues, or epilogues, an AI narrator can do it for you. Through text to speech software, you can automatically create the narrative and voiceover for your podcast in the language and tone you want in a matter of minutes by simply uploading the script to the platform.

Creating good voice overs for your animated explainer videos or product demos or games typically meant investing a lot of money on recording equipment and hiring professional voice actors. Not anymore. With AI text to speech platforms, you can add natural sounding voices to your animated video to make them more engaging and captivating. In fact, with text to speech software, you can give each character in your animated video or game, a unique voice.

Customer Support Executives

Integrating realistic text to voice software with an IVR system enables customer service agents to concentrate more on complex customers rather than common queries. TTS-enabled IVR systems are capable of gathering information and providing responses to customers as necessary in a way that sounds just like an actual customer service agent.

Furthermore, TTS systems also eliminate the need for IVR businesses to schedule voiceover retakes months in advance. With TTS systems, businesses can render a new voiceover in minutes creating thousands of iterations within a few clicks.

Text to speech is a game-changer for students of all ages and educational levels. By converting written text into spoken words, students can enhance their learning experience and comprehension. Text to speech technology can read content out aloud, making it easier for students to absorb information while multitasking. It is particularly useful for students with dyslexia, ADHD, or other learning disabilities as it provides them with an alternative way to consume educational content. Furthermore, the tool can also be used to add narrations to presentations, explainer videos, how-to videos, and more.

Be it corporate trainers, fitness trainers, or lifestyle instructors, text to speech can be used to create engaging and accessible learning materials. For example, fitness trainers can convert written content into audio-based workout routines and personalized exercise plans. This helps to increase engagement levels and knowledge retention among the audience.

Similarly, corporate trainers can also use TTS to create presentations on employee policies and other organizational practices. It makes the coursework highly engaging and improves employee performance at many levels. Additionally, using audio course materials is a great way to respect the staff with disabilities and give everyone equal access to training.

Content Creators

Content creators, including social media users, bloggers, writers, influencers, and authors, can leverage text to speech to enhance their productivity and reach a broader audience.

This technology enables content creators to convert their written articles, scripts, blog posts, or eBooks into high-quality audio files quickly in multiple languages instead of manually recording the voiceover.

Consequently, it opens up new avenues for content consumption. This allows readers to listen to the content while performing other tasks or when reading isn’t feasible, such as during commutes or workouts.

Video Producers

Video creators can easily add voiceovers or narration to their videos, eliminating the need for hiring voice actors or spending hours recording audio. This not only saves time and resources but also ensures consistent and professional-sounding voiceovers.

Murf: The Ultimate Text to Speech Software

If you are looking for a text to speech generator that can create stunning voiceovers for your tutorials, presentations, or videos, Murf is the one to go for.

Murf can generate human-like, realistic, and natural-sounding voices. Its pièce de résistance is that Murf can do it in over 120+ unique voices in 20+ languages.

This text aloud reader also allows you to tweak the pitch of the voice, add pauses or emphasis, and alter the speed of the output to get the output just the way you want it.

And the best part? Murf is extremely easy to use. Just type or paste in your script, choose your preferred voice in the language you want, and hit play. Murf will do the rest.

Create Engaging Content with Murf's AI Voices

Murf text to audio converter can be used in a number of scenarios to elevate the quality of your overall content. Let's look at a few use cases where Murf can help and why it’s the best text to speech reader out there:

E-learning Videos

Murf’s free text to speech reader can help you create e-learning videos in multiple languages that will make your content accessible to a global audience. You can also increase the engagement of your e-learning video by adding emotions and expressions to your content.

Presentations

Murf’s AI voices can add a touch of professionalism to your presentations to help drive home those key points. You can use Murf to narrate your slides, explain your concepts, or tell the story of your brand in the exact tone and style you envisioned.

You can also use this free text to speech reader to make your audiobooks sound as if they its been narrated by an actual person.

With Murf, you can also mix and match different voices for the various characters in the audiobook to take your storytelling up a few notches.

Sales and Marketing Videos

Murf can also enhance your sales and marketing videos with persuasive and professional voiceovers. You can use these videos to showcase your products, services, or offers and tailor them in multiple languages to advertise to a potentially global audience.

Product Demos

Finally, Murf can help you create informative and engaging product demo videos that showcase your product’s features and benefits in the best possible light.

Key Features of Murf Text to Speech

Apart from enabling users to enhance the quality of their voiceover content with compelling, nuanced, and natural sounding text to speech voices, Murf offers an intuitive voice user interface and the ability to customize and control the voiceover output with features like pitch, speed, emphasis, pause, pronunciation and more.

More than Just a Text to Speech Software

Tired of hearing monotonous, robotic-sounding voiceovers? Not anymore. With Murf, enhance the quality of your content with compelling, nuanced, and natural sounding text to speech that replicate the subtleties of human voice. Fine-tune your voiceover narration and add more character to an AI voice with features such as Emphasis, Pronunciation, Speed, and more! From inviting and conversational to excited and loud to empathetic and authoritative, we have AI voices that span different intonations and emotions. Murf AI text to speech (TTS) supports Arabic, Chinese, Danish, Dutch, English, Finnish, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Norwegian, Portuguese, Romanian, Russian, Spanish, Tamil, and Turkish. Some of these languages also support multiple accents. For example, our English language AI voices support British, Australian, American, and Indian accents. Our Spanish AI voices support Mexican and Spain accents. The TTS online software also offers users the ability to add background audio or music to their content. Murf studio, in fact, comes with a curated selection of royalty-free music in their gallery that the user can choose from to add some music to their video. You can also upload your own audio files or even import from external sources like YouTube, Vimeo, and other video websites. Murf's text to sound has a voice changer feature that lets you upload your existing recording and revamp it with professional AI voice in a single click. You can change your voice to an AI voice in three simple steps: transcribe the audio, choose an AI voice, and regenerate the audio in a new voice. It's as easy as pie.

Additionally, the tool also supports an AI translation feature that enables you to convert your scripts and voiceovers into multiple languages in minutes. With Murf AI Translate, you can convert your projects into 20 different global and regional languages, making them accessible to a broader audience and expanding your reach.

Summing It Up

Murf is a powerful text to speech reader that can help you create engaging and professional voiceovers for your videos, presentations , and so much more.

To put it in short, with Murf, you can:

Save a ton of money that would have otherwise been spent on voice actors and renting out studio spaces.
Widen your reach to a global audience with its support for over 120+ unique voices in over 20+ languages.
Make your content accessible to anyone with visual or specific cognitive disabilities.

So, what are you waiting for? Sign up for a free trial of Murf today!

Murf supports Text to speech in

Free Text To Speech Reader

1 Select voice John Kelly
2 Select talking speed 0.5 0.6 0.7 0.8 0.9 Normal Speed 1.1 1.2 1.3 1.4 1.5 2.0 3.0
3 Select pitch +1.8 +1.7 +1.6 +1.5 +1.4 +1.3 +1.2 +1.1 1.0 -0.9 -0.8 -0.7 -0.6
Vocalize Vocalizing
Download Vocalizing

Examples of text-to-speech translation

About VoxWorker.com

What is voxworker, multiple languages, variety of voices, file formats, easy to use, usage options.

Export Audio

Free Text To Speech Reader

Instantly reads out loud text & pdf with natural sounding voices online - works out of the box. drop the text and click play..

Drag text or pdf files to the text-box, or directly type/paste in text. Select language and click Play. Remembers text and caret position between sessions. Works on Chrome and Safari, desktop and mobile. Enjoy listening :)

Best Text to Speech Online

Online speech synthesizer, single click to read out loud any text
Listen instead of reading
Multiple languages and voices
Reads PDF files too

TTSReader-X

Chrome extension
Listen to ANY website without leaving the page
Adds a 'play' functionality to Chrome
Clean page for readability and / or print

Try it Now for FREE

TTSReader / Android

Podcast any written content
Save data - works offline too

Get it on the Play store

Fun, Online, Free. Listen to great content

Drag, drop & play (or directly copy text & play). That’s it. No downloads. No logins. No passwords. No fuss. Simply fun to use and listen to great content. Great for listening in the background. Great for proof-reading. Great for kids and more. Learn more, including a YouTube we made, here .

Multilingual, Natural Voices

We facilitate high-quality natural-sounding voices from different sources. There are male & female voices, in different accents and different languages. Choose the voice you like, insert text, click play to generate the synthesized speech and enjoy listening.

Exit, Come Back & Play from Where You Stopped

TTSReader remembers the article and last position when paused, even if you close the browser. This way, you can come back to listening right where you previously left. Works on Chrome & Safari on mobile too. Ideal for listening to articles.

Better than Podcasts

In many aspects, synthesized speech has advantages over recorded podcasts. Here are some: First of all - you have unlimited - free - content. That includes high-quality articles and books, that are not available on podcasts. Second - it’s free. Third - it uses almost no data - so it’s available offline too, and you save money. If you like listening on the go, as while driving or walking - get our free Android Text Reader App .

Read PDF Files, Texts & Websites

TTSReader extracts the text from pdf files, and reads it out loud. Also useful for simply copying text from pdf to anywhere. In addition, it highlights the text currently being read - so you can follow with your eyes. If you specifically want to listen to websites - such as blogs, news, wiki - you should get our free extension for Chrome

Commercial-Ready

Use our apps for commercial purposes. Generated audio can be used for YouTubes, games, telephony and more. To export the generated speech into high-quality audio files, you can either use our Android app , or record them, as explained here . Read more for ttsreader’s commercial terms. Read more

We love to hear your feedback. Here’s what users said about us:

The new male voice is great. It is quite melodic and natural, much more so then other sites I have tried to use. This is a GREAT tool, well done thanks!

ttsreader.com

This product works amazingly well. I use it to edit my books, pasting in a chapter, having it read back to me while I edit the original. Cuts down my book edit time by over 50% !

Multiple voices from different nationalities. Easy to use interface. Paste text and it will speak. Can create mp3 files.

ttsreader for Android

Great app. Can handle long texts, something other apps can’t. Highly recommended!

What a great App! exactly what i needed, a reader to provide me content efficiently.

ttsreader-x for Chrome

Amazon's Kindle Fire - Can Now Read Websites

on June 6, 2017

Amazon’s Kindle Fire - Can Now Read Websites As TTSReader is Now Available on Amazon’s App Store Get it now for FREE Exciting news! Kindle lovers now got upgraded with some new great features. TTSReader on the Kindle can read out loud any text, pdf and website. It uses the latest algorithms to extract only the relevant text out of the usually-cluttered websites. Great for listening to Wiki articles for instance, blogs and more.

Android Gets the Best In Class Websites Reader

Android Gets Best In Class Websites Reader - With Latest Update to TTSReader Pro Start listening now for FREE Exciting news, as Android’s TTSReader Pro app, has been updated to use TTSReaderX’s algorithms to extract only the relevant text out of websites. This is super important for a text-to-speech website reader, as otherwise the reader would start reading out loud all the ads, menus, sharing buttons and more clutter.

Commercial Licensing & Terms

on May 10, 2017

When is a Commercial License Necessary Using ttsreader.com within your institution If you are a company, or organization, using ttsreader.com, please use our paypal donate link. If you are a personal user, or an educational institute - ttsreader.com is free, no need to even donate - you are welcome, of course :). Using the generated speech for commercial purposes Recording and using the audio generated by TTSReader in a commercial application (ie publishing)

Export Speech to Audio Files

How to Record Audio Played on PC (Speakers) for Free Need to record audio from TTSReader, YouTube or other? Here’s how in a few simple steps (includes screenshots). No need to record the speakers - you can record the audio from within the pc itself. It will be of higher audio quality - as it’s the original digital signal, clear and without ambient noise. Also, no need to purchase a software for that.

See All Posts

Want to see more?

Visit our company's page, to see more of our speech to text (dictation) and text to speech apps for desktops and mobile. For news and tips from our developers visit our blog.

Text to Voice Over Generator

Convert text to voice over online.

Want to make your text content more accessible, engaging, and easy to listen to? Transform any of your text files into lifelike voiceovers! With over 130 languages and dialects to choose from, you can generate speech with realistic human intonation. Plus, you can pick from over 100 voice profiles that best suit your content and effortlessly create captivating audio or video content to share with your audience or team using our text to voice over converter.

Liven up your content with +100 voice profiles

Transform your text into high-quality studio sound narration with our diverse, ready-to-use AI voices. Choose a voice profile that best fits your audience and elevate their audio experience.

Enhance your sound experience

Effortlessly eliminate background noise from your podcasts, extract crystal-clear audio from YouTube videos, seamlessly merge or rearrange music tracks, or enhance the clarity of your voiceovers with our state-of-the-art AI audio enhancer .

Generate voice overs without having to hit the record button

Create content faster that strikes the right chord with your audience using our text to voice over tool. Copy and paste your script, select a voice, preview, and save your new audio.

Add some flair to your audio with music and sound effects

Produce professional-sounding podcasts, interviews, learning courses, and voiceovers for videos that will captivate your audience. Add background music, sound effects, or transitions to keep your audience hooked!

How to use our text to voice over tool:

Click on the Get Started button above to open Flixier in your browser. To access the Text to Speech option, you must first open the Library Tab on the left side.

Now just paste your text into the field on the right side and select your preferred language from the drop-down menu. Then, choose the best voice to charm your audience. You can even listen to different voices with the Preview option until you find the perfect fit. Once you're happy with your text-to-voice-over, click the Add to My Media button to add the new audio to your Library directly.

Once your text-to-voice-over audio file is created on Flixier, it will be automatically saved in your media library. You can either download it as an MP3, store it on cloud storage services, or share it directly with your audience. Simply click on the Export button and select Audio to have it saved as MP3 on your device. This is a very streamlined process that can be done quickly and easily without leaving your browser.

What people say about Flixier

I'm so relieved I found Flixier. I have a YouTube channel with over 700k subscribers and Flixier allows me to collaborate seamlessly with my team, they can work from any device at any time plus, renders are cloud powered and super super fast on any computer.

My main criteria for an editor was that the interface is familiar and most importantly that the renders were in the cloud and super fast. Flixier more than delivered in both. I've now been using it daily to edit Facebook videos for my 1M follower page.

I’ve been looking for a solution like Flixier for years. Now that my virtual team and I can edit projects together on the cloud with Flixier, it tripled my company’s video output! Super easy to use and unbelievably quick exports.

Frequently asked questions.

A text-to-speech generator simply turns any written text into speech without the need to record yourself. With Flixier's text-to-speech tool, you can create content faster and in over 130 languages, making it more accessible to wider audiences.

Flixier text-to-speech tool uses an advanced AI technology to analyze any given text and automatically create realistic-sounding speech with accents and intonations of human-like voices.

Flixier can create audio content in over 130 languages based on your script. You can even customize your voiceover by choosing from over 100 different voice profiles, including male, female, and child voices with different accents.

Need more than a text to voice over tool?

Edit easily, publish in minutes, collaborate in real-time, other text to speech tools, articles, tools and tips, unlock the potential of your pc.

Guide Center

Realistic Voice AI

Lifelike and Powerful AI-Powered Free Online Text to Speech

Try the tool (any language)

How it works

Welcome to Realistic Voice, the leading AI Text-to-Speech platform that brings your written words to life with astonishing realism. Our advanced system utilizes state-of-the-art neural network models to generate natural and human-like speech patterns. So, how does it work? First, you simply input your text into our intuitive interface. Our powerful algorithms then analyze the input, taking into account various linguistic and contextual factors. Next, the system employs deep learning techniques to generate an audio waveform that closely resembles human speech. The resulting output preserves nuances such as intonation, rhythm, and even emotional expressions, ensuring an immersive and authentic auditory experience. Whether you’re a content creator, a developer, or someone looking for a lifelike voice for their project, Realistic Voice is your ultimate solution for converting text into captivating spoken content.

Text-to-Speech technology has revolutionized the way we engage with written content, opening up a wide range of exciting possibilities. With its versatility and natural-sounding voices, TTS can be utilized across various domains. For instance, authors and publishers can transform their books into engaging audiobooks, reaching a wider audience and providing an immersive storytelling experience. Documentaries and educational videos can benefit from TTS by adding a professional and captivating voiceover that enhances the viewer’s understanding and engagement. Content creators on platforms like YouTube and vlogs can use TTS to generate dynamic and expressive voices that accompany their videos, making them more engaging and accessible to diverse audiences. Additionally, TTS can bring poetry to life, providing a unique way to experience and appreciate literary works. From accessibility solutions for individuals with visual impairments to interactive voice-based applications and virtual assistants, the applications of TTS are vast and continually expanding, enabling seamless integration of written content into the auditory realm.

Free text to speech tool

How to use our text to speech (tts) tool.

A text-to-speech reader has the function of reading out loud any text you input. Our tool can read text in over 50 languages and even offers multiple text-to-speech voices for a few widely spoken languages such as English.

Step #1 : Write or paste your text in the input box. You also have the option of uploading a txt file.
Step #2 : Choose your desired language and speaker. You can try out different speakers if there are more available and choose the one you prefer.
Step #3 : Choose the speed of reading. You can set up the text to be read out loud faster or slower than the default.
Step #4 : Choose the font for the text. We recommend a smaller font if you have a large text and want to avoid scrolling, or a bigger font to follow the text while easily read aloud.
Step #5 : Tick the “I’m not a robot” checkbox in the bottom right of the screen.
Step #6 : Press the play button on the bottom of the text box to hear your text read out loud.
Step #7 : Get a share link for the resulting audio file or download it as an mp3. Our tool generates high quality TTS that is easy to understand by everyone.

Choose from 50 languages

Our free text to speech tool offers various languages and natural sounding voices to choose from. We made an effort to make our TTS reader available for as many people as possible by including the most commonly spoken languages worldwide.

We have languages available for the following regions:

Middle East
South-East Asia
Middle Asia (India)
North America

Benefits of using text to speech

TTS is widely used as assistive technology that helps people with reading and visual impairments understand a text. For example:

Visually impaired individuals greatly benefit from having a program read texts out loud to them.
Dyslexic individuals will also benefit from a text to talk reader because they can understand texts more easily.
Children with reading impairments can use text readers to understand lessons easier.
A text to voice tool is also of great help for people with severe speech impairments. Our web browser TTS tool allows them to type what they want to say and instantly play the audio to the person they wish to communicate with.

What is text to speech?

Text to speech is a tool or program that takes text or words input by the user and reads them out loud. It’s used as an assistive technology for people with reading, visual and speech impairments and as a productivity tool.

How does text to speech work?

Text to speech tools use speech synthesis to read texts out loud. The simplest form of speech synthesis uses snippets of human speech to deliver a coherent and natural-sounding message. These snippets are taken from vast libraries of human sounds, words, phrases etc., and they can be used to verbalize almost anything digitally.

You'll probably also like

Explore our range of complimentary tools designed to enhance your experience.

Grow revenue and improve engagement rates by sending personalized, action-driven texts to your customers, staff, and suppliers.

No need to log in! Try Our AI Text To Speech (TTS) Free Tool Here

Select the language, voices, and emotions from different voices that we have and find the best one that goes well with your content.

Turn Text into Amazing Speech: 500+ Natural Sounding AI Voices in 140+ Languages!

500+ AI natural Sounding Voices
Supports 140+ Different Languages
Voices with Real Emotions
Advanced Editor

On4t Voices Has Emotions That Make Them Sound Super Natural

Want to make your voices laugh, happy cry, excited? Check out the emotions that you can add to your voices.

Try Out On4t’s Unique AI Text to Speech Voices That Sound Like Real Human

Want cool voices for your videos? ON4T’s text-to-speech tool has lots of AI voices just for you. It's like having a voice actor, but faster and cheaper. You can make your own special voice-overs easily. It's great for lots of videos and saves you time and money. With our tool, making fun and interesting videos is easy and quick!

Christopher

Get Instant, Perfect Voiceovers in 140+ Languages.

Just a few taps and your text becomes speech in any language you want. Fast, easy, and just for you!

AZERBAIJANI

Over 500+ Human- Sounding Voice Overs for Everyone to read aloud

Multilingual support in English Text To Speech and 140+ other languages

Generate Speech from a Document to Excellent Quality audio version

Add Background Music to Enhance Clarity and Attractiveness with online Text to Speech MP3

Customize the Speed, Pronunciation, and Pitch of the selected natural sounding Voice as Per Your Preference

Undetectable Standard Sounding Voice Overs for Various Situations

Get Multiple Audio Files Against a Single Input Text

Explore and Choose the Perfect Voice Type , Tone, pitch, & Speed

Make Your Text to voice tone More Cheerful, Unfriendly, Whispering, Sad, and Friendly

Entirely Web-based Application that Can Be Accessed without Installation

Merge Multiple Text to Audio Files in One Larger File for Easy Storing and Sharing

It Is Hard For People To Identify The Voices By On4t And Real Voiceovers

Which one do you think is the real voice, and which one is the voice generated by on4t?

On4t’s TTS is best for:

We have happy customers who are using our voices to create tutorial videos, sales pitches, webinars, and much more.

Turning Text To Speech Using On4t Is Easier Than Scrolling Through Your Social Media Feed.

No more need to pay freelancers or voiceover artists anymore

Type or paste your text into the textbox.

Select the voices and style by previewing the voices.

Click Create Voiceover, and once it is ready, download and use it!

With ON4T’s Text to Speech, You Can Make

realistic AI voices and utilize them to generate custom voiceovers for marketers, product developers, authors, and podcasters using our cutting-edge online Text-to-speech service!

Marketers, say goodbye to the hassle and high costs of hiring voiceover artists. With ON4T's Text to Speech, creating the perfect voice for your marketing projects is super easy. Whether you're a small business owner, a digital marketing agency, or a freelance content creator, our platform lets you effortlessly turn your written content into high-quality, natural-sounding AI voices. Compare different voice outputs to find the one that best fits your brand and boosts your engagement.

Check out how our text-to-speech tool can make learning way cooler. You can turn tough and technical topics into easy-to-understand audio. This means you can change written stuff into fun, spoken lessons. It's great for students who want to learn while they're on the bus or just chilling in bed. It's all about making learning easy to grasp and more enjoyable.

Who doesn't love listening to great stories? Now, you can turn your books into audiobooks and reach more listeners. Our free online text-to-speech tool makes it super easy. Just pick a voice you like, copy and paste your story, and click to create the audio. It's a simple way to bring your stories to life and share them with more people!

Want to make your customer service even better? Use our awesome AI voice generator to answer common questions about your products and services. It's so good your customers might just think they're talking to a real person! This means they get quick, clear answers, making everyone's day a bit brighter. Just record your FAQs with our text-to-speech tool, and you're all set to provide top-notch support.

Need to show off your cool new products? Use our text-to-speech reader to make awesome voiceovers for your website, sales pages, tutorials, and demo videos. It's a great way to make your products shine and explain how they work. Plus, it's quick and easy, so you can spend more time getting great feedback on what you've created.

Making cool voiceovers for your podcasts is super quick and easy with our tool. You can make your podcasts sound just right and get more people to listen. Our voices are here to help you get more views and make everything simpler. So, say goodbye to hard stuff and hello to awesome podcast voiceovers!

Text-to-Speech Generation vs. Human VoiceOvers

Why use our text-to-speech reader? It's all about making things easier, cheaper, and faster for you. Instead of finding and paying experts, our tool lets you make cool voiceovers on your own. It's a quick, budget-friendly way to get great-sounding voiceovers without the fuss.

Waste time hiring perfect voiceover artists
Turnaround time of average 1 week
Learn editing skills to use voices
Spend money and time to record again
Super easy to use
Takes less than a minute for an output
Beginner friendly interface
Lifetime updates and new features

How Does On4T's Text-to-Speech Tool Work?

Our text-to-speech tool.

Think of On4T as a super-smart helper that turns what you write into cool, real-sounding speech in just seconds. It's like having a robot read your text out loud. Using it is a piece of cake: type in your text, pick a voice that fits your project, and boom! The tool changes your text into a voiceover that sounds just like a person talking. You can save this as an audio file and use it for anything – podcasts, learning videos, ads, you name it. And the best part? Your listeners won't even guess it's made by ON4T’s AI!

Give Our Human-Sounding AI Voices a Try

Why is Text-to-Speech Super Cool? (Awesome Features)

Text-to-speech is packed with special features that make it the best choice for fast and natural-sounding voiceovers. It's like a magic tool for making amazing AI voices!

500+ AI Voices

Dive into our huge collection of top-quality voices. Choose from male, female, and even kids' voices, all ready for you to use.

Adjust Tone, Pitch & Speed

With our tool, changing how the voice sounds is easy. Make it whisper, shout, or speak at the speed you like. It's all about making the voice just right for your project.

Various Accents

Make your voiceovers feel real with different English accents. American, UK, Canadian, Australian, Indian, South African, Irish, and British accents are all there for you.

140+ Languages

Our awesome Text-to-Speech tool supports loads of languages. From English to Japanese, German to Arabic, and many more, we've got you covered.

Set Voice Emotions

Need your AI voiceover to sound serious, soft, angry, or happy? Our tool makes your voice sound just like a real person, no matter the emotion.

Create Voice Overs Like a Pro

Highlight important words and control how your voiceover sounds. Add pauses and change sentence lengths to get the perfect effect.

Support for All Video Editing Software

The audio files you make with our tool work with any video and audio editing software. It's super easy to add them to your projects.

Optimize Your Efficiency

Save time and boost your productivity with our natural-sounding text reader. It can read your emails and documents out loud in a clear AI voice, so you can listen to your text anytime and stay on top of your game.

Use ON4T’s Text-to-Speech for Multiple Purposes

Sales videos.

Grab your audience's attention with an awesome sales video. Our Text-to-Speech voice generator gives you voiceovers that sound just like a real person. They're great for drawing people in, sparking interest, and bringing customers your way. Whether it's for a cool discount offer or introducing a new product, our AI voice text reader helps you create the perfect voiceover quickly and easily.

Educational Videos

Teachers, now you can make your own voiceovers for lessons and educational videos with On4t's AI Text-to-Speech generator. It's perfect for explaining tricky stuff in a way that's fun and easy to understand. Our tool lets you add feelings to your words, making learning more lively. Creating custom voiceovers for your classes is now super easy and fun!

School Lessons

Want to make your lessons even more exciting? Use our AI voiceover artists to turn your lessons into cool videos. Their voices are clear and friendly, perfect for making learning fun. Plus, with our Text-to-speech voices, you can easily translate lessons into different languages. This means students can listen to them while they're on the bus, out for a run, or even while playing games. Learning becomes more accessible and fun for everyone!

Advertisement and Promotional Videos

In the fast-moving business world, showing off new products is key. Our Text-to-Speech software is here to help. Add voiceover text to scenes without real actors and make your ads more interesting, all while keeping an eye on your budget. Whether you're a startup or a big company, our software is perfect for everyone's advertising needs.

Documentary Videos

Want a voice as impactful as the ones in National Geographic or Discovery Channel documentaries? You've come to the right place! With our huge selection of AI text-to-speech voices, you can create documentaries that really make an impression. A strong and engaging voice is key to sharing important info and keeping viewers hooked. Use our online Text-to-speech to tell inspiring stories or share exciting incidents in a way that really grabs attention.

Audio Books

Turn your blog post, written content, journals, novels, or research work into professional audiobooks that people can listen to while cooking, walking, traveling, or in the car. Our AI text to Speech emotion-based engine is ideal for translating your words into perfect voice and adding emotions to the voice tone, turning it flawless for commercial purposes of creating audiobooks.

E-commerce Videos

Explaining how a product works is crucial for attracting online users. Add various female voices, male, old, or child voices in different accents to your e-commerce videos with On4t Text-to-Speech reader. Maximize the potential of your e-commerce campaign by utilizing multilingual languages, top-quality AI voice Generator Text to Speech, and distinctive voice tones.

Enhance your podcast brand with custom commercials, sound bites, and engaging content. Use our online Text-to-Speech tool to create a professional podcasting customer experience here. Engage your audience with audio content and provide additional support through custom voiceovers.

Health Videos

Guide your audience about health-related issues and remedies to counter them easily in a gentle and kind tone. Choose the unique voice, style, native language, and gender of your preference and create the best quality health videos featuring the perfect voiceover without hiring freelancers.

For Individuals with Visual Impairments

On4t's text-to-speech generator is a game-changer for people with visual impairments, dyslexia, or other disabilities. It turns written words into spoken ones, making things easier to understand. If reading is tough or if you have trouble seeing, our Text to Speech online software is here to help. It changes your reading material into audio so you can just listen to it. Super convenient and helpful!

Common Questions Asked Related to ON4T Text-to-Speech

Can i use this text-to-speech without installing any application, can we use ai voices for youtube monetized content, how many voices are available in this tts tool, can i access this ai text to voice converter on mac and android, what if i get stuck at some point while converting text to speech, what if i don't like the quality of the voices, what is a text-to-speech generator, how does on4t’s text to audio converter tool work, on4t text to speech pricing.

We offer a 30-day free trial and a no-questions-asked money-back guarantee. If you're not happy with our Text-to-Speech tool, you'll get a full refund within 24 hours. Terms and conditions apply.

$19 / Monthly

Unlimited Voiceovers
500,000 Characters
500+ Voices
Unlimited Projects
12k Characters Per Clip
Merge Unlimited Audios
Commercial License

$39 / Quarterly

1,500,000 Characters

$49 / Yearly

Agency Plan

4,000.000 Characters

Discover ON4T Premium with Zero Cost: Try it Free Today

Discover more:

AI Speech to Text: Revolutionizing Transcription

Table of contents.

In the ever-evolving landscape of technology, AI Speech to Text technology stands out as a beacon of innovation, especially in how we handle and process language. This technology, which encompasses everything from automatic speech recognition (ASR) to audio transcription , is reshaping industries, enhancing accessibility, and streamlining workflows.

What is Speech to Text?

Speech to Text, often abbreviated as speech-to-text , refers to the technology used to transcribe spoken language into written text. This can be applied to various audio sources, such as video files , podcasts , and even real-time conversations. Thanks to advancements in machine learning and natural language processing , today’s speech recognition systems are more accurate and faster than ever.

Core Technologies and Terminology

ASR (Automatic Speech Recognition) : This is the engine that drives transcription services, converting speech into a string of text.
Speech Models : These are trained on extensive datasets containing thousands of hours of audio files in multiple languages, such as English, Spanish, French, and German, to ensure accurate transcription .
Speaker Diarization : This feature identifies different speakers in an audio, making it ideal for video transcription and audio files from meetings or interviews.
Natural Language Processing (NLP) : Used to enhance the context understanding and summarization of the transcribed text.

Applications and Use Cases

Speech-to-text technology is highly versatile, supporting a range of applications:

Video Content : From generating subtitles to creating searchable text databases.
Podcasts : Enhancing accessibility with transcripts that include timestamps , making specific content easy to find.
Real-time Applications : Like live event captioning and customer support, where latency and transcription accuracy are critical.

Building Your Own Speech to Text System

For those interested in building their own system, numerous resources are available:

Open Source Tools : Software like Whisper and frameworks that allow customization and integration into existing workflows.
APIs and SDKs : Platforms like Google Cloud offer robust APIs that facilitate the integration of speech-to-text capabilities into apps and services, complete with detailed tutorials .
On-Premises Solutions : For businesses needing to keep data in-house for security reasons, on-premises setups are also viable.
AI tools : AI speech to text or AI transcription tools like Speechify work right in your browser.

Challenges and Considerations

While the technology is impressive, it’s not without its challenges. Word error rate (WER) remains a significant metric for assessing the quality of transcription services. Additionally, the ability to accurately capture specific words or phrases and sentiment analysis can vary depending on the speech models used and the complexity of the audio.

Pricing and Accessibility

The cost of using speech-to-text services can vary. Many providers offer a tiered pricing model based on usage, with some offering free tiers for startups or small-scale applications. Accessibility is also a key focus, with efforts to support multiple languages and dialects expanding rapidly.

The Future of Speech to Text

Looking ahead, the integration of speech-to-text technology in daily life and business processes is only going to deepen. With continuous improvements in speech models , low-latency applications, and the embrace of multi-language support , the potential to bridge communication gaps and enhance data accessibility is immense. As artificial intelligence and machine learning evolve, so too will the capabilities of speech-to-text technologies, making every interaction more engaging and informed.

Whether you are a pro looking to integrate advanced speech-to-text APIs into a complex system, or a newcomer eager to experiment with open-source software , the world of AI speech to text offers endless possibilities. Dive into this technology to unlock new levels of efficiency and innovation in your projects and products.

Try Speechify AI Transcription

Pricing : Free to try

Effortlessly transcribe any video in a snap. Just upload your audio or video and hit “Transcribe” for the most precise transcription.

Boasting support for over 20 languages, Speechify Video Transcription stands out as the premier AI transcription service.

Speechify AI Transcription Features

Easy to use UI
Multilingual transcription
Transcribe directly from YouTube or upload a video
Transcribe your video in minutes
Great for individuals to large teams

Speechify is the best option for AI transcription. Move seamlessly between the suite of products in Speechify Studio or use just AI transcription. Try it for yourself, for free !

Frequently Asked Questions

is there an ai for speech to text.

Yes, AI technologies that perform speech to text, like automatic speech recognition (ASR) systems, utilize advanced machine learning models and natural language processing to transcribe audio files and real-time speech accurately.

Which AI converts audio to text?

AI models such as Google Cloud’s Speech-to-Text and OpenAI’s Whisper are popular choices that convert audio to text. They offer features like speaker diarization, support for multiple languages, and high transcription accuracy.

How do I convert AI voice to text?

To convert AI voice to text, you can use speech-to-text APIs provided by platforms like Google Cloud, which allow integration into existing applications to transcribe audio files, including podcasts and video content, in real-time.

What is the AI that converts voice to text?

AI that converts voice to text involves automatic speech recognition technologies, like those offered by Google Cloud and OpenAI Whisper. These AIs are designed to provide accurate transcription of natural language from audio and video files.

Previous Real-Time AI Dubbing with Voice Preservation
Next AI Speech Recognition: Everything You Should Know

Cliff Weitzman

Cliff Weitzman is a dyslexia advocate and the CEO and founder of Speechify, the #1 text-to-speech app in the world, totaling over 100,000 5-star reviews and ranking first place in the App Store for the News & Magazines category. In 2017, Weitzman was named to the Forbes 30 under 30 list for his work making the internet more accessible to people with learning disabilities. Cliff Weitzman has been featured in EdSurge, Inc., PC Mag, Entrepreneur, Mashable, among other leading outlets.

Recent Blogs

AI Speech Recognition: Everything You Should Know

Real-Time AI Dubbing with Voice Preservation

How to Add Voice Over to Video: A Step-by-Step Guide

Voice Simulator & Content Creation with AI-Generated Voices

Convert Audio and Video to Text: Transcription Has Never Been Easier.

How to Record Voice Overs Properly Over Gameplay: Everything You Need to Know

Voicemail Greeting Generator: The New Way to Engage Callers

How to Avoid AI Voice Scams

Character AI Voices: Revolutionizing Audio Content with Advanced Technology

Best AI Voices for Video Games

How to Monetize YouTube Channels with AI Voices

Multilingual Voice API: Bridging Communication Gaps in a Diverse World

Resemble.AI vs ElevenLabs: A Comprehensive Comparison

Apps to Read PDFs on Mobile and Desktop

How to Convert a PDF to an Audiobook: A Step-by-Step Guide

AI for Translation: Bridging Language Barriers

IVR Conversion Tool: A Comprehensive Guide for Healthcare Providers

Best AI Speech to Speech Tools

AI Voice Recorder: Everything You Need to Know

The Best Multilingual AI Speech Models

Program that will Read PDF Aloud: Yes it Exists

How to Convert Your Emails to an Audiobook: A Step-by-Step Tutorial

How to Convert iOS Files to an Audiobook

How to Convert Google Docs to an Audiobook

How to Convert Word Docs to an Audiobook

Alternatives to Deepgram Text to Speech API

Is Text to Speech HSA Eligible?

Can You Use an HSA for Speech Therapy?

Surprising HSA-Eligible Items

Ultimate guide to ElevenLabs

Speechify text to speech helps you save time

Popular blogs, the best celebrity voice generators in 2024.

YouTube Text to Speech: Elevating Your Video Content with Speechify

The 7 best alternatives to Synthesia.io

Everything you need to know about text to speech on TikTok

The 10 best text-to-speech apps for android.

How to convert a PDF to speech

The top girl voice changers

How to use Siri text to speech

Obama text to speech

Robot voice generators: the futuristic frontier of audio creation, pdf read aloud: free & paid options.

Alternatives to FakeYou text to speech

All about deepfake voices, tiktok voice generator, text to speech goanimate, the best celebrity text to speech voice generators, pdf audio reader, how to get text to speech indian voices, elevating your anime experience with anime voice generators, best text to speech online.

Only available on iPhone and iPad

To access our catalog of 100,000+ audiobooks, you need to use an iOS device.

Coming to Android soon...

Join the waitlist

Enter your email and we will notify you as soon as Speechify Audiobooks is available for you.

You’ve been added to the waitlist. We will notify you as soon as Speechify Audiobooks is available for you.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
My Account Login
Explore content
About the journal
Publish with us
Sign up for alerts
Open access
Published: 08 April 2024

A neural speech decoding framework leveraging deep learning and speech synthesis

Xupeng Chen 1 na1 ,
Ran Wang 1 na1 ,
Amirhossein Khalilian-Gourtani ORCID: orcid.org/0000-0003-1376-9583 2 ,
Leyao Yu 2 , 3 ,
Patricia Dugan 2 ,
Daniel Friedman 2 ,
Werner Doyle 4 ,
Orrin Devinsky 2 ,
Yao Wang ORCID: orcid.org/0000-0003-3199-3802 1 , 3 na2 &
Adeen Flinker ORCID: orcid.org/0000-0003-1247-1283 2 , 3 na2

Nature Machine Intelligence ( 2024 ) Cite this article

5493 Accesses

372 Altmetric

Metrics details

Neural decoding

A preprint version of the article is available at bioRxiv.

Decoding human speech from neural signals is essential for brain–computer interface (BCI) technologies that aim to restore speech in populations with neurological deficits. However, it remains a highly challenging task, compounded by the scarce availability of neural signals with corresponding speech, data complexity and high dimensionality. Here we present a novel deep learning-based neural speech decoding framework that includes an ECoG decoder that translates electrocorticographic (ECoG) signals from the cortex into interpretable speech parameters and a novel differentiable speech synthesizer that maps speech parameters to spectrograms. We have developed a companion speech-to-speech auto-encoder consisting of a speech encoder and the same speech synthesizer to generate reference speech parameters to facilitate the ECoG decoder training. This framework generates natural-sounding speech and is highly reproducible across a cohort of 48 participants. Our experimental results show that our models can decode speech with high correlation, even when limited to only causal operations, which is necessary for adoption by real-time neural prostheses. Finally, we successfully decode speech in participants with either left or right hemisphere coverage, which could lead to speech prostheses in patients with deficits resulting from left hemisphere damage.

Decoding speech perception from non-invasive brain recordings

Alexandre Défossez, Charlotte Caucheteux, … Jean-Rémi King

Speech synthesis from neural decoding of spoken sentences

Gopala K. Anumanchipalli, Josh Chartier & Edward F. Chang

Restoring speech intelligibility for hearing aid users with deep learning

Peter Udo Diehl, Yosef Singer, … Veit M. Hofmann

Speech loss due to neurological deficits is a severe disability that limits both work life and social life. Advances in machine learning and brain–computer interface (BCI) systems have pushed the envelope in the development of neural speech prostheses to enable people with speech loss to communicate 1 , 2 , 3 , 4 , 5 . An effective modality for acquiring data to develop such decoders involves electrocorticographic (ECoG) recordings obtained in patients undergoing epilepsy surgery 4 , 5 , 6 , 7 , 8 , 9 , 10 . Implanted electrodes in patients with epilepsy provide a rare opportunity to collect cortical data during speech with high spatial and temporal resolution, and such approaches have produced promising results in speech decoding 4 , 5 , 8 , 9 , 10 , 11 .

Two challenges are inherent to successfully carrying out speech decoding from neural signals. First, the data to train personalized neural-to-speech decoding models are limited in duration, and deep learning models require extensive training data. Second, speech production varies in rate, intonation, pitch and so on, even within a single speaker producing the same word, complicating the underlying model representation 12 , 13 . These challenges have led to diverse speech decoding approaches with a range of model architectures. Currently, public code to test and replicate findings across research groups is limited in availability.

Earlier approaches to decoding and synthesizing speech spectrograms from neural signals focused on linear models. These approaches achieved a Pearson correlation coefficient (PCC) of ~0.6 or lower, but with simple model architectures that are easy to interpret and do not require large training datasets 14 , 15 , 16 . Recent research has focused on deep neural networks leveraging convolutional 8 , 9 and recurrent 5 , 10 , 17 network architectures. These approaches vary across two major dimensions: the intermediate latent representation used to model speech and the speech quality produced after synthesis. For example, cortical activity has been decoded into an articulatory movement space, which is then transformed into speech, providing robust decoding performance but with a non-natural synthetic voice reconstruction 17 . Conversely, some approaches have produced naturalistic reconstruction leveraging wavenet vocoders 8 , generative adversarial networks (GAN) 11 and unit selection 18 , but achieve limited accuracy. A recent study in one implanted patient 19 provided both robust accuracies and a naturalistic speech waveform by leveraging quantized HuBERT features 20 as an intermediate representation space and a pretrained speech synthesizer that converts the HuBERT features into speech. However, HuBERT features do not carry speaker-dependent acoustic information and can only be used to generate a generic speaker’s voice, so they require a separate model to translate the generic voice to a specific patient’s voice. Furthermore, this study and most previous approaches have employed non-causal architectures, which may limit real-time applications, which typically require causal operations.

To address these issues, in this Article we present a novel ECoG-to-speech framework with a low-dimensional intermediate representation guided by subject-specific pre-training using speech signal only (Fig. 1 ). Our framework consists of an ECoG decoder that maps the ECoG signals to interpretable acoustic speech parameters (for example, pitch, voicing and formant frequencies), as well as a speech synthesizer that translates the speech parameters to a spectrogram. The speech synthesizer is differentiable, enabling us to minimize the spectrogram reconstruction error during training of the ECoG decoder. The low-dimensional latent space, together with guidance on the latent representation generated by a pre-trained speech encoder, overcomes data scarcity issues. Our publicly available framework produces naturalistic speech that highly resembles the speaker’s own voice, and the ECoG decoder can be realized with different deep learning model architectures and using different causality directions. We report this framework with multiple deep architectures (convolutional, recurrent and transformer) as the ECoG decoder, and apply it to 48 neurosurgical patients. Our framework performs with high accuracy across the models, with the best performance obtained by the convolutional (ResNet) architecture (PCC of 0.806 between the original and decoded spectrograms). Our framework can achieve high accuracy using only causal processing and relatively low spatial sampling on the cortex. We also show comparable speech decoding from grid implants on the left and right hemispheres, providing a proof of concept for neural prosthetics in patients suffering from expressive aphasia (with damage limited to the left hemisphere), although such an approach must be tested in patients with damage to the left hemisphere. Finally, we provide a publicly available neural decoding pipeline ( https://github.com/flinkerlab/neural_speech_decoding ) that offers flexibility in ECoG decoding architectures to push forward research across the speech science and prostheses communities.

The upper part shows the ECoG-to-speech decoding pipeline. The ECoG decoder generates time-varying speech parameters from ECoG signals. The speech synthesizer generates spectrograms from the speech parameters. A separate spectrogram inversion algorithm converts the spectrograms to speech waveforms. The lower part shows the speech-to-speech auto-encoder, which generates the guidance for the speech parameters to be produced by the ECoG decoder during its training. The speech encoder maps an input spectrogram to the speech parameters, which are then fed to the same speech synthesizer to reproduce the spectrogram. The speech encoder and a few learnable subject-specific parameters in the speech synthesizer are pre-trained using speech signals only. Only the upper part is needed to decode the speech from ECoG signals once the pipeline is trained.

ECoG-to-speech decoding framework

Our ECoG-to-speech framework consists of an ECoG decoder and a speech synthesizer (shown in the upper part of Fig. 1 ). The neural signals are fed into an ECoG decoder, which generates speech parameters, followed by a speech synthesizer, which translates the parameters into spectrograms (which are then converted to a waveform by the Griffin–Lim algorithm 21 ). The training of our framework comprises two steps. We first use semi-supervised learning on the speech signals alone. An auto-encoder, shown in the lower part of Fig. 1 , is trained so that the speech encoder derives speech parameters from a given spectrogram, while the speech synthesizer (used here as the decoder) reproduces the spectrogram from the speech parameters. Our speech synthesizer is fully differentiable and generates speech through a weighted combination of voiced and unvoiced speech components generated from input time series of speech parameters, including pitch, formant frequencies, loudness and so on. The speech synthesizer has only a few subject-specific parameters, which are learned as part of the auto-encoder training (more details are provided in the Methods Speech synthesizer section). Currently, our speech encoder and speech synthesizer are subject-specific and can be trained using any speech signal of a participant, not just those with corresponding ECoG signals.

In the next step, we train the ECoG decoder in a supervised manner based on ground-truth spectrograms (using measures of spectrogram difference and short-time objective intelligibility, STOI 8 , 22 ), as well as guidance for the speech parameters generated by the pre-trained speech encoder (that is, reference loss between speech parameters). By limiting the number of speech parameters (18 at each time step; Methods section Summary of speech parameters ) and using the reference loss, the ECoG decoder can be trained with limited corresponding ECoG and speech data. Furthermore, because our speech synthesizer is differentiable, we can back-propagate the spectral loss (differences between the original and decoded spectrograms) to update the ECoG decoder. We provide multiple ECoG decoder architectures to choose from, including 3D ResNet 23 , 3D Swin Transformer 24 and LSTM 25 . Importantly, unlike many methods in the literature, we employ ECoG decoders that can operate in a causal manner, which is necessary for real-time speech generation from neural signals. Note that, once the ECoG decoder and speech synthesizer are trained, they can be used for ECoG-to-speech decoding without using the speech encoder.

Data collection

We employed our speech decoding framework across N = 48 participants who consented to complete a series of speech tasks (Methods section Experiments design). These participants, as part of their clinical care, were undergoing treatment for refractory epilepsy with implanted electrodes. During the hospital stay, we acquired synchronized neural and acoustic speech data. ECoG data were obtained from five participants with hybrid-density (HB) sampling (clinical-research grid) and 43 participants with low-density (LD) sampling (standard clinical grid), who took part in five speech tasks: auditory repetition (AR), auditory naming (AN), sentence completion (SC), word reading (WR) and picture naming (PN). These tasks were designed to elicit the same set of spoken words across tasks while varying the stimulus modality. We provided 50 repeated unique words (400 total trials per participant), all of which were analysed locked to the onset of speech production. We trained a model for each participant using 80% of available data for that participant and evaluated the model on the remaining 20% of data (with the exception of the more stringent word-level cross-validation).

Speech decoding performance and causality

We first aimed to directly compare the decoding performance across different architectures, including those that have been employed in the neural speech decoding literature (recurrent and convolutional) and transformer-based models. Although any decoder architecture could be used for the ECoG decoder in our framework, employing the same speech encoder guidance and speech synthesizer, we focused on three representative models for convolution (ResNet), recurrent (LSTM) and transformer (Swin) architectures. Note that any of these models can be configured to use temporally non-causal or causal operations. Our results show that ResNet outperformed the other models, providing the highest PCC across N = 48 participants (mean PCC = 0.806 and 0.797 for non-causal and causal, respectively), closely followed by Swin (mean PCC = 0.792 and 0.798 for non-causal and causal, respectively) (Fig. 2a ). We found the same when evaluating the three models using STOI+ (ref. 26 ), as shown in Supplementary Fig. 1a . The causality of machine learning models for speech production has important implications for BCI applications. A causal model only uses past and current neural signals to generate speech, whereas non-causal models use past, present and future neural signals. Previous reports have typically employed non-causal models 5 , 8 , 10 , 17 , which can use neural signals related to the auditory and speech feedback that is unavailable in real-time applications. Optimally, only the causal direction should be employed. We thus compared the performance of the same models with non-causal and causal temporal operations. Figure 2a compares the decoding results of causal and non-causal versions of our models. The causal ResNet model (PCC = 0.797) achieved a performance comparable to that of the non-causal model (PCC = 0.806), with no significant differences between the two (Wilcoxon two-sided signed-rank test P = 0.093). The same was true for the causal Swin model (PCC = 0.798) and its non-causal (PCC = 0.792) counterpart (Wilcoxon two-sided signed-rank test P = 0.196). In contrast, the performance of the causal LSTM model (PCC = 0.712) was significantly inferior to that of its non-causal (PCC = 0.745) version (Wilcoxon two-sided signed-rank test P = 0.009). Furthermore, the LSTM model showed consistently lower performance than ResNet and Swin. However, we did not find significant differences between the causal ResNet and causal Swin performances (Wilcoxon two-sided signed-rank test P = 0.587). Because the ResNet and Swin models had the highest performance and were on par with each other and their causal counterparts, we chose to focus further analyses on these causal models, which we believe are best suited for prosthetic applications.

a , Performances of ResNet, Swin and LSTM models with non-causal and causal operations. The PCC between the original and decoded spectrograms is evaluated on the held-out testing set and shown for each participant. Each data point corresponds to a participant’s average PCC across testing trials. b , A stringent cross-validation showing the performance of the causal ResNet model on unseen words during training from five folds; we ensured that the training and validation sets in each fold did not overlap in unique words. The performance across all five validation folds was comparable to our trial-based validation, denoted for comparison as ResNet (identical to the ResNet causal model in a ). c – f , Examples of decoded spectrograms and speech parameters from the causal ResNet model for eight words (from two participants) and the PCC values for the decoded and reference speech parameters across all participants. Spectrograms of the original ( c ) and decoded ( d ) speech are shown, with orange curves overlaid representing the reference voice weight learned by the speech encoder ( c ) and the decoded voice weight from the ECoG decoder ( d ). The PCC between the decoded and reference voice weights is shown on the right across all participants. e , Decoded and reference loudness parameters for the eight words, and the PCC values of the decoded loudness parameters across participants (boxplot on the right). f , Decoded (dashed) and reference (solid) parameters for pitch ( f 0 ) and the first two formants ( f 1 and f 2 ) are shown for the eight words, as well as the PCC values across participants (box plots to the right). All box plots depict the median (horizontal line inside the box), 25th and 75th percentiles (box) and 25th or 75th percentiles ± 1.5 × interquartile range (whiskers) across all participants ( N = 48). Yellow error bars denote the mean ± s.e.m. across participants.

Source data

To ensure our framework can generalize well to unseen words, we added a more stringent word-level cross-validation in which random (ten unique) words were entirely held out during training (including both pre-training of the speech encoder and speech synthesizer and training of the ECoG decoder). This ensured that different trials from the same word could not appear in both the training and testing sets. The results shown in Fig. 2b demonstrate that performance on the held-out words is comparable to our standard trial-based held-out approach (Fig. 2a , ‘ResNet’). It is encouraging that the model can decode unseen validation words well, regardless of which words were held out during training.

Next, we show the performance of the ResNet causal decoder on the level of single words across two representative participants (LD grids). The decoded spectrograms accurately preserve the spectro-temporal structure of the original speech (Fig. 2c,d ). We also compare the decoded speech parameters with the reference parameters. For each parameter, we calculated the PCC between the decoded time series and the reference sequence, showing average PCC values of 0.781 (voice weight, Fig. 2d ), 0.571 (loudness, Fig. 2e ), 0.889 (pitch f 0 , Fig. 2f ), 0.812 (first formant f 1 , Fig. 2f ) and 0.883 (second formant f 2 , Fig. 2f ). Accurate reconstruction of the speech parameters, especially the pitch, voice weight and first two formants, is essential for accurate speech decoding and naturalistic reconstruction that mimics a participant’s voice. We also provide a non-causal version of Fig. 2 in Supplementary Fig. 2 . The fact that both non-causal and causal models can yield reasonable decoding results is encouraging.

Left-hemisphere versus right-hemisphere decoding

Most speech decoding studies have focused on the language- and speech-dominant left hemisphere 27 . However, little is known about decoding speech representations from the right hemisphere. To this end, we compared left- versus right-hemisphere decoding performance across our participants to establish the feasibility of a right-hemisphere speech prosthetic. For both our ResNet and Swin decoders, we found robust speech decoding from the right hemisphere (ResNet PCC = 0.790, Swin PCC = 0.798) that was not significantly different from that of the left (Fig. 3a , ResNet independent t -test, P = 0.623; Swin independent t -test, P = 0.968). A similar conclusion held when evaluating STOI+ (Supplementary Fig. 1b , ResNet independent t -test, P = 0.166; Swin independent t -test, P = 0.114). Although these results suggest that it may be feasible to use neural signals in the right hemisphere to decode speech for patients who suffer damage to the left hemisphere and are unable to speak 28 , it remains unknown whether intact left-hemisphere cortex is necessary to allow for speech decoding from the right hemisphere until tested in such patients.

a , Comparison between left- and right-hemisphere participants using causal models. No statistically significant differences (ResNet independent t -test, P = 0.623; Swin Wilcoxon independent t -test, P = 0.968) in PCC values exist between left- ( N = 32) and right- ( N = 16) hemisphere participants. b , An example hybrid-density ECoG array with a total of 128 electrodes. The 64 electrodes marked in red correspond to a LD placement. The remaining 64 green electrodes, combined with red electrodes, reflect HB placement. c , Comparison between causal ResNet and causal Swin models for the same participant across participants with HB ( N = 5) or LD ( N = 43) ECoG grids. The two models show similar decoding performances from the HB and LD grids. d , Decoding PCC values across 50 test trials by the ResNet model for HB ( N = 5) participants when all electrodes are used versus when only LD-in-HB electrodes ( N = 5) are considered. There are no statistically significant differences for four out of five participants (Wilcoxon two-sided signed-rank test, P = 0.114, 0.003, 0.0773, 0.472 and 0.605, respectively). All box plots depict the median (horizontal line inside box), 25th and 75th percentiles (box) and 25th or 75th percentiles ± 1.5 × interquartile range (whiskers). Yellow error bars denote mean ± s.e.m. Distributions were compared with each other as indicated, using the Wilcoxon two-sided signed-rank test and independent t -test. ** P < 0.01; NS, not significant.

Effect of electrode density

Next, we assessed the impact of electrode sampling density on speech decoding, as many previous reports use higher-density grids (0.4 mm) with more closely spaced contacts than typical clinical grids (1 cm). Five participants consented to hybrid grids (Fig. 3b , HB), which typically had LD electrode sampling but with additional electrodes interleaved. The HB grids provided a decoding performance similar to clinical LD grids in terms of PCC values (Fig. 3c ), with a slight advantage in STOI+, as shown in Supplementary Fig. 3b . To ascertain whether the additional spatial sampling indeed provides improved speech decoding, we compared models that decode speech based on all the hybrid electrodes versus only the LD electrodes in participants with HB grids (comparable to our other LD participants). Our findings (Fig. 3d ) suggest that the decoding results were not significantly different from each other (with the exception of participant 2) in terms of PCC and STOI+ (Supplementary Fig. 3c ). Together, these results suggest that our models can learn speech representations well from both high and low spatial sampling of the cortex, with the exciting finding of robust speech decoding from the right hemisphere.

Contribution analysis

Finally, we investigated which cortical regions contribute to decoding to provide insight for the targeted implantation of future prosthetics, especially on the right hemisphere, which has not yet been investigated. We used an occlusion approach to quantify the contributions of different cortical sites to speech decoding. If a region is involved in decoding, occluding the neural signal in the corresponding electrode (that is, setting the signal to zero) will reduce the accuracy (PCC) of the speech reconstructed on testing data (Methods section Contribution analysis ). We thus measured each region’s contribution by decoding the reduction in the PCC when the corresponding electrode was occluded. We analysed all electrodes and participants with causal and non-causal versions of the ResNet and Swin decoders. The results in Fig. 4 show similar contributions for the ResNet and Swin models (Supplementary Figs. 8 and 9 describe the noise-level contribution). The non-causal models show enhanced auditory cortex contributions compared with the causal models, implicating auditory feedback in decoding, and underlying the importance of employing only causal models during speech decoding because neural feedback signals are not available for real-time decoding applications. Furthermore, across the causal models, both the right and left hemispheres show similar contributions across the sensorimotor cortex, especially on the ventral portion, suggesting the potential feasibility of right-hemisphere neural prosthetics.

Visualization of the contribution of each cortical location to the decoding result achieved by both causal and non-causal decoding models through an occlusion analysis. The contribution of each electrode region in each participant is projected onto the standardized Montreal Neurological Institute (MNI) brain anatomical map and then averaged over all participants. Each subplot shows the causal or non-causal contribution of different cortical locations (red indicates a higher contribution; yellow indicates a lower contribution). For visualization purposes, we normalized the contribution of each electrode location by the local grid density, because there were multiple participants with non-uniform density.

Our novel pipeline can decode speech from neural signals by leveraging interchangeable architectures for the ECoG decoder and a novel differentiable speech synthesizer (Fig. 5 ). Our training process relies on estimating guidance speech parameters from the participants’ speech using a pre-trained speech encoder (Fig. 6a ). This strategy enabled us to train ECoG decoders with limited corresponding speech and neural data, which can produce natural-sounding speech when paired with our speech synthesizer. Our approach was highly reproducible across participants ( N = 48), providing evidence for successful causal decoding with convolutional (ResNet; Fig. 6c ) and transformer (Swin; Fig. 6d ) architectures, both of which outperformed the recurrent architecture (LSTM; Fig. 6e ). Our framework can successfully decode from both high and low spatial sampling with high levels of decoding performance. Finally, we provide potential evidence for robust speech decoding from the right hemisphere as well as the spatial contribution of cortical structures to decoding across the hemispheres.

Our speech synthesizer generates the spectrogram at time t by combining a voiced component and an unvoiced component based on a set of speech parameters at t . The upper part represents the voice pathway, which generates the voiced component by passing a harmonic excitation with fundamental frequency ${f}_{0}^{\;t}$ through a voice filter (which is the sum of six formant filters, each specified by formant frequency ${f}_{i}^{\;t}$ and amplitude ${a}_{i}^{t}$ ). The lower part describes the noise pathway, which synthesizes the unvoiced sound by passing white noise through an unvoice filter (consisting of a broadband filter defined by centre frequency ${f}_{\hat{u}}^{\;t}$ , bandwidth ${b}_{\hat{u}}^{t}$ and amplitude ${a}_{\hat{u}}^{t}$ , and the same six formant filters used for the voice filter). The two components are next mixed with voice weight α t and unvoice weight 1 − α t , respectively, and then amplified by loudness L t . A background noise (defined by a stationary spectrogram B ( f )) is finally added to generate the output spectrogram. There are a total of 18 speech parameters at any time t , indicated in purple boxes.

a , The speech encoder architecture. We input a spectrogram into a network of temporal convolution layers and channel MLPs that produce speech parameters. b , c , The ECoG decoder ( c ) using the 3D ResNet architecture. We first use several temporal and spatial convolutional layers with residual connections and spatiotemporal pooling to generate downsampled latent features, and then use corresponding transposed temporal convolutional layers to upsample the features to the original temporal dimension. We then apply temporal convolution layers and channel MLPs to map the features to speech parameters, as shown in b . The non-causal version uses non-causal temporal convolution in each layer, whereas the causal version uses causal convolution. d , The ECoG decoder using the 3D Swin architecture. We use three or four stages of 3D Swin blocks with spatial-temporal attention (three blocks for LD and four blocks for HB) to extract the features from the ECoG signal. We then use the transposed versions of temporal convolution layers as in c to upsample the features. The resulting features are mapped to the speech parameters using the same structure as shown in b . Non-causal versions apply temporal attention to past, present and future tokens, whereas the causal version applies temporal attention only to past and present tokens. e , The ECoG decoder using LSTM layers. We use three LSTM layers and one layer of channel MLP to generate features. We then reuse the prediction layers in b to generate the corresponding speech parameters. The non-causal version employs bidirectional LSTM in each layer, whereas the causal version uses unidirectional LSTM.

Our decoding pipeline showed robust speech decoding across participants, leading to PCC values within the range 0.62–0.92 (Fig. 2a ; causal ResNet mean 0.797, median 0.805) between the decoded and ground-truth speech across several architectures. We attribute our stable training and accurate decoding to the carefully designed components of our pipeline (for example, the speech synthesizer and speech parameter guidance) and the multiple improvements ( Methods sections Speech synthesizer , ECoG decoder and Model training ) over our previous approach on the subset of participants with hybrid-density grids 29 . Previous reports have investigated speech- or text-decoding using linear models 14 , 15 , 30 , transitional probability 4 , 31 , recurrent neural networks 5 , 10 , 17 , 19 , convolutional neural networks 8 , 29 and other hybrid or selection approaches 9 , 16 , 18 , 32 , 33 . Overall, our results are similar to (or better than) many previous reports (54% of our participants showed higher than 0.8 for the decoding PCC; Fig. 3c ). However, a direct comparison is complicated by multiple factors. Previous reports vary in terms of the reported performance metrics, as well as the stimuli decoded (for example, continuous speech versus single words) and the cortical sampling (that is, high versus low density, depth electrodes compared with surface grids). Our publicly available pipeline, which can be used across multiple neural network architectures and tested on various performance metrics, can facilitate the research community to conduct more direct comparisons while still adhering to a high accuracy of speech decoding.

The temporal causality of decoding operations, critical for real-time BCI applications, has not been considered by most previous studies. Many of these non-causal models relied on auditory (and somatosensory) feedback signals. Our analyses show that non-causal models rely on a robust contribution from the superior temporal gyrus (STG), which is mostly eliminated using a causal model (Fig. 4 ). We believe that non-causal models would show limited generalizability to real-time BCI applications due to their over-reliance on feedback signals, which may be absent (if no delay is allowed) or incorrect (if a short latency is allowed during real-time decoding). Some approaches used imagined speech, which avoids feedback during training 16 , or showed generalizability to mimed production lacking auditory feedback 17 , 19 . However, most reports still employ non-causal models, which cannot rule out feedback during training and inference. Indeed, our contribution maps show robust auditory cortex recruitment for the non-causal ResNet and Swin models (Fig. 4 , in contrast to their causal counterparts, which decode based on more frontal regions. Furthermore, the recurrent neural networks that are widely used in the literature 5 , 19 are typically bidirectional, producing non-causal behaviours and longer latencies for prediction during real-time applications. Unidirectional causal results are typically not reported. The recurrent network we tested performed the worst when trained with one direction (Fig. 2a , causal LSTM). Although our current focus was not real-time decoding, we were able to synthesize speech from neural signals with a delay of under 50 ms (Supplementary Table 1 ), which provides minimal auditory delay interference and allows for normal speech production 34 , 35 . Our data suggest that causal convolutional and transformer models can perform on par with their non-causal counterparts and recruit more relevant cortical structures for real-time decoding.

In our study we have leveraged an intermediate speech parameter space together with a novel differentiable speech synthesizer to decode subject-specific naturalistic speech (Fig. 1 . Previous reports used varying approaches to model speech, including an intermediate kinematic space 17 , an acoustically relevant intermediate space using HuBERT features 19 derived from a self-supervised speech masked prediction task 20 , an intermediate random vector (that is, GAN) 11 or direct spectrogram representations 8 , 17 , 36 , 37 . Our choice of speech parameters as the intermediate representation allowed us to decode subject-specific acoustics. Our intermediate acoustic representation led to significantly more accurate speech decoding than directly mapping ECoG to the speech spectrogram 38 , and than mapping ECoG to a random vector, which is then fed to a GAN-based speech synthesizer 11 (Supplementary Fig. 10 ). Unlike the kinematic representation, our acoustic intermediate representation using speech parameters and the associated speech synthesizer enables our decoding pipeline to produce natural-sounding speech that preserves subject-specific characteristics, which would be lost with the kinematic representation.

Our speech synthesizer is motivated by classical vocoder models for speech production (generating speech by passing an excitation source, harmonic or noise, through a filter 39 , 40 and is fully differentiable, facilitating the training of the ECoG decoder using spectral losses through backpropagation. Furthermore, the guidance speech parameters needed for training the ECoG decoder can be obtained using a speech encoder that can be pre-trained without requiring neural data. Thus, it could be trained using older speech recordings or a proxy speaker chosen by the patient in the case of patients without the ability to speak. Training the ECoG decoder using such guidance, however, would require us to revise our current training strategy to overcome the challenge of misalignment between neural signals and speech signals, which is a scope of our future work. Additionally, the low-dimensional acoustic space and pre-trained speech encoder (for generating the guidance) using speech signals only alleviate the limited data challenge in training the ECoG-to-speech decoder and provide a highly interpretable latent space. Finally, our decoding pipeline is generalizable to unseen words (Fig. 2b ). This provides an advantage compared to the pattern-matching approaches 18 that produce subject-specific utterances but with limited generalizability.

Many earlier studies employed high-density electrode coverage over the cortex, providing many distinct neural signals 5 , 10 , 17 , 30 , 37 . One question we directly addressed was whether higher-density coverage improves decoding. Surprisingly, we found a high decoding performance in terms of spectrogram PCC with both low-density and higher (hybrid) density grid coverages (Fig. 3c ). Furthermore, comparing the decoding performance obtained using all electrodes in our hybrid-density participants versus using only the low-density electrodes in the same participants revealed that the decoding did not differ significantly (albeit for one participant; Fig. 3d ). We attribute these results to the ability of our ECoG decoder to extract speech parameters from neural signals as long as there is sufficient perisylvian coverage, even in low-density participants.

A striking result was the robust decoding from right hemisphere cortical structures as well as the clear contribution of the right perisylvian cortex. Our results are consistent with the idea that syllable-level speech information is represented bilaterally 41 . However, our findings suggest that speech information is well-represented in the right hemisphere. Our decoding results could directly lead to speech prostheses for patients who suffer from expressive aphasia or apraxia of speech. Some previous studies have shown limited right-hemisphere decoding of vowels 42 and sentences 43 . However, the results were mostly mixed with left-hemisphere signals. Although our decoding results provide evidence for a robust representation of speech in the right hemisphere, it is important to note that these regions are likely not critical for speech, as evidenced by the few studies that have probed both hemispheres using electrical stimulation mapping 44 , 45 . Furthermore, it is unclear whether the right hemisphere would contain sufficient information for speech decoding if the left hemisphere were damaged. It would be necessary to collect right-hemisphere neural data from left-hemisphere-damaged patients to verify we can still achieve acceptable speech decoding. However, we believe that right-hemisphere decoding is still an exciting avenue as a clinical target for patients who are unable to speak due to left-hemisphere cortical damage.

There are several limitations in our study. First, our decoding pipeline requires speech training data paired with ECoG recordings, which may not exist for paralysed patients. This could be mitigated by using neural recordings during imagined or mimed speech and the corresponding older speech recordings of the patient or speech by a proxy speaker chosen by the patient. As discussed earlier, we would need to revise our training strategy to overcome the temporal misalignment between the neural signal and the speech signal. Second, our ECoG decoder models (3D ResNet and 3D Swin) assume a grid-based electrode sampling, which may not be the case. Future work should develop model architectures that are capable of handling non-grid data, such as strips and depth electrodes (stereo intracranial electroencephalogram (sEEG)). Importantly, such decoders could replace our current grid-based ECoG decoders while still being trained using our overall pipeline. Finally, our focus in this study was on word-level decoding limited to a vocabulary of 50 words, which may not be directly comparable to sentence-level decoding. Specifically, two recent studies have provided robust speech decoding in a few chronic patients implanted with intracranial ECoG 19 or a Utah array 46 that leveraged a large amount of data available in one patient in each study. It is noteworthy that these studies use a range of approaches in constraining their neural predictions. Metzger and colleagues employed a pre-trained large transformer model leveraging directional attention to provide the guidance HuBERT features for their ECoG decoder. In contrast, Willet and colleagues decoded at the level of phonemes and used transition probability models at both phoneme and word levels to constrain decoding. Our study is much more limited in terms of data. However, we were able to achieve good decoding results across a large cohort of patients through the use of a compact acoustic representation (rather than learnt contextual information). We expect that our approach can help improve generalizability for chronically implanted patients.

To summarize, our neural decoding approach, capable of decoding natural-sounding speech from 48 participants, provides the following major contributions. First, our proposed intermediate representation uses explicit speech parameters and a novel differentiable speech synthesizer, which enables interpretable and acoustically accurate speech decoding. Second, we directly consider the causality of the ECoG decoder, providing strong support for causal decoding, which is essential for real-time BCI applications. Third, our promising decoding results using low sampling density and right-hemisphere electrodes shed light on future neural prosthetic devices using low-density grids and in patients with damage to the left hemisphere. Last, but not least, we have made our decoding framework open to the community with documentation ( https://github.com/flinkerlab/neural_speech_decoding ), and we trust that this open platform will help propel the field forward, supporting reproducible science.

Experiments design

We collected neural data from 48 native English-speaking participants (26 female, 22 male) with refractory epilepsy who had ECoG subdural electrode grids implanted at NYU Langone Hospital. Five participants underwent HB sampling, and 43 LD sampling. The ECoG array was implanted on the left hemisphere for 32 participants and on the right for 16. The Institutional Review Board of NYU Grossman School of Medicine approved all experimental procedures. After consulting with the clinical-care provider, a research team member obtained written and oral consent from each participant. Each participant performed five tasks 47 to produce target words in response to auditory or visual stimuli. The tasks were auditory repetition (AR, repeating auditory words), auditory naming (AN, naming a word based on an auditory definition), sentence completion (SC, completing the last word of an auditory sentence), visual reading (VR, reading aloud written words) and picture naming (PN, naming a word based on a colour drawing).

For each task, we used the exact 50 target words with different stimulus modalities (auditory, visual and so on). Each word appeared once in the AN and SC tasks and twice in the others. The five tasks involved 400 trials, with corresponding word production and ECoG recording for each participant. The average duration of the produced speech in each trial was 500 ms.

Data collection and preprocessing

The study recorded ECoG signals from the perisylvian cortex (including STG, inferior frontal gyrus (IFG), pre-central and postcentral gyri) of 48 participants while they performed five speech tasks. A microphone recorded the subjects’ speech and was synchronized to the clinical Neuroworks Quantum Amplifier (Natus Biomedical), which captured ECoG signals. The ECoG array consisted of 64 standard 8 × 8 macro contacts (10-mm spacing) for 43 participants with low-density sampling. For five participants with hybrid-density sampling, the ECoG array also included 64 additional interspersed smaller electrodes (1 mm) between the macro contacts (providing 10-mm centre-to-centre spacing between macro contacts and 5-mm centre-to-centre spacing between micro/macro contacts; PMT Corporation) (Fig. 3b ). This Food and Drug Administration (FDA)-approved array was manufactured for this study. A research team member informed participants that the additional contacts were for research purposes during consent. Clinical care solely determined the placement location across participants (32 left hemispheres; 16 right hemispheres). The decoding models were trained separately for each participant using all trials except ten randomly selected ones from each task, leading to 350 trials for training and 50 for testing. The reported results are for testing data only.

We sampled ECoG signals from each electrode at 2,048 Hz and downsampled them to 512 Hz before processing. Electrodes with artefacts (for example, line noise, poor contact with the cortex, high-amplitude shifts) were rejected. The electrodes with interictal and epileptiform activity were also excluded from the analysis. The mean of a common average reference (across all remaining valid electrodes and time) was subtracted from each individual electrode. After the subtraction, a Hilbert transform extracted the envelope of the high gamma (70–150 Hz) component from the raw signal, which was then downsampled to 125 Hz. A reference signal was obtained by extracting a silent period of 250 ms before each trial’s stimulus period within the training set and averaging the signals over these silent periods. Each electrode’s signal was normalized to the reference mean and variance (that is, z -score). The data-preprocessing pipeline was coded in MATLAB and Python. For participants with noisy speech recordings, we applied spectral gating to remove stationary noise from the speech using an open-source tool 48 . We ruled out the possibility that our neural data suffer from a recently reported acoustic contamination (Supplementary Fig. 5 ) by following published approaches 49 .

To pre-train the auto-encoder, including the speech encoder and speech synthesizer, unlike our previous work in ref. 29 , which completely relied on unsupervised training, we provided supervision for some speech parameters to improve their estimation accuracy further. Specifically, we used the Praat method 50 to estimate the pitch and four formant frequencies ( ${f}_{ {{{\rm{i}}}} = {1}\,{{{\rm{to}}}}\,4}^{t}$ , in hertz) from the speech waveform. The estimated pitch and formant frequency were resampled to 125 Hz, the same as the ECoG signal and spectrogram sampling frequency. The mean square error between these speech parameters generated by the speech encoder and those estimated by the Praat method was used as a supervised reference loss, in addition to the unsupervised spectrogram reconstruction and STOI losses, making the training of the auto-encoder semi-supervised.

Speech synthesizer

Our speech synthesizer was inspired by the traditional speech vocoder, which generates speech by switching between voiced and unvoiced content, each generated by filtering a specific excitation signal. Instead of switching between the two components, we use a soft mix of the two components, making the speech synthesizer differentiable. This enables us to train the ECoG decoder and the speech encoder end-to-end by minimizing the spectrogram reconstruction loss with backpropagation. Our speech synthesizer can generate a spectrogram from a compact set of speech parameters, enabling training of the ECoG decoder with limited data. As shown in Fig. 5 , the synthesizer takes dynamic speech parameters as input and contains two pathways. The voice pathway applies a set of formant filters (each specified by the centre frequency ${f}_{i}^{\;t}$ , bandwidth ${b}_{i}^{t}$ that is dependent on ${f}_{i}^{\;t}$ , and amplitude ${a}_{i}^{t}$ ) to the harmonic excitation (with pitch frequency f 0 ) and generates the voiced component, V t ( f ), for each time step t and frequency f . The noise pathway filters the input white noise with an unvoice filter (consisting of a broadband filter defined by centre frequency ${f}_{\hat{u}}^{\;t}$ , bandwidth ${b}_{\hat{u}}^{t}$ and amplitude ${a}_{\hat{u}}^{t}$ and the same six formant filters used for the voice filter) and produces the unvoiced content, U t ( f ). The synthesizer combines the two components with a voice weight α t ∈ [0, 1] to obtain the combined spectrogram ${\widetilde{S}}^{t}{(\;f\;)}$ as

Factor α t acts as a soft switch for the gradient to flow back through the synthesizer. The final speech spectrogram is given by

where L t is the loudness modulation and B ( f ) the background noise. We describe the various components in more detail in the following.

Formant filters in the voice pathway

We use multiple formant filters in the voice pathway to model formants that represent vowels and nasal information. The formant filters capture the resonance in the vocal tract, which can help recover a speaker’s timbre characteristics and generate natural-sounding speech. We assume the filter for each formant is time-varying and can be derived from a prototype filter G i ( f ), which achieves maximum at a centre frequency ${f}_{i}^{{{\;{\rm{proto}}}}}$ and has a half-power bandwidth ${b}_{i}^{{{{\rm{proto}}}}}$ . The prototype filters have learnable parameters and will be discussed later. The actual formant filter at any time is written as a shifted and scaled version of G i ( f ). Specifically, at time t , given an amplitude ${\left({a}_{i}^{t}\right)}$ , centre frequency ${\left(\;{f}_{i}^{\;t}\right)}$ and bandwidth ${\left({b}_{i}^{t}\right)}$ , the frequency-domain representation of the i th formant filter is

where f max is half of the speech sampling frequency, which in our case is 8,000 Hz.

Rather than letting the bandwidth parameters ${b}_{i}^{t}$ be independent variables, based on the empirically observed relationships between ${b}_{i}^{t}$ and the centre frequencies ${f}_{i}^{\;t}$ , we set

The threshold frequency f θ , slope a and baseline bandwidth b 0 are three parameters that are learned during the auto-encoder training, shared among all six formant filters. This parameterization helps to reduce the number of speech parameters to be estimated at every time sample, making the representation space more compact.

Finally the filter for the voice pathway with N formant filters is given by ${F}_{{{{\rm{v}}}}}^{\;t}{(\;f\;)}={\mathop{\sum }\nolimits_{i = 1}^{N}{F}_{i}^{\;t}(\;f\;)}$ . Previous studies have shown that two formants ( N = 2) are enough for intelligible reconstruction 51 , but we use N = 6 for more accurate synthesis in our experiments.

Unvoice filters

We construct the unvoice filter by adding a single broadband filter ${F}_{\hat{u}}^{\;t}{(\;f\;)}$ to the formant filters for each time step t . The broadband filter ${F}_{\hat{u}}^{\;t}{(\;f\;)}$ has the same form as equation ( 1 ) but has its own learned prototype filter ${G}_{\hat{u}}{(f)}$ . The speech parameters corresponding to the broadband filter include ${\left({\alpha }_{\hat{u}}^{t},\,{f}_{\hat{u}}^{\;t},\,{b}_{\hat{u}}^{t}\right)}$ . We do not impose a relationship between the centre frequency ${f}_{\hat{u}}^{\;t}$ and the bandwidth ${b}_{\hat{u}}^{t}$ . This allows more flexibility in shaping the broadband unvoice filter. However, we constrain ${b}_{\hat{u}}^{t}$ to be larger than 2,000 Hz to capture the wide spectral range of obstruent phonemes. Instead of using only the broadband filter, we also retain the N formant filters in the voice pathway ${F}_{i}^{\;t}$ for the noise pathway. This is based on the observation that humans perceive consonants such as /p/ and /d/ not only by their initial bursts but also by their subsequent formant transitions until the next vowel 52 . We use identical formant filter parameters to encode these transitions. The overall unvoice filter is ${F}_{{{{\rm{u}}}}}^{\;t}{(\;f\;)}={F}_{\hat{u}}^{\;t}(\;f\;)+\mathop{\sum }\nolimits_{i = 1}^{N}{F}_{i}^{\;t}{(\;f\;)}$ .

Voice excitation

We use the voice filter in the voice pathway to modulate the harmonic excitation. Following ref. 53 , we define the harmonic excitation as ${h}^{t}={\mathop{\sum }\nolimits_{k = 1}^{K}{h}_{k}^{t}}$ , where K = 80 is the number of harmonics.

The value of the k th resonance at time step t is ${h}_{k}^{t}={\sin (2\uppi k{\phi }^{t})}$ with ${\phi }^{t}={\mathop{\sum }\nolimits_{\tau = 0}^{t}{f}_{0}^{\;\tau }}$ , where ${f}_{0}^{\;\tau }$ is the fundamental frequency at time τ . The spectrogram of h t forms the harmonic excitation in the frequency domain H t ( f ), and the voice excitation is ${V}^{\;t}{(\;f\;)}={F}_{{{{\rm{v}}}}}^{t}{(\;f\;)}{H}^{\;t}{(\;f\;)}$ .

Noise excitation

The noise pathway models consonant sounds (plosives and fricatives). It is generated by passing a stationary Gaussian white noise excitation through the unvoice filter. We first generate the noise signal n ( t ) in the time domain by sampling from the Gaussian process ${{{\mathcal{N}}}}{(0,\,1)}$ and then obtain its spectrogram N t ( f ). The spectrogram of the unvoiced component is ${U}^{\;t}{(\;f\;)}={F}_{u}^{\;t}{(\;f\;)}{N}^{\;t}{(\;f\;)}$ .

Summary of speech parameters

The synthesizer generates the voiced component at time t by driving a harmonic excitation with pitch frequency ${f}_{0}^{\;t}$ through N formant filters in the voice pathway, each described by two parameters ( ${f}_{ i}^{\;t},\,{a}_{ i}^{t}$ ). The unvoiced component is generated by filtering a white noise through the unvoice filter consisting of an additional broadband filter with three parameters ( ${f}_{\hat{u}}^{\;t},\,{b}_{\hat{u}}^{t},\,{a}_{\hat{u}}^{t}$ ). The two components are mixed based on the voice weight α t and further amplified by the loudness value L t . In total, the synthesizer input includes 18 speech parameters at each time step.

Unlike the differentiable digital signal processing (DDSP) in ref. 53 , we do not directly assign amplitudes to the K harmonics. Instead, the amplitude in our model depends on the formant filters, which has two benefits:

The representation space is more compact. DDSP requires 80 amplitude parameters ${a}_{k}^{t}$ for each of the 80 harmonic components ${f}_{k}^{\;t}$ ( k = 1, 2, …, 80) at each time step. In contrast, our synthesizer only needs a total of 18 parameters.

The representation is more disentangled. For human speech, the vocal tract shape (affecting the formant filters) is largely independent of the vocal cord tension (which determines the pitch). Modelling these two separately leads to a disentangled representation.

In contrast, DDSP specifies the amplitude for each harmonic component directly resulting in entanglement and redundancy between these amplitudes. Furthermore, it remains uncertain whether the amplitudes ${a}_{k}^{t}$ could be effectively controlled and encoded by the brain. In our approach, we explicitly model the formant filters and fundamental frequency, which possess clear physical interpretations and are likely to be directly controlled by the brain. Our representation also enables a more robust and direct estimation of the pitch.

Speaker-specific synthesizer parameters

Prototype filters.

Instead of using a predetermined prototype formant filter shape, for example, a standard Gaussian function, we learn a speaker-dependent prototype filter for each formant to allow more expressive and flexible formant filter shapes. We define the prototype filter G i ( f ) of the i th formant as a piecewise linear function, linearly interpolated from g i [ m ], m = 1, …, M , with the amplitudes of the filter at M being uniformly sampled frequencies in the range [0, f max ]. We constrain g i [ m ] to increase and then decrease monotonically so that G i ( f ) is unimodal and has a single peak value of 1. Given g i [ m ], m = 1, …, M , we can determine the peak frequency ${f}_{i}^{\;{{{\rm{proto}}}}}$ and the half-power bandwidth ${b}_{i}^{{{{\rm{proto}}}}}$ of G i ( f ).

The prototype parameters g i [ m ], m = 1, …, M of each formant filter are time-invariant and are determined during the auto-encoder training. Compared with ref. 29 , we increase M from 20 to 80 to enable more expressive formant filters, essential for synthesizing male speakers’ voices.

We similarly learn a prototype filter for the broadband filter G û ( f ) for the unvoiced component, which is specified by M parameters g û ( m ).

Background noise

The recorded sound typically contains background noise. We assume that the background noise is stationary and has a specific frequency distribution, depending on the speech recording environment. This frequency distribution B ( f ) is described by K parameters, where K is the number of frequency bins ( K = 256 for females and 512 for males). The K parameters are also learned during auto-encoder training. The background noise is added to the mixed speech components to generate the final speech spectrogram.

To summarize, our speech synthesizer has the following learnable parameters: the M = 80 prototype filter parameters for each of the N = 6 formant filters and the broadband filters (totalling M ( N + 1) = 560), the three parameters f θ , a and b 0 relating the centre frequency and bandwidth for the formant filters (totalling 18), and K parameters for the background noise (256 for female and 512 for male). The total number of parameters for female speakers is 834, and that for male speakers is 1,090. Note that these parameters are speaker-dependent but time-independent, and they can be learned together with the speech encoder during the training of the speech-to-speech auto-encoder, using the speaker’s speech only.

Speech encoder

The speech encoder extracts a set of (18) speech parameters at each time point from a given spectrogram, which are then fed to the speech synthesizer to reproduce the spectrogram.

We use a simple network architecture for the speech encoder, with temporal convolutional layers and multilayer perceptron (MLP) across channels at the same time point, as shown in Fig. 6a . We encode pitch ${f}_{0}^{\;t}$ by combining features generated from linear and Mel-scale spectrograms. The other 17 speech parameters are derived by applying temporal convolutional layers and channel MLP to the linear-scale spectrogram. To generate formant filter centre frequencies ${f}_{i = 1\,{{{\rm{to}}}}\,6}^{\;t}$ , broadband unvoice filter frequency ${f}_{\hat{u}}^{\;t}$ and pitch ${f}_{0}^{\;t}$ , we use sigmoid activation at the end of the corresponding channel MLP to map the output to [0, 1], and then de-normalize it to real values by scaling [0, 1] to predefined [ f min , f max ]. The [ f min , f max ] values for each frequency parameter are chosen based on previous studies 54 , 55 , 56 , 57 . Our compact speech parameter space facilitates stable and easy training of our speech encoder. Models were coded using PyTorch version 1.21.1 in Python.

ECoG decoder

In this section we present the design details of three ECoG decoders: the 3D ResNet ECoG decoder, the 3D Swin transformer ECoG decoder and the LSTM ECoG decoder. The models were coded using PyTorch version 1.21.1 in Python.

3D ResNet ECoG decoder

This decoder adopts the ResNet architecture 23 for the feature extraction backbone of the decoder. Figure 6c illustrates the feature extraction part. The model views the ECoG input as 3D tensors with spatiotemporal dimensions. In the first layer, we apply only temporal convolution to the signal from each electrode, because the ECoG signal exhibits more temporal than spatial correlations. In the subsequent parts of the decoder, we have four residual blocks that extract spatiotemporal features using 3D convolution. After downsampling the electrode dimension to 1 × 1 and the temporal dimension to T /16, we use several transposed Conv layers to upsample the features to the original temporal size T . Figure 6b shows how to generate the different speech parameters from the resulting features using different temporal convolution and channel MLP layers. The temporal convolution operation can be causal (that is, using only past and current samples as input) or non-causal (that is, using past, current and future samples), leading to causal and non-causal models.

3D Swin Transformer ECoG decoder

Swin Transformer 24 employs the window and shift window methods to enable self-attention of small patches within each window. This reduces the computational complexity and introduces the inductive bias of locality. Because our ECoG input data have three dimensions, we extend Swin Transformer to three dimensions to enable local self-attention in both temporal and spatial dimensions among 3D patches. The local attention within each window gradually becomes global attention as the model merges neighbouring patches in deeper transformer stages.

Figure 6d illustrates the overall architecture of the proposed 3D Swin Transformer. The input ECoG signal has a size of T × H × W , where T is the number of frames and H × W is the number of electrodes at each frame. We treat each 3D patch of size 2 × 2 × 2 as a token in the 3D Swin Transformer. The 3D patch partitioning layer produces ${\frac{T}{2}\times \frac{H}{2}\times \frac{W}{2}}$ 3D tokens, each with a 48-dimensional feature. A linear embedding layer then projects the features of each token to a higher dimension C (=128).

The 3D Swin Transformer comprises three stages with two, two and six layers, respectively, for LD participants and four stages with two, two, six and two layers for HB participants. It performs 2 × 2 × 2 spatial and temporal downsampling in the patch-merging layer of each stage. The patch-merging layer concatenates the features of each group of 2 × 2 × 2 temporally and spatially adjacent tokens. It applies a linear layer to project the concatenated features to one-quarter of their original dimension after merging. In the 3D Swin Transformer block, we replace the multi-head self-attention (MSA) module in the original Swin Transformer with the 3D shifted window multi-head self-attention module. It adapts the other components to 3D operations as well. A Swin Transformer block consists of a 3D shifted window-based MSA module followed by a feedforward network (FFN), a two-layer MLP. Layer normalization is applied before each MSA module and FFN, and a residual connection is applied after each module.

Consider a stage with T × H × W input tokens. If the 3D window size is P × M × M , we partition the input into ${\lceil \frac{T}{P}\rceil \times \lceil \frac{H}{M}\rceil \times \lceil \frac{W}{M}\rceil}$ non-overlapping 3D windows evenly. We choose P = 16, M = 2. We perform the multi-head self-attention within each 3D window. However, this design lacks connection across adjacent windows, which may limit the representation power of the architecture. Therefore, we extend the shifted 2D window mechanism of the Swin Transformer to shifted 3D windows. In the second layer of the stage, we shift the window by $\left({\frac{P}{2},\,\frac{M}{2},\,\frac{M}{2}}\right)$ tokens along the temporal, height and width axes from the previous layer. This creates cross-window connections for the self-attention module. This shifted 3D window design enables the interaction of electrodes with longer spatial and temporal distances by connecting neighbouring tokens in non-overlapping 3D windows in the previous layer.

The temporal attention in the self-attention operation can be constrained to be causal (that is, each token only attends to tokens temporally before it) or non-causal (that is, each token can attend to tokens temporally before or after it), leading to the causal and non-causal models, respectively.

LSTM decoder

The decoder uses the LSTM architecture 25 for the feature extraction in Fig. 6e . Each LSTM cell is composed of a set of gates that control the flow of information: the input gate, the forget gate and the output gate. The input gate regulates the entry of new data into the cell state, the forget gate decides what information is discarded from the cell state, and the output gate determines what information is transferred to the next hidden state and can be output from the cell.

In the LSTM architecture, the ECoG input would be processed through these cells sequentially. For each time step T , the LSTM would take the current input x t and the previous hidden state h t − 1 and would produce a new hidden state h t and output y t . This process allows the LSTM to maintain information over time and is particularly useful for tasks such as speech and neural signal processing, where temporal dependencies are critical. Here we use three layers of LSTM and one linear layer to generate features to map to speech parameters. Unlike 3D ResNet and 3D Swin, we keep the temporal dimension unchanged across all layers.

Model training

Training of the speech encoder and speech synthesizer.

As described earlier, we pre-train the speech encoder and the learnable parameters in the speech synthesizer to perform a speech-to-speech auto-encoding task. We use multiple loss terms for the training. The modified multi-scale spectral (MSS) loss is inspired by ref. 53 and is defined as

Here, S t ( f ) denotes the ground-truth spectrogram and ${\widehat{S}}^{t}{(\;f\;)}$ the reconstructed spectrogram in the linear scale, ${S}_{{{{\rm{mel}}}}}^{t}{(\;f\;)}$ and ${\widehat{S}}_{{{{\rm{mel}}}}}^{t}{(\;f\;)}$ are the corresponding spectrograms in the Mel-frequency scale. We sample the frequency range [0, 8,000 Hz] with K = 256 bins for female participants. For male participants, we set K = 512 because they have lower f 0 , and it is better to have a higher resolution in frequency.

To improve the intelligibility of the reconstructed speech, we also introduce the STOI loss by implementing the STOI+ metric 26 , which is a variation of the original STOI metric 8 , 22 . STOI+ 26 discards the normalization and clipping step in STOI and has been shown to perform best among intelligibility evaluation metrics. First, a one-third octave band analysis 22 is performed by grouping Discrete Fourier transform (DFT) bins into 15 one-third octave bands with the lowest centre frequency set equal to 150 Hz and the highest centre frequency equal to ~4.3 kHz. Let ${\hat{x}(k,\,m)}$ denote the k th DFT bin of the m th frame of the ground-truth speech. The norm of the j th one-third octave band, referred to as a time-frequency (TF) unit, is then defined as

where k 1 ( j ) and k 2 ( j ) denote the one-third octave band edges rounded to the nearest DFT bin. The TF representation of the processed speech ${\hat{y}}$ is obtained similarly and denoted by Y j ( m ). We then extract the short-time temporal envelopes in each band and frame, denoted X j , m and Y j , m , where ${X}_{j,\,m}={\left[{X}_{j}{(m-N+1)},\,{X}_{j}{(m-N+2)},\,\ldots ,\,{X}_{j}{(m)}\right]}^{\rm{T}}$ , with N = 30. The STOI+ metric is the average of the PCC d j , m between X j , m and Y j , m , overall j and m (ref. 26 ):

We use the negative of the STOI+ metric as the STOI loss:

where J and M are the total numbers of frequency bins ( J = 15) and frames, respectively. Note that L STOI is differentiable with respect to ${\widehat{S}}^{t}{(\;f\;)}$ , and thus can be used to update the model parameters generating the predicted spectrogram ${\widehat{S}}^{t}{(\;f\;)}$ .

To further improve the accuracy for estimating the pitch ${\widetilde{f}}_{0}^{\;t}$ and formant frequencies ${\widetilde{f}}_{{{{\rm{i}}}} = {1}\,{{{\rm{to}}}}\,4}^{\;t}$ , we add supervisions to them using the formant frequencies extracted by the Praat method 50 . The supervision loss is defined as

where the weights β i are chosen to be β 1 = 0.1, β 2 = 0.06, β 3 = 0.03 and β 4 = 0.02, based on empirical trials. The overall training loss is defined as

where the weighting parameters λ i are empirically optimized to be λ 1 = 1.2 and λ 2 = 0.1 through testing the performances on three hybrid-density participants with different parameter choices.

Training of the ECoG decoder

With the reference speech parameters generated by the speech encoder and the target speech spectrograms as ground truth, the ECoG decoder is trained to match these targets. Let us denote the decoded speech parameters as ${\widetilde{C}}_{j}^{\;t}$ , and their references as ${C}_{j}^{\;t}$ , where j enumerates all speech parameters fed to the speech synthesizer. We define the reference loss as

where weighting parameters λ j are chosen as follows: voice weight λ α = 1.8, loudness λ L = 1.5, pitch ${\lambda }_{{f}_{0}}={0.4}$ , formant frequencies ${\lambda }_{{f}_{1}}={3},\,{\lambda }_{{f}_{2}}={1.8},\,{\lambda }_{{f}_{3}}={1.2},\,{\lambda }_{{f}_{4}}={0.9},\,{\lambda }_{{f}_{5}}={0.6},\,{\lambda }_{{f}_{6}}={0.3}$ , formant amplitudes ${\lambda }_{{a}_{1}}={4},\,{\lambda }_{{a}_{2}}={2.4},\,{\lambda }_{{a}_{3}}={1.2},\,{\lambda }_{{a}_{4}}={0.9},\,{\lambda }_{{a}_{5}}={0.6},\,{\lambda }_{{a}_{6}}={0.3}$ , broadband filter frequency ${\lambda }_{{f}_{\hat{u}}}={10}$ , amplitude ${\lambda }_{{a}_{\hat{u}}}={4}$ , bandwidth ${\lambda }_{{b}_{\hat{u}}}={4}$ . Similar to speech-to-speech auto-encoding, we add supervision loss for pitch and formant frequencies derived by the Praat method and use the MSS and STOI loss to measure the difference between the reconstructed spectrograms and the ground-truth spectrogram. The overall training loss for the ECoG decoder is

where weighting parameters λ i are empirically optimized to be λ 1 = 1.2, λ 2 = 0.1 and λ 3 = 1, through the same parameter search process as described for training the speech encoder.

We use the Adam optimizer 58 with hyper-parameters lr = 10 −3 , β 1 = 0.9 and β 2 = 0.999 to train both the auto-encoder (including the speech encoder and speech synthesizer) and the ECoG decoder. We train a separate set of models for each participant. As mentioned earlier, we randomly selected 50 out of 400 trials per participant as the test data and used the rest for training.

Evaluation metrics

In this Article, we use the PCC between the decoded spectrogram and the actual speech spectrogram to evaluate the objective quality of the decoded speech, similar to refs. 8 , 18 , 59 .

We also use STOI+ 26 , as described in Methods section Training of the ECoG decoder to measure the intelligibility of the decoded speech. The STOI+ value ranges from −1 to 1 and has been reported to have a monotonic relationship with speech intelligibility.

Contribution analysis with the occlusion method

To measure the contribution of the cortex region under each electrode to the decoding performance, we adopted an occlusion-based method that calculates the change in the PCC between the decoded and the ground-truth spectrograms when an electrode signal is occluded (that is, set to zeros), as in ref. 29 . This method enables us to reveal the critical brain regions for speech production. We used the following notations: S t ( f ), the ground-truth spectrogram; ${\hat{{{{{S}}}}}}^{t}{(\;f\;)}$ , the decoded spectrogram with ‘intact’ input (that is, all ECoG signals are used); ${\hat{{{{{S}}}}}}_{i}^{t}{(\;f\;)}$ , the decoded spectrogram with the i th ECoG electrode signal occluded; r ( ⋅ , ⋅ ), correlation coefficient between two signals. The contribution of i th electrode for a particular participant is defined as

where Mean{ ⋅ } denotes averaging across all testing trials of the participant.

We generate the contribution map on the standardized Montreal Neurological Institute (MNI) brain anatomical map by diffusing the contribution of each electrode of each participant (with a corresponding location in the MNI coordinate) into the adjacent area within the same anatomical region using a Gaussian kernel and then averaging the resulting map from all participants. To account for the non-uniform density of the electrodes in different regions and across the participants, we normalize the sum of the diffused contribution from all the electrodes at each brain location by the total number of electrodes in the region across all participants.

We estimate the noise level for the contribution map to assess the significance of our contribution analysis. To derive the noise level, we train a shuffled model for each participant by randomly pairing the mismatched speech segment and ECoG segment in the training set. We derive the average contribution map from the shuffled models for all participants using the same occlusion analysis as described earlier. The resulting contribution map is used as the noise level. Contribution levels below the noise levels at corresponding cortex locations are assigned a value of 0 (white) in Fig. 4 .

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this Article.

Data availability

The data of one participant who consented to the release of the neural and audio data are publicly available through Mendeley Data at https://data.mendeley.com/datasets/fp4bv9gtwk/2 (ref. 60 ). Although all participants consented to share their data for research purposes, not all participants agreed to share their audio publicly. Given the sensitive nature of audio speech data we will share data with researchers that directly contact the corresponding author and provide documentation that the data will be strictly used for research purposes and will comply with the terms of our study IRB. Source data are provided with this paper.

Code availability

The code is available at https://github.com/flinkerlab/neural_speech_decoding ( https://doi.org/10.5281/zenodo.10719428 ) 61 .

Schultz, T. et al. Biosignal-based spoken communication: a survey. IEEE / ACM Trans. Audio Speech Lang. Process. 25 , 2257–2271 (2017).

Google Scholar

Miller, K. J., Hermes, D. & Staff, N. P. The current state of electrocorticography-based brain-computer interfaces. Neurosurg. Focus 49 , E2 (2020).

Article Google Scholar

Luo, S., Rabbani, Q. & Crone, N. E. Brain-computer interface: applications to speech decoding and synthesis to augment communication. Neurotherapeutics 19 , 263–273 (2022).

Moses, D. A., Leonard, M. K., Makin, J. G. & Chang, E. F. Real-time decoding of question-and-answer speech dialogue using human cortical activity. Nat. Commun. 10 , 3096 (2019).

Moses, D. A. et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria. N. Engl. J. Med. 385 , 217–227 (2021).

Herff, C. & Schultz, T. Automatic speech recognition from neural signals: a focused review. Front. Neurosci. 10 , 429 (2016).

Rabbani, Q., Milsap, G. & Crone, N. E. The potential for a speech brain-computer interface using chronic electrocorticography. Neurotherapeutics 16 , 144–165 (2019).

Angrick, M. et al. Speech synthesis from ECoG using densely connected 3D convolutional neural networks. J. Neural Eng. 16 , 036019 (2019).

Sun, P., Anumanchipalli, G. K. & Chang, E. F. Brain2Char: a deep architecture for decoding text from brain recordings. J. Neural Eng. 17 , 066015 (2020).

Makin, J. G., Moses, D. A. & Chang, E. F. Machine translation of cortical activity to text with an encoder–decoder framework. Nat. Neurosci. 23 , 575–582 (2020).

Wang, R. et al. Stimulus speech decoding from human cortex with generative adversarial network transfer learning. In Proc. 2020 IEEE 17th International Symposium on Biomedical Imaging ( ISBI ) (ed. Amini, A.) 390–394 (IEEE, 2020).

Zelinka, P., Sigmund, M. & Schimmel, J. Impact of vocal effort variability on automatic speech recognition. Speech Commun. 54 , 732–742 (2012).

Benzeghiba, M. et al. Automatic speech recognition and speech variability: a review. Speech Commun. 49 , 763–786 (2007).

Martin, S. et al. Decoding spectrotemporal features of overt and covert speech from the human cortex. Front. Neuroeng. 7 , 14 (2014).

Herff, C. et al. Towards direct speech synthesis from ECoG: a pilot study. In Proc. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society ( EMBC ) (ed. Patton, J.) 1540–1543 (IEEE, 2016).

Angrick, M. et al. Real-time synthesis of imagined speech processes from minimally invasive recordings of neural activity. Commun. Biol 4 , 1055 (2021).

Anumanchipalli, G. K., Chartier, J. & Chang, E. F. Speech synthesis from neural decoding of spoken sentences. Nature 568 , 493–498 (2019).

Herff, C. et al. Generating natural, intelligible speech from brain activity in motor, premotor and inferior frontal cortices. Front. Neurosci. 13 , 1267 (2019).

Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature 620 , 1037–1046 (2023).

Hsu, W.-N. et al. Hubert: self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Trans. Audio Speech Lang. Process. 29 , 3451–3460 (2021).

Griffin, D. & Lim, J. Signal estimation from modified short-time fourier transform. IEEE Trans. Acoustics Speech Signal Process. 32 , 236–243 (1984).

Taal, C. H., Hendriks, R. C., Heusdens, R. & Jensen, J. A short-time objective intelligibility measure for time-frequency weighted noisy speech. In Proc. 2010 IEEE International Conference on Acoustics, Speech and Signal Processing (ed. Douglas, S.) 4214–4217 (IEEE, 2010).

He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ) (ed. Bajcsy, R.) 770–778 (IEEE, 2016).

Liu, Z. et al. Swin Transformer: hierarchical vision transformer using shifted windows. In Proc. 2021 IEEE / CVF International Conference on Computer Vision ( ICCV ) (ed. Dickinson, S.) 9992–10002 (IEEE, 2021).

Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9 , 1735–1780 (1997).

Graetzer, S. & Hopkins, C. Intelligibility prediction for speech mixed with white Gaussian noise at low signal-to-noise ratios. J. Acoust. Soc. Am. 149 , 1346–1362 (2021).

Hickok, G. & Poeppel, D. The cortical organization of speech processing. Nat. Rev. Neurosci. 8 , 393–402 (2007).

Trupe, L. A. et al. Chronic apraxia of speech and Broca’s area. Stroke 44 , 740–744 (2013).

Wang, R. et al. Distributed feedforward and feedback cortical processing supports human speech production. Proc. Natl Acad. Sci. USA 120 , e2300255120 (2023).

Mugler, E. M. et al. Differential representation ofÿ articulatory gestures and phonemes in precentral and inferior frontal gyri. J. Neurosci. 38 , 9803–9813 (2018).

Herff, C. et al. Brain-to-text: decoding spoken phrases from phone representations in the brain. Front. Neurosci. 9 , 217 (2015).

Kohler, J. et al. Synthesizing speech from intracranial depth electrodes using an encoder-decoder framework. Neurons Behav. Data Anal. Theory https://doi.org/10.51628/001c.57524 (2022).

Angrick, M. et al. Towards closed-loop speech synthesis from stereotactic EEG: a unit selection approach. In Proc. 2022 IEEE International Conference on Acoustics , Speech and Signal Processing ( ICASSP ) (ed. Li, H.) 1296–1300 (IEEE, 2022).

Ozker, M., Doyle, W., Devinsky, O. & Flinker, A. A cortical network processes auditory error signals during human speech production to maintain fluency. PLoS Biol. 20 , e3001493 (2022).

Stuart, A., Kalinowski, J., Rastatter, M. P. & Lynch, K. Effect of delayed auditory feedback on normal speakers at two speech rates. J. Acoust. Soc. Am. 111 , 2237–2241 (2002).

Verwoert, M. et al. Dataset of speech production in intracranial electroencephalography. Sci. Data 9 , 434 (2022).

Berezutskaya, J. et al. Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models. J. Neural Eng. 20 , 056010 (2023).

Wang, R., Wang, Y. & Flinker, A. Reconstructing speech stimuli from human auditory cortex activity using a WaveNet approach. In Proc. 2018 IEEE Signal Processing in Medicine and Biology Symposium ( SPMB ) (ed. Picone, J.) 1–6 (IEEE, 2018).

Flanagan, J. L. Speech Analysis Synthesis and Perception Vol. 3 (Springer, 2013).

Serra, X. & Smith, J. Spectral modeling synthesis: a sound analysis/synthesis system based on a deterministic plus stochastic decomposition. Comput. Music J. 14 , 12–24 (1990).

Cogan, G. B. et al. Sensory–motor transformations for speech occur bilaterally. Nature 507 , 94–98 (2014).

Ibayashi, K. et al. Decoding speech with integrated hybrid signals recorded from the human ventral motor cortex. Front. Neurosci. 12 , 221 (2018).

Soroush, P. Z. et al. The nested hierarchy of overt, mouthed and imagined speech activity evident in intracranial recordings. NeuroImage 269 , 119913 (2023).

Tate, M. C., Herbet, G., Moritz-Gasser, S., Tate, J. E. & Duffau, H. Probabilistic map of critical functional regions of the human cerebral cortex: Broca’s area revisited. Brain 137 , 2773–2782 (2014).

Long, M. A. et al. Functional segregation of cortical regions underlying speech timing and articulation. Neuron 89 , 1187–1193 (2016).

Willett, F. R. et al. A high-performance speech neuroprosthesis. Nature 620 , 1031–1036 (2023).

Shum, J. et al. Neural correlates of sign language production revealed by electrocorticography. Neurology 95 , e2880–e2889 (2020).

Sainburg, T., Thielk, M. & Gentner, T. Q. Finding, visualizing and quantifying latent structure across diverse animal vocal repertoires. PLoS Comput. Biol. 16 , e1008228 (2020).

Roussel, P. et al. Observation and assessment of acoustic contamination of electrophysiological brain signals during speech production and sound perception. J. Neural Eng. 17 , 056028 (2020).

Boersma, P. & Van Heuven, V. Speak and unSpeak with PRAAT. Glot Int. 5 , 341–347 (2001).

Chang, E. F., Raygor, K. P. & Berger, M. S. Contemporary model of language organization: an overview for neurosurgeons. J. Neurosurgery 122 , 250–261 (2015).

Jiang, J., Chen, M. & Alwan, A. On the perception of voicing in syllable-initial plosives in noise. J. Acoust. Soc. Am. 119 , 1092–1105 (2006).

Engel, J., Hantrakul, L., Gu, C. & Roberts, A. DDSP: differentiable digital signal processing. In Proc. 8th International Conference on Learning Representations https://openreview.net/forum?id=B1x1ma4tDr (Open.Review.net, 2020).

Flanagan, J. L. A difference limen for vowel formant frequency. J. Acoust. Soc. Am. 27 , 613–617 (1955).

Schafer, R. W. & Rabiner, L. R. System for automatic formant analysis of voiced speech. J. Acoust. Soc. Am. 47 , 634–648 (1970).

Fitch, J. L. & Holbrook, A. Modal vocal fundamental frequency of young adults. Arch. Otolaryngol. 92 , 379–382 (1970).

Stevens, S. S. & Volkmann, J. The relation of pitch to frequency: a revised scale. Am. J. Psychol. 53 , 329–353 (1940).

Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) http://arxiv.org/abs/1412.6980 (arXiv, 2015).

Angrick, M. et al. Interpretation of convolutional neural networks for speech spectrogram regression from intracranial recordings. Neurocomputing 342 , 145–151 (2019).

Chen, X. ECoG_HB_02. Mendeley data, V2 (Mendeley, 2024); https://doi.org/10.17632/fp4bv9gtwk.2

Chen, X. & Wang, R. Neural speech decoding 1.0 (Zenodo, 2024); https://doi.org/10.5281/zenodo.10719428

Download references

Acknowledgements

This Work was supported by the National Science Foundation under grants IIS-1912286 and 2309057 (Y.W. and A.F.) and National Institute of Health grants R01NS109367, R01NS115929 and R01DC018805 (A.F.).

Author information

These authors contributed equally: Xupeng Chen, Ran Wang.

These authors jointly supervised this work: Yao Wang, Adeen Flinker.

Authors and Affiliations

Electrical and Computer Engineering Department, New York University, Brooklyn, NY, USA

Xupeng Chen, Ran Wang & Yao Wang

Neurology Department, New York University, Manhattan, NY, USA

Amirhossein Khalilian-Gourtani, Leyao Yu, Patricia Dugan, Daniel Friedman, Orrin Devinsky & Adeen Flinker

Biomedical Engineering Department, New York University, Brooklyn, NY, USA

Leyao Yu, Yao Wang & Adeen Flinker

Neurosurgery Department, New York University, Manhattan, NY, USA

Werner Doyle

You can also search for this author in PubMed Google Scholar

Contributions

Y.W. and A.F. supervised the research. X.C., R.W., Y.W. and A.F. conceived research. X.C., R.W., A.K.-G., L.Y., P.D., D.F., W.D., O.D. and A.F. performed research. X.C., R.W., Y.W. and A.F. contributed new reagents/analytic tools. X.C., R.W., A.K.-G., L.Y. and A.F. analysed data. P.D. and D.F. provided clinical care. W.D. provided neurosurgical clinical care. O.D. assisted with patient care and consent. X.C., Y.W. and A.F. wrote the paper.

Corresponding author

Correspondence to Adeen Flinker .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Machine Intelligence thanks the anonymous reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Figs. 1–10, Table 1 and audio files list.

Reporting Summary

Supplementary audio 1.

Example original and decoded audios for eight words.

Supplementary Audio 2

Example original and decoded words from low density participants.

Supplementary Audio 3

Example original and decoded words from hybrid density participants.

Supplementary Audio 4

Example original and decoded words from left hemisphere low density participants.

Supplementary Audio 5

Example original and decoded words from right hemisphere low density participants.

Source Data Fig. 2

Data for Fig, 2a,b,d,e,f.

Source Data Fig. 3

Data for Fig, 3a,c,d.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Chen, X., Wang, R., Khalilian-Gourtani, A. et al. A neural speech decoding framework leveraging deep learning and speech synthesis. Nat Mach Intell (2024). https://doi.org/10.1038/s42256-024-00824-8

Download citation

Received : 29 July 2023

Accepted : 08 March 2024

Published : 08 April 2024

DOI : https://doi.org/10.1038/s42256-024-00824-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Explore articles by subject
Guide to authors
Editorial policies

IMAGES

Free text to speech software with natural voices
Free Text to Speech Online with Natural Voices
Text to speech natural human voice natural reader with best ai voice
Free And Paid Natural Voice Text To Speech Software
The Best Guide to Text To Speech Mp3 With Natural Voices
2023 Top 5 Natural Text-to-Speech Online and PC

VIDEO

💥 Text To Speech 🍉 ASMR Cake Storytime || @Brianna Guidryy || POVs Tiktok Compilations 2023 #104
NaturalReader Mobile
Convert Text to Speech with AI Voiceovers
#1 AI Text to Speech App
How to speak in natural voice
Convert Text To Speech using Natural Reader commercial version # By Internet World

COMMENTS

Free Text to Speech Online with Realistic AI Voices
Text to speech (TTS) is a technology that converts text into spoken audio. It can read aloud PDFs, websites, and books using natural AI voices. Text-to-speech (TTS) technology can be helpful for anyone who needs to access written content in an auditory format, and it can provide a more inclusive and accessible way of communication for many ...
Speechise: Free Text to Speech Online with Natural Voices
Natural-Sounding Voices. The AI Text-to-Speech (TTS) technology powers our free reader with high-quality voices so you can enjoy the timeless advantages of listening. Do More with Your Time. With our app, you can get through documents, articles, PDFs, and emails effortlessly, freeing your hands and eyes. ...
Realistic Text to Speech converter & AI Voice generator
Just type or paste your text, generate the voice-over, and download the audio file. Create realistic Voiceovers online! Insert any text to generate speech and download audio mp3 or wav for any purpose. Speak a text with AI-powered voices.You can convert text to voice for free for reference only. For all features, purchase the paid plans.
Text to Speech Online with Natural Voices
Text to Speech Online with Realistic Voices. Convert your text to +100 natural sounding voices. Free MP3 Download and Audio hosting with HTML embed audio player. Text-to-Speech API. Read any website aloud.
Free Text to Speech Online with Natural Voices
From Text to Speech, NaturalVoicer is the Best. If you need to generate voices for YouTube, Vimeo, PPT or a website, or simply have difficulty reading large amounts of text, you should definitely consider Naturalvoicer. Fast, easy, and natural-sounding, it's the perfect online voice generator.
Free Text to Speech Online with 120+ Realistic TTS Voices
Murf: The Ultimate Text to Speech Software. If you are looking for a text to speech generator that can create stunning voiceovers for your tutorials, presentations, or videos, Murf is the one to go for. Murf can generate human-like, realistic, and natural-sounding voices. Its pièce de résistance is that Murf can do it in over 120+ unique ...
Text-to-Speech AI: Lifelike Speech Synthesis
Turn text into natural-sounding speech in 220+ voices across 40+ languages and variants with an API powered by Google's machine learning technology.
Free Text to Speech Online Service with Natural Voices
Free Text to Speech Online Service with Natural Voices. Hello, I'm one of the voices you can use to speech enable content, devices, applications and more. When I read your text, it sounds like this. Please note that the maximum number of characters is 10000. Vocalize.
#1 Text To Speech (TTS) Reader Online. Free & Unlimited
#1 Text To Speech. Type or upload any text, file, website & book for listening online, proofreading, reading-along or generating professional mp3 voice-overs. ... Murf is a text-to-speech tool offering 200+ natural voices for creating high-quality voiceovers for e-learning, podcasts, YouTubes & audiobooks, simplifying audio content production.
TTSReader
Free. Text To Speech Reader. Instantly reads out loud text & PDF with natural sounding voices. Online - works out of the box. Drop the text and click play. Drag text or pdf files to the text-box, or directly type/paste in text. Select language and click Play. Remembers text and caret position between sessions.
Free AI Text To Speech Online
High quality free text to speech online. Use AI text to speech to create realistic AI voices for games, videos, podcasts, and more for free. 0:00 / 0:00. ElevenLabs ll Eleven Labs. ... Wall of Text to Speech Voices From natural conversational tones to professional narrations, we have a voice for every context. Grace (Professionally Cloned ...
AI Voice Generator & Text to Speech
Rated the best text to speech (TTS) software online. Create premium AI voices for free and generate text-to-speech voiceovers in minutes with our character AI voice generator. Use free text to speech AI to convert text to mp3 in 29 languages with 100+ voices.
Text to Voice Over Generator
Transform your text to voice over. Now just paste your text into the field on the right side and select your preferred language from the drop-down menu. Then, choose the best voice to charm your audience. You can even listen to different voices with the Preview option until you find the perfect fit.
TextToSpeech.io
TextToSpeech.io - Free online Text to Speech reader. TextToSpeech.io is a Free online Text To Speech Reader service. Accurate with natural voices, multilingual. Real time. Free & always will be. The TTS reader is available again for Guest users with limitations. Please check our FAQs for more details. You can register an account to get more ...
Text To Speech: #1 Free TTS Online With Realistic AI Voices
What is text to speech. Text to speech, also known as TTS, read aloud, or even speech synthesis.It simply means using artificial intelligence to read words aloud be; it from a PDF, email, docs, or any website.There isn't a voice artist recording phrases or words, or even the entire article.
Free Text to Speech Online
Free Online text to speech with 225+ natural sounding voices. Download your files as mp3🎧 or WAV. Create stunning audio files for personal and business purposes. ... High-fidelity speech synthesis is the process of generating natural-sounding speech from text. This technology has a wide range of applications, from helping visually-impaired ...
Realistic Voice AI
Interactive Voice-based Applications. Integrating text-to-speech into voice assistants or chatbots, allowing users to interact via spoken commands and responses. Language Learning and Pronunciation. Assisting language learners in improving pronunciation skills by providing accurate and natural voice models. E-Learning Courses.
Free text to speech online
Our tool can read text in over 50 languages and even offers multiple text-to-speech voices for a few widely spoken languages such as English. Step #1: Write or paste your text in the input box. You also have the option of uploading a txt file. Step #2: Choose your desired language and speaker. You can try out different speakers if there are ...
AI Voice Generator: Free Text to Speech Online
Engage your audience with the perfect voice you can create with the free AI voice generator. Upload your script and choose from over 120 AI voices in 20+ languages, including Spanish, Chinese, and French. Infuse a human element by customizing the voice's speed, pitch, emotion, and tonality. Seamlessly add a voice to any Canva video, design ...
Free Text to Speech Online
TTSMaker is a free text-to-speech tool and an online text reader that can convert text to speech, it supports 100+ languages and 100+ voice styles, powerful neural network makes speech sound more natural, you can listen online, or download audio files in mp3, wav format.
Best Online Text to Speech Generators in 2024
Snapshot of Our Top 3 Online Text to Speech Generators. 1. MyEdit - Best Overall Text to Speech Generator. MyEdit stands out as a powerful AI-driven text-to-speech generator that caters to various needs, including automatic transcription of lectures, interviews, and content creation. Supporting an impressive array of languages such as English ...
Free Text-to-Speech Online with Natural AI Voices
ON4T's text-to-speech tool has lots of AI voices just for you. It's like having a voice actor, but faster and cheaper. You can make your own special voice-overs easily. It's great for lots of videos and saves you time and money. With our tool, making fun and interesting videos is easy and quick!
AI Speech To Text: Revolutionizing Transcription
Speech to Text, often abbreviated as speech-to-text, refers to the technology used to transcribe spoken language into written text. This can be applied to various audio sources, such as video files, podcasts, and even real-time conversations. Thanks to advancements in machine learning and natural language processing, today's speech ...
A neural speech decoding framework leveraging deep learning and speech
The upper part shows the ECoG-to-speech decoding pipeline. The ECoG decoder generates time-varying speech parameters from ECoG signals. The speech synthesizer generates spectrograms from the ...