Disclaimer

YES, I do agree with you. AI generated content is bad. I don’t believe that it has a place, especially when it’s competing with independent creators. However, I was bored and wanted to see if I could.

Introducing the PlantMatters Podcast!

— Not actually being released —

I created an AI animated podcast of two characters gossiping over the latest headlines. Silly? Definitely. Useful? Probably not.

Here are the steps I used to get it to work:

1: Get the news

I used the NYT rss feed to get the latest headlines, which were then interpreted in python into a format better fit for a natural language model.

2: Generate the script

The headlines then needed to be turned into a script for our two gossiping characters. This was the easiest! With a simple prompt and an LLM Api I could generate a usable and parsable script every time.

3: Generate the audio

This was a little more complicated. Because google hasn’t released their NotebookLM-style two-speaker text-to-speech generator, I had to create my own solution. This involves creating separate audio each time the script changes speaker, and then stitching all of the audio together.

4: Grab the subtitles

Again, pretty easy. With google’s speech-to-text generator I could easily get a timed transcript of the entire conversation. Then, all I had to do was convert it into a .srt, and then a .ass for use as hard-subtitles during video playback.

5: Animate characters to the audio

This was easily the hardest step. There aren’t many tools I found for creating animations from only audio, and almost of all of them were prohibitively expensive for a side project, but Adobe Express’s animate-from-audio tool was free!

The first issue is that Adobe only animates a single character. This was simple enough, I split the audio in half with one audio file for character A’s dialogue and another file for character B’s. Then I could use these separate dialogues to create two separate videos.

This is where it got sillier. Because there is no Api for the animate-from-audio tool, I had to write code to manipulate the website in a way that signed me in, uploaded my generated audio, and downloaded the final mp4 once it was finished.

6: Assembling the animated videos into the final product

There were a few steps to this, all of which required ffmpeg.

1. Split each video in half, and merge them together, resulting in an mp4 with both characters in it
2. Replace the new mp4’s audio with the original audio generated in step 3
3. Add in the subtitles generated in step 4
4. Overlay the podcast banner image at the top of the video

7: You’re done!!

You’ve just generated your own AI animated video in less than 8 minutes of runtime, and costing less than $1. Pretty fun!

Can I view the code?

Ha!

No.

I don’t want this type of content to become more popular than it already is.

Sorry, not sorry.