AI Basics AI Video Production Process - Planning
Hello
I will be leading this introductory course on creating videos with AI
and I am Shim Dongyup
To produce a video with AI
you need to go through several steps before you can complete the final output
Even though AI has improved enough to skip many parts of the process
there are still essential steps required when generating AI videos
So in this lecture, I’ll guide you through the essential workflow for AI video production
using theory, hands-on practice, and example cases
so you can create AI videos yourself
This is a beginner course
but we’re not just learning how to use tools
We’re training how to think creatively with AI
In our first session we’ll look at
the full AI video production process from planning to editing
and then practice how to use AI
specifically in the planning stage
I’ll explain everything from the basics
so feel free to follow along comfortably
AI Video Production Process
First the AI video production process
You may have seen ads or short films
with a note at the bottom saying
this video was created 100 percent with AI
For a while people were more surprised that those were
made 100% with AI than by its quality
but now what matters is
the final output itself and AI-generated videos are
reaching a very high level of quality
AI-generated video content can be divided into two types
100% AI and Hybrid AI
The difference is when everything from image generation
to the final video is made only with prompts
we call it 100 percent AI
When AI is used together with real footage
or images shot in advance
we call that hybrid AI
So when we make a video with 100% AI
What process do we follow?
We can break it down into six main steps
First is idea development
Second is text planning
Third is image generation
Fourth is video generation
Fifth is music and voice creation
Sixth is final editing and completion
Steps 1 and 2 are the planning stage or pre-production
Steps 3 to 5 are the production stage
Step 6 is the post-production stage
Now let’s look at each stage
First is the planning stage
This is the idea development stage
The first step is expanding your initial idea based on keywords
A video plan can start from a single word
Using an LLM based on the word
you can expand that word into a wide range of ideas
LLM stands for large language model
and refers to a massive deep learning model
trained on an enormous amount of data
The most well-known LLMs are OpenAI’s GPT
and Google’s Gemini
An LLM can generate many ideas
but you need to refine your prompts
to fit the content you want to create
so you can get results that match your final goal
By asking clear questions this way
you can shape the output and you can also customize
the planning ideas to match your own style
For example if you enter keywords like perfume commercial or sci-fi thriller
the LLM will suggest a variety of concepts
and storylines that fit the theme
And if you give more detailed conditions
like a floral daily perfume commercial
targeting women in their 20s
You’ll get results that fit your needs even better
And when you customize it like with GPTs
even if you only enter
something like a perfume commercial
it will automatically generate
a 30-second emotional ad idea
with a kitschy yet romantic mood
because it follows the style you preset
When used well, an LLM can
take you beyond simple ideas and give you
detailed and customized planning concepts
Second is text planning and prompt writing
If the first step gave you a broad set of
ideas for the video
The second step is where you define the purpose of the video
and write specific plans and prompts that match that purpose
Depending on whether it’s an ad a film a short-form clip or an educational video
the direction tone and structure of the plan will be completely different
And the prompts you create
for image generation will change accordingly
For example an ad is copy-focused a film is plot-focused
and a short-form clip needs strong impact in the first three seconds
So you plan the video based on these points
and then write prompts for image generation
So once the basic ideas come out in part one
Step two expands them into a concrete plan
and then moves on to writing prompts
that visualize that plan through image generation
When you ask the LLM to generate images
you can say something like
create Midjourney prompts for a cinematic 30-cut commercial
based on the attached ad concept
and it will generate the prompts you need
Once the initial idea is shaped into a concrete plan
and all the prompts for actual production are ready
We move on to the real production stage
The third step in the AI video process is image generation
This is where the visual tone and mood of the video are decided
For image generation you can use tools like
MidJourney, Whisk, Dreamina, and ImageFX
Each one has its own style and strengths
so it’s important to try different platforms to find the image style you want
When generating images it’s important to keep
the character style tone and manner and brand identity consistent throughout the whole video
To maintain this consistency
it’s best to stick to one image-generation platform
whenever possible
To keep the tone and manner
of the final video consistent
using shared prompt words like romantic cinematic or neon cyberpunk
is very helpful
for setting the overall mood
And if you want your own brand identity reflected in the video
it’s best to apply that from the very first image creation
To do that it helps to practice building your own
composition color schemes props and overall style
It’s also important to create a storyboard
Because it helps you check the overall consistency of the images
A storyboard lets you see the story visually at a glance
and you can add or remove parts that feel empty or excessive
making the content more solid
With a storyboard you can see the story visually at a glance
and it becomes much easier to add or remove anything that feels empty or excessive
so the final content becomes stronger
Next is video generation
Once all the images are ready
you use them to create the video clips
Now for video generation
There are many ways to create AI videos
and new methods keep coming out
but let’s look at the most commonly used types of AI video generation
First is T2V text-to-video
You describe the video you want through a prompt
and the system generates the video directly
You can generate it with natural language and in many different languages
These days people also use human-readable structured formats
like JSON or YAML quite often
Next is I2V image-to-video
This method uses an image
as the base to create the video
It’s usually used together with a prompt
and since you already have a reference image it produces more stable results
and makes it easier to match the style you want compared to T2V
where the outcome can be less predictable
However if you don’t know how to create the right reference image
this method can be difficult to use
Next is V2V video-to-video
This method uses an existing video or a generated video
to create a new one
Many people use restyle method
when they want to change an existing video
or when they want to change or remove
certain elements inside the video
Next is E2V elements-to-video
This method takes multiple elements
or images and combines them into a single video
You can make it play randomly with different clips
or you can use prompts to control the movements you want
It’s very useful for people who find it hard
to combine multiple separate elements into
one coherent image or scene
Now let’s look at video-generation platforms
There are already many platforms that support AI video creation
and new ones are continuing to appear
Some of the most talked about platforms right now are
Runway, Veo3 Kling, and Higgsfield
And because each of these platforms
has its own strengths and unique features
you need to understand and use those advantages well
to create a good video
Now let’s look at Veo3
It’s a video generation platform by Google
and if you add the audio and dialogue to the prompt
It can create a finished video with lip sync
But its generation rules are quite strict
and it doesn’t offer many extra features beyond video generation itself
Next is Kling
Kling offers a wide range of features from image generation to video editing
adding clips and applying various effects
and it is known for
producing very stable high-quality results
However its credits are relatively expensive
and it sometimes fails to
fully understand the prompt
Next is Runway
Runway is an early video platform that maintains shapes well
and has strong V2V features like Act 2 and RF
However it has the drawback that movements often don’t match what you want
Next is Higgsfield
Higgsfield creates realistic styles that work well for social media
and it can generate many VFX-style videos that were hard to make with prompts
using simple one-click tools
which makes it very effective for diverse video creation
However the overall video quality still leaves something to be desired
The third step is storyboard-based video generation
Creating each video clip
according to the storyboard
lets you check the story as you go
and it helps you understand the overall flow
when producing a story-driven video
It also makes it easier to see
which scene you need to create next
So working this way is very effective
The fourth step is creating cinematic direction
Using various VFX effects
or applying cinematic and advertising-style composition
and framing
or incorporating unique objects
can help create rich and distinctive visuals that are unique
to AI-generated videos
The fifth step is generating music and voice
Images play a big role in a video
but without sound effects, music, or voice
the overall quality drops significantly
So this is a crucial element you should never skip
In the past music and sound were
hard to access unless you were a professional
but now you can create
the music sound effects and dialogue you want through various platforms
One of the most effective platforms for creating BGM
is Suno
where you can easily generate music
by entering the lyrics and the mood or genre you want
and it also offers professional features
for detailed editing
There are also platforms like MMaudio
that generate sound effects automatically
based on the video you upload
BGM and sound effects
make the video feel much richer
So it’s important to remember that they are essential elements
Another important element is the voice used in videos with dialogue
The choice of voice tone intonation and emotion
shapes the overall tone and mood of the video
So it is a very important factor
In the past AI voices were easy to notice
but now platforms like ElevenLabs and Korean tools like Typecast
and Supertone
can create dialogue that sounds very realistic and expressive
The sixth step is the editing stage
After creating all the AI video clips through these steps
The final and most important stage is editing
How you edit can even change
the entire story of the video so it is an extremely important part of the process
First you upload all the video clips music and voice tracks
and sync them together
Then you arrange the cuts according to the storyboard
and adjust them as needed to build a stronger final video
Once the rough cut is created by placing all the videos
and audio together
You review the whole piece
and strengthen the story
by adding what’s missing and removing anything excessive
or unnecessary
There are many editing tools now including AI-based platforms
One of the most widely used is CapCut which is intuitive
and offers various video effects and audio options
making it a very convenient tool especially for beginners
Planning Stage Practice
Now we’ll take some time to apply
what we learned and create something in practice
Before we begin the planning practice
remember that there is no single correct platform
So it’s important to explore more tools and production methods
beyond the ones I introduce
Also since this explanation is given in a short time
please keep in mind that
we can’t cover every detail
Now let’s move from theory to practice
and create a short AI video together
We’ll go through each step while keeping
the six-step process in mind
So the first step is idea development
Let’s look at how to use GPT for this step
Every video begins with an idea
And a good idea starts with a good question
As we saw earlier
A clear and specific initial prompt is essential
So first we’ll practice
how to refine an idea by using step-by-step requests
Instead of asking for five synopses at once
it’s better to approach it step by step like this
First ask it to suggest core themes for the content you want to create
For example you can request five core themes
for a short film starring a cat
Once you run it you’ll see the suggested themes like this
Then simply choose the one you like the most from the list
Now let’s develop the first suggested theme
from the list
The next step is to ask for a more detailed explanation of the idea you selected
Like this, you can request
a detailed description of the main character the cat’s personality
and the owner’s traits based on the chosen idea
Then you’ll receive a detailed explanation like this
With this level of detail
you can start building your synopsis
Next you ask it to create a three-scene synopsis
based on the characters and story elements you developed above
When you run it you’ll see that it generates
a scenario made up of three scenes
From here you can refine each scene by adding details
such as camera movement, lighting, and other techniques to make it more complete
The next step is to translate the scenario
into visual language that Midjourney can understand
meaning into prompts
From this point the quality of the results can vary significantly
When writing a prompt the main focus
is deciding what you want it to draw
Here, we won’t generate every shot from each scene
We’ll create only the key image for each one
So I’ll give a command like this
Next, you choose three key shots that represent the scenario
and ask it to create Midjourney image prompts for them
Once you run the command
you’ll see prompts created for
the three representative cuts from the scenario
Of course if you want to show a more detailed story
you can adjust the number of cuts
in the command
and generate as many images as you need
Another important point is that
the style of the cuts is
not specified here
So the images may come out in very different tones
such as cartoon-like or fully realistic
To avoid this it’s better to provide more specific details in the prompt
If you choose three key shots that represent the scenario
and specify a broad style such as Japanese animation
then ask it to create the prompts
You’ll get images with consistent style
So it’s important to keep this in mind when you work on this step
When you follow this process you’ll see that
the prompt includes descriptions related to Japanese animation styles
and references to artists associated with that style
So far we’ve looked at the overall process of creating an AI video
and practiced
how AI can be used in the planning stage
As you can see, creating an AI video
requires many steps
and it’s not something that comes out with a single click
To use AI video effectively
it’s helpful to take advantage of what sets it apart from traditional video production
Using virtual models or virtual environments
that are difficult to film in real life is one approach
But as we learned today using AI in the planning stage
also helps expand your ideas
beyond what’s in your head
So as a creator it’s good to keep exploring and using AI in many different ways
I hope this session helped you do that and this concludes the lecture
Thank you
Summary AI Video Production Process
Types of AI Video Production 100% AI Production: A method where everything, from image generation to the final video, is created entirely through prompts
Hybrid AI Production: A method that uses real images or video sources along with AI to create the final output
Steps for 100% AI Production
1. Idea Development: Using an LLM based on keywords 2. Text Planning and Prompt Writing
3. Image Generation: Using tools like MidJourney Whisk Dreamina and ImageFX 4. Video Generation: Using platforms like Veo3 Kling Runway and Higgsfield
5. Music and Voice Generation: Using tools like Suno MMaudio ElevenLabs Typecast and Supertone 6. Editing
Planning Stage Practice Ideation Using GPT Refining Ideas Through a Step-by-Step Approach
Planning Steps Select the core theme Create a synopsis made of three scenes
Generate English prompts that Midjourney can understand Create key images for each scene
Prompt Guide Sample
Idea Development (Example) [For ChatGPT]
Create five synopsis ideas in Korean for a 30-second short film related to a cat with a dramatic and witty cinematic mood formatted as three 10-second scenes
Text Planning (Example) [For ChatGPT] The Cat’s Revenge
Scene 1 (10s): The owner secretly eats a late-night snack under the kitchen light while the cat glares from under the table.
Scene 2 (10s): The cat suddenly jumps knocks over a cup and the snack spills onto the floor. The owner panics.
Scene 3 (10s): The cat sits proudly on the spilled food and stares into the camera as if saying "it’s mine."
Create an English Midjourney prompt to generate the following content in a cute animation style